Visual impedance control method for automatic assembly of products and system and robot thereof

Abstract

This invention discloses a visual impedance control method, system, and robot for automated product assembly, belonging to the field of automated product assembly control technology. The control method includes: acquiring a set of pixel coordinates s of multiple feature points in a base coordinate system. t and the set of pixel coordinates s in the mechanical gripper coordinate system o And the interaction force h between the mechanical claw and the external environment o The desired speed v of the robotic gripper is determined based on the control law. d The angular acceleration 'a' is adjusted based on the PD algorithm to ensure that the actual motion state of the robotic gripper follows the desired motion state; this is achieved by adjusting the joint angle ξ, angular velocity 'a', angular acceleration 'a', and interaction force 'h'. o Determine the expected torque μ; update the joint angle ξ, angular velocity, and interaction force h of the manipulator based on the expected torque μ. o The actual position x and actual velocity v of the robotic gripper are then redefined. Through the above process, the motion state and interaction force of the robotic gripper are dynamically adjusted to ensure stable product gripping without compromising product performance.

Description

Technical Field

[0001] This invention belongs to the field of product automatic assembly control technology, and more specifically, relates to a visual impedance control method and system for automatic product assembly, and a robot. Background Technology

[0002] Product assembly typically requires manual force to grasp products in the production workshop and place them onto mounting devices along a suitable path to complete the installation. For a long time, assembly work was done manually by assembly workers. However, with the rapid development of industrial automation, more and more assembly work has been transferred to machines. While assembly machines can quickly complete predetermined assembly tasks, their control over the interaction force applied to the grasping product is not precise. If the interaction force is too large during movement, the product may be squeezed, deformed, or damaged; if the interaction force is too small, the product may fall, seriously affecting the assembly speed and quality of the entire automated assembly line. Currently, the interaction force applied by robotic grippers when moving products is generally fixed. However, in reality, the required interaction force during movement is related to the product's movement state. If the interaction force is not adjusted in a timely manner based on the movement state, or vice versa, the aforementioned problems can easily occur.

[0003] Therefore, it is necessary to design a suitable automated product assembly solution to reduce automated product assembly problems, improve the speed and quality of overall product assembly, and enhance the level of industrial automation. Summary of the Invention

[0004] To address the aforementioned deficiencies or improvement needs of existing technologies, this invention provides a visual impedance control method and system, as well as a robot, for automated product assembly. The purpose is to adjust the interaction force between the mechanical gripper and the product in real time during product grasping, maintaining stable product grasping without damaging product performance, thereby improving product assembly speed and quality.

[0005] To achieve the above objectives, according to one aspect of the present invention, a visual impedance control system for automated product assembly is provided, comprising:

[0006] The visual impedance control module is used to obtain the pixel coordinate set s of multiple feature points in the base coordinate system. t and the set of pixel coordinates s in the mechanical gripper coordinate system o And the interaction force h between the mechanical claw and the external environment o Based on control law Determine the desired speed v of the robotic gripper. d and expected acceleration In the formula, the feature point is located on the target component to be reached, the base coordinate system is defined on the target component, and the mechanical claw coordinate system is defined on the mechanical claw; Indicates the rate of change of image error. v c M represents the camera's motion speed. o D o K and J are symmetric positive matrices representing virtual mass, tilting, and stiffness, respectively; o J c Let k represent the Jacobian matrices of the mechanical gripper and the camera, respectively; s Indicates the gain coefficient;

[0007] The Cartesian motion control module is used to adjust the angular acceleration 'a' based on the PD algorithm so that the actual motion state of the robotic gripper follows the desired motion state, which includes the desired velocity 'v'. d and expected acceleration

[0008] Inverse kinematics module, used to calculate joint angle ξ and angular velocity Angular acceleration a and interaction force h o Determine the expected torque μ;

[0009] The environment interaction module is used to update the joint angle ξ and angular velocity of the robotic gripper based on the expected torque μ. and interaction force h o ;

[0010] The positive kinematics module is used to calculate the latest joint angles ξ and angular velocities of the manipulator. Determine the actual position x and actual velocity v of the robotic gripper.

[0011] In one embodiment, it further includes: a vision processing module, the vision processing module comprising:

[0012] Pixel coordinate unit: used to take pictures of the mechanical claw and the target part by the camera to obtain the origin o of the mechanical claw coordinate system, the origin t of the base coordinate system, and the pixel coordinates of each feature point in the same pixel coordinate system;

[0013] Pixel coordinate set determination unit: used to calculate the pixel coordinate set s of each feature point in the base coordinate system based on the origin of the base coordinate system and the pixel coordinates of each feature point in the pixel coordinate system. t =[s t.1 ,s t.2 ,……,s t.n ] T And based on the origin of the mechanical gripper coordinate system and the pixel coordinates of each feature point in the pixel coordinate system, the set of pixel coordinates s of each feature point in the mechanical gripper coordinate system is calculated. o =[s o.1 ,s o.2 ,……,s o.n ]T wherein s t.i represents the pixel coordinate of the i-th feature point in the base coordinate system, s o.i represents the pixel coordinate of the i-th feature point in the mechanical gripper coordinate system, and n is the number of feature points,

[0014] s t.i = (x i , y i ) - (x t , y t )

[0015] s o.i = (x i , y i ) - (x o , y o )

[0016] wherein (x i , y i ) represents the pixel coordinate of the i-th feature point in the pixel coordinate system, (x t , y t ) represents the pixel coordinate of the origin of the base coordinate system in the pixel coordinate system, and (x o , y o ) represents the pixel coordinate of the origin of the mechanical gripper coordinate system in the pixel coordinate system.

[0017] In one embodiment, the visual processing module further comprises:

[0018] a coordinate conversion unit configured to convert the pixel coordinate into a coordinate in a world coordinate system based on a coordinate conversion model, wherein the coordinate conversion model is:

[0019]

[0020] wherein z c represents the distance of the camera optical center in the z direction of the camera coordinate system, u and v represent the pixel coordinate in the pixel coordinate system, dx and dy represent the width and length of the pixel, u0 and v0 represent the horizontal coordinate and vertical coordinate of the origin of the image coordinate system in the pixel coordinate system, respectively, f represents the focal length, R and t represent the rotation matrix and translation parameter of the homogeneous coordinate transformation, respectively, and X, Y and Z represent the coordinate in the world coordinate system.

[0021] In one embodiment, the Cartesian motion control module is configured to obtain the desired position x d , the desired velocity v d and the desired acceleration of the mechanical gripper, and to adjust the actual acceleration of the mechanical gripper according to a PD algorithm to make the actual position x and the actual velocity v approach the desired position x d , and the desired velocity v d , and then determine the angular acceleration a according to the actual acceleration , wherein K P is a proportional coefficient, K D is a differential coefficient.

[0022] In one of the embodiments, the number of the feature points is greater than or equal to 3, so that the mechanical gripper coordinate system coincides with the base coordinate system when the origin of the mechanical gripper coordinate system reaches the origin of the base coordinate system.

[0023] In one of the embodiments, the mechanical gripper is provided with a clamp, and the mechanical gripper clamps the component to be assembled and moves to the target component for assembly through the clamp.

[0024] In one of the embodiments, the product is a 3C product.

[0025] According to another aspect of the present application, a visual impedance control method for automatic assembly of products is provided, comprising:

[0026] Step S1: obtaining a pixel coordinate set s t of a plurality of feature points in a base coordinate system, a pixel coordinate set s o of the plurality of feature points in a mechanical gripper coordinate system, and an interaction force h o of the mechanical gripper with the outside world, determining a desired velocity v d and a desired acceleration of the mechanical gripper based on a control law , wherein the feature points are on a target component to be reached, the base coordinate system is defined on the target component, and the mechanical gripper coordinate system is defined on the mechanical gripper; denoting a variation speed of image error, v c is a camera motion speed, M o , D o and K are symmetric positive matrices of virtual mass, dumping and stiffness respectively; J o , J c denote Jacobian matrices of the mechanical gripper and the camera respectively; k s denotes a gain coefficient;

[0027] Step S2: adjusting the angular acceleration a based on a PD algorithm to make the actual motion state of the mechanical gripper follow the desired motion state, wherein the desired motion state comprises the desired velocity v d and the desired acceleration

[0028] Step S3: determining the joint angle ξ, the angular velocity the angular acceleration a and the interaction force ho determining an expected torque μ;

[0029] Step S4: updating the joint angle ξ, angular velocity and interaction force h o of the mechanical gripper based on the expected torque μ;

[0030] Step S5: determining the actual position x and actual velocity v of the mechanical gripper based on the latest joint angle ξ and angular velocity of the mechanical gripper;

[0031] Step S6: determining whether the base coordinate system and the mechanical gripper coordinate system coincide, if yes, ending the movement, otherwise, jumping to Step S1.

[0032] In one embodiment, the step S1 includes: t acquiring a pixel coordinate set s o of each feature point in the base coordinate system and a pixel coordinate set s

[0033] taking a picture of the mechanical gripper and the target component through a camera to acquire pixel coordinates of the origin of the mechanical gripper coordinate system, the origin of the base coordinate system and each feature point in the same pixel coordinate system;

[0034] calculating the pixel coordinate set s t of each feature point in the base coordinate system based on the origin of the base coordinate system and the coordinates of each feature point in the pixel coordinate system, and calculating the pixel coordinate set s o of each feature point in the mechanical gripper coordinate system based on the origin of the mechanical gripper coordinate system and the coordinates of each feature point in the pixel coordinate system.

[0035] A product automatic assembly robot is provided, which includes a robot body, a mechanical gripper at the end of the robot body and a visual impedance control system for the product automatic assembly.

[0036] Overall, compared with the prior art, the above technical solutions conceived by the present application can achieve the following beneficial effects:

[0037] The motion state and the interaction force of the mechanical gripper moving product are dynamically adjusted, and the adjustment of the two is interrelated. First, based on a specific control law formula, the expected speed and the expected acceleration of the mechanical gripper in the next time sequence are determined according to the current interaction force state of the mechanical gripper, the current moving speed of the camera, and the current distance from the target component. In this process, the determination of the expected speed and the expected acceleration not only considers the distance and the moving speed of the associated camera, but also considers the current interaction force. The speed and the acceleration are adjusted within the action range of the current interaction force to avoid inappropriate motion state adjustment and to ensure that the product is stably held or deformed. After the expected speed and the expected acceleration in the next time sequence are determined, the actual angular acceleration of the mechanical gripper is adjusted based on the PD algorithm to ensure that the actual motion state of the mechanical gripper follows the expected motion state. Then, the expected torque is determined based on the adjustment amount of the actual angular acceleration, the actual motion state of the mechanical gripper, and the interaction force, and the motion state and the interaction force of the mechanical gripper are adjusted based on the expected torque. In this process, since the torque of the mechanical gripper changes, the interaction force also changes accordingly. Finally, the updated actual position x and the actual speed v of the mechanical gripper are determined based on the motion state of the mechanical gripper, and the next time sequence control is entered again. Through the above process, the motion state and the interaction force of the mechanical gripper moving product are dynamically adjusted to ensure that the product is stably gripped and the performance of the product is not damaged, and the product assembly speed and quality are improved. BRIEF DESCRIPTION OF DRAWINGS

[0038] Figure 1 is a structural block diagram of a visual impedance control system for product automatic assembly in an embodiment;

[0039] Figure 2 is a step flow chart of a visual impedance control method for product automatic assembly in an embodiment;

[0040] Figure 3 is a structural schematic diagram of a product automatic assembly robot in an embodiment. DETAILED DESCRIPTION

[0041] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application. In addition, the technical features involved in each embodiment of the present application described below can be combined with each other as long as they do not conflict with each other.

[0042] As shown in Figure 1 , in an embodiment, the visual impedance control system for product automatic assembly includes a visual impedance control module, a Cartesian motion control module, an inverse kinematics module, an environment interaction module, and a forward kinematics module. The following describes each module in detail.

[0043] The visual impedance control module is used to obtain a pixel coordinate set s of the plurality of feature points in the base coordinate system t and a pixel coordinate set s in the mechanical gripper coordinate system o and an interaction force h between the mechanical gripper and the external environment o , based on a control law to determine a desired velocity v of the mechanical gripper d and a desired acceleration

[0044] wherein the feature points are on a target component to be reached. The component gripped by the mechanical gripper is defined as a component to be assembled, and the component waiting at a distance from the component to be assembled is defined as a target component. The present application needs to automatically move the component to be assembled to the target component by the mechanical gripper for assembly. The target component and the component to be assembled are indefinite in form. For example, the target component can be a part of a product, and the component to be assembled can be another part of the product matched with the target component; or the target component is a mounting device, and the component to be assembled is a product. The feature points are a plurality of feature points selected on the target component.

[0045] The base coordinate system is defined on the target component, and the target component is fixed during assembly. Therefore, the base coordinate system is also fixed after being established. The mechanical gripper coordinate system is defined on the mechanical gripper. Since the mechanical gripper is moving, the mechanical gripper coordinate system is also moving with the mechanical gripper, but is stationary relative to the mechanical gripper.

[0046] After the base coordinate system and the mechanical gripper coordinate system are defined, the pixel coordinates of each feature point in the two coordinate systems are calculated respectively to form a pixel coordinate set s of the plurality of feature points in the base coordinate system t and a pixel coordinate set s in the mechanical gripper coordinate system o .

[0047] In an embodiment, the number of feature points is greater than or equal to 3 so that when the origin of the mechanical gripper coordinate system reaches the origin of the base coordinate system, the mechanical gripper coordinate system coincides with the base coordinate system. It can be understood that product assembly requires strict alignment. By making the mechanical gripper coordinate system coincide with the base coordinate system, it can be ensured that the gripped product to be assembled and the target product are aligned as required for smooth assembly action.

[0048] In an embodiment, the pixel coordinate set s of the plurality of feature points in the base coordinate system t and the pixel coordinate set s in the mechanical gripper coordinate system o are determined by a visual processing module, and the visual processing module includes a pixel coordinate unit and a pixel coordinate set determination unit.

[0049] Specifically, the pixel coordinate unit is configured to take pictures of the mechanical gripper and the target component by the camera to obtain pixel coordinates of the mechanical gripper coordinate system origin, the base coordinate system origin, and each feature point in the same pixel coordinate system.

[0050] The mechanical gripper coordinate system origin is defined as point o, (x o ,y o ) represents the pixel coordinates of the mechanical gripper coordinate system origin, the origin of the base coordinate system is point t, (x t ,y t ) represents the pixel coordinates of the base coordinate system origin, and the pixel coordinates of the i-th feature point in the pixel coordinate system are (x i ,y i ).

[0051] The pixel coordinate set determination unit is configured to calculate the pixel coordinate set s t of each feature point in the base coordinate system based on the base coordinate system origin and the pixel coordinates of each feature point in the pixel coordinate system, and calculate the pixel coordinate set s o of each feature point in the mechanical gripper coordinate system based on the mechanical gripper coordinate system origin and the coordinates of each feature point in the pixel coordinate system.

[0052] Take four feature points as an example for illustration.

[0053] The pixel coordinate set s t of the four feature points in the base coordinate system is s t.1 , s t.2 , s t.3 , s t.4 T , and the pixel coordinate set s o of the four feature points in the mechanical gripper coordinate system is s o.1 , s o.2 , s o.3 , s o.4 T .

[0054] The pixel coordinates of the i-th feature point in the base coordinate system are:

[0055] s t.i = (x i ,y i ) - (x t ,y t )

[0056] The pixel coordinates of the i-th feature point in the mechanical gripper coordinate system are:

[0057] s o.i = (x i ,y i ) - (x o ,y o )​​

[0058] At this time, the image error is:

[0059] e = s o -s t

[0060] The image error represents the deviation between the mechanical claw coordinate system and the base coordinate system, and the target of the mechanical claw movement is to make the mechanical claw coordinate system coincide with the base coordinate system so as to align the to-be-assembled part with the target part and assemble them.

[0061] The image error change rate is:

[0062]

[0063] In the formula, represents the change rate of each coordinate position in the pixel coordinate set s o represents the change rate of each coordinate position in the pixel coordinate set s t .

[0064] In calculation, the image error change rate can be deformed as:

[0065]

[0066] In the formula, J0, J c are the Jacobian matrices of the mechanical claw and the camera respectively, J t represents the change relationship of v c and , v c is the motion speed of the camera.

[0067] Then, based on the current image error change rate , the current motion speed v c of the camera, the current interaction force h o , the expected speed v d and the expected acceleration of the mechanical claw are determined. The control law formula is:

[0068]

[0069] Therefore, in the visual impedance control module, the relationship between the mechanical claw motion speed, the mechanical claw motion acceleration, the image error change rate, the camera motion speed and the interaction force is comprehensively considered, the mechanical claw can be accurately controlled, and the product can be stably grabbed without damaging the performance of the product.

[0070] ​In an embodiment, since the coordinate data in the world coordinate system is also needed when the data processing is performed, the visual processing module further comprises a coordinate conversion unit configured to convert the pixel coordinates into the coordinates in the world coordinate system based on a coordinate conversion model.

[0071] Specifically, the coordinate conversion model is:

[0072]

[0073] wherein z c represents the distance of the camera optical center in the z direction of the camera coordinate system, x and y represent the pixel coordinates in the pixel coordinate system, dx and dy represent the width and length of the pixel, u d and v d respectively represent the horizontal and vertical coordinates of the image coordinate system origin in the pixel coordinate system, f represents the focal length, R and t respectively represent the rotation matrix and translation parameter of the homogeneous coordinate transformation, and X, Y and Z represent the coordinates in the world coordinate system.

[0074] For example, the pixel coordinates (x o , y o ) of the mechanical gripper coordinate system origin can be converted into the coordinates (X o , Y o , Z o ) in the world coordinate system, the pixel coordinates (x t , y t ) of the base coordinate system origin can be converted into the coordinates (X t , Y t , Z t ) in the world coordinate system, and the pixel coordinates (x i , y i ) of the i-th feature point in the pixel coordinate system can be converted into the coordinates (X i , Y i , Z i ) in the world coordinate system.

[0075] In an embodiment, since the base coordinate system is fixed, the base coordinate system can be directly selected as the world coordinate system.

[0076] In an embodiment, considering the decoupling requirement, M o , D o and K are set as diagonal matrices when used in practice.

[0077] In an embodiment, the derivation process of the Jacobian matrix J t is as follows:

[0078]

[0079]

[0080] J t =[J t,1 ,J t,2 ,J t,3 ,……,J t,n ]

[0081] where n is the number of feature points, represents the pixel coordinate change speed of the i-th feature point in the base coordinate system, is the z-direction projection component of the i-th feature point in the base coordinate system relative to the camera coordinate system, X t,i and Y t,i are the horizontal and vertical coordinates of the i-th feature point in the base coordinate system, X t,i = X i - X t , Y t,i = Y i - Y t , R c is the rotation change matrix of the camera coordinate system relative to the world coordinate system, v c is the camera motion speed, J t,i establishes the change relationship between and v c .

[0082] In an embodiment, the derivation process of the mechanical gripper Jacobian matrix J o is as follows:

[0083]

[0084]

[0085] J o = [J o,1 , J o,2 , J o,3 , …, J o,n ]

[0086] where, represents the pixel coordinate change speed of the i-th feature point in the mechanical gripper coordinate system, X o,i and Y o,i are the horizontal and vertical coordinates of the i-th feature point in the mechanical gripper coordinate system, X o,i = X i - X o , Y t,o = Y i - Y o , I3 represents a 3-order unit vector, S() is an operation for writing a column vector into a skew-symmetric matrix, represents the rotation matrix of the mechanical gripper coordinate system relative to the camera coordinate system, v is the coordinate of the i-th feature point in the mechanical gripper coordinate system. o J represents the velocity of the robotic gripper relative to the world coordinate system. o,i Establish and v o v c The changing relationship.

[0087] Camera Jacobian Matrix J c =J o +J t .

[0088] Understandably, the above Jacobian matrices can be derived from the modeling relationships.

[0089] In one embodiment, k s This represents the gain coefficient, which is usually set to 1.

[0090] The Cartesian motion control module is used to adjust the angular acceleration 'a' based on the PD algorithm so that the actual motion state of the robotic gripper follows the desired motion state, which includes the desired velocity 'v'. d and expected acceleration

[0091] Specifically, obtain the desired position x of the robotic gripper. d Expected speed v d and expected acceleration Actual position x and actual velocity v, according to the PD algorithm Adjusting the actual acceleration of the mechanical gripper To make the actual position x and the actual velocity v approach the desired position x d Expected speed v d Then based on the actual acceleration Determine the angular acceleration a; where K P It is the proportionality coefficient, K D These are the differential coefficients.

[0092] Understandably, the desired velocity v can be calculated using the designed visual impedance control method. d and expected acceleration For the desired velocity v d The desired position x can be obtained by performing time integration. d .

[0093] The Cartesian motion control module is used to make the actual motion state of the robotic gripper follow the desired motion state. The designed PD algorithm is as follows:

[0094]

[0095] Since position, velocity and acceleration are interrelated, the above-mentioned PD algorithm is used for adjustment, when the actual velocity v and position x are less than the expected values, the adjusted acceleration is higher than the expected acceleration, so as to increase the values of the actual velocity v and position x, and make them quickly follow the expected values, on the contrary, when the actual velocity v and position x are greater than the expected values, the adjusted acceleration is lower than the expected acceleration, so as to decrease the values of the actual velocity v and position x, and make them quickly approach the expected values.

[0096] Specifically, the conversion formula of the acceleration and the angular acceleration a is as follows:

[0097]

[0098] In the formula, ω is the angular velocity, is the matrix composed of the derivatives of the items of J(ξ) matrix with respect to time.

[0099] The inverse kinematics module is used for determining the expected torque μ according to the joint angle ξ, the angular velocity and the interaction force h o .

[0100] The environment interaction module is used for updating the joint angle ξ, the angular velocity and the interaction force h o of the mechanical gripper based on the expected torque μ.

[0101] The forward kinematics module is used for determining the actual position x and the actual velocity v of the mechanical gripper based on the latest joint angle ξ and the angular velocity of the mechanical gripper.

[0102] The inverse kinematics module, the environment interaction module and the forward kinematics module can be implemented according to conventional technologies, and thus will not be described herein.

[0103] In an embodiment, the above-mentioned assembled product can be a 3C product, but is not limited thereto.

[0104] In an embodiment, the above-mentioned mechanical gripper is provided with a clamp, and the mechanical gripper clamps the component to be assembled by the clamp and moves to the target component to perform the assembly.

[0105] Correspondingly, the present application also relates to a visual impedance control method for automatic assembly of a product, as shown in Figure 2 , the main steps of which are as follows:

[0106] Step S1: obtaining a pixel coordinate set s t of a plurality of feature points in a base coordinate system and a pixel coordinate set s o of the plurality of feature points in a mechanical gripper coordinate system and an interaction force h of the mechanical gripper with the outside world.​​​o , based on the control law determining the desired speed v of the mechanical gripper d and the desired acceleration

[0107] wherein the feature points are on a target component to be reached, the base coordinate system is defined on the target component, and the mechanical gripper coordinate system is defined on the mechanical gripper; denotes the image error variation speed, v c is the camera motion speed, M o , D o and K are respectively the symmetric positive matrix of virtual mass, dumping and stiffness; J o , J c denote the Jacobian matrix of the mechanical gripper and the camera respectively; k s denotes the gain coefficient.

[0108] Step S2: adjusting the angular acceleration a based on the PD algorithm to make the actual motion state of the mechanical gripper follow the desired motion state, the desired motion state including the desired speed v d and the desired acceleration

[0109] Step S3: determining the expected torque μ according to the joint angle ξ, the angular speed the angular acceleration a and the interaction force h o .

[0110] Step S4: updating the joint angle ξ, the angular speed and the interaction force h o of the mechanical gripper based on the expected torque μ.

[0111] Step S5: determining the actual position x and the actual speed v of the mechanical gripper based on the latest joint angle ξ and the angular speed of the mechanical gripper.

[0112] Step S6: judging whether the base coordinate system coincides with the mechanical gripper coordinate system, if yes, ending the movement, otherwise, jumping to Step S1.

[0113] Specifically, the process of acquiring the pixel coordinate set s t of the plurality of feature points in the base coordinate system and the pixel coordinate set s o of the plurality of feature points in the mechanical gripper coordinate system in Step S1 includes:

[0114] Step S11: taking a photo of the mechanical gripper and the target component by the camera to acquire the pixel coordinates of the origin of the mechanical gripper coordinate system, the origin of the base coordinate system and each feature point in the same pixel coordinate system.

[0115] Step S12: calculating the pixel coordinate set s of each feature point in the base coordinate system based on the base coordinate system origin and the coordinates of each feature point in the pixel coordinate system t , and calculating the pixel coordinate set s of each feature point in the mechanical claw coordinate system based on the mechanical claw coordinate system origin and the coordinates of each feature point in the pixel coordinate system o .

[0116] The specific implementation process of each step can be referred to the above description, and will not be repeated here.

[0117] Correspondingly, the application also relates to a product automatic assembly robot, as shown in the figure, comprising a robot body 1, a mechanical claw 2 at the end of the robot body, and the product automatic assembly visual impedance control system (not shown in the figure) described above, which is used for controlling the mechanical claw to hold a component to be assembled and move to a target component for assembly. Figure 3

[0118] Those skilled in the art can easily understand that the above is only a preferred embodiment of the application, and is not used to limit the application, and any modification, equivalent replacement and improvement within the spirit and principle of the application should be included in the protection scope of the application.​

Claims

1. A visual impedance control system for automated product assembly, characterized in that, include: The visual impedance control module is used to obtain the pixel coordinate set of multiple feature points in the base coordinate system. and the set of pixel coordinates in the robotic gripper coordinate system And the interaction force between the mechanical claw and the external environment. Based on control law Determine the desired speed of the robotic gripper. and expected acceleration In the formula, the feature point is located on the target component to be reached, the base coordinate system is defined on the target component and the base coordinate system is the world coordinate system, and the mechanical claw coordinate system is defined on the mechanical claw. Indicates the rate of change of image error. , Represents pixel coordinate set The rate of change of each coordinate position in the graph Represents pixel coordinate set The rate of change of each coordinate position in the graph For camera movement speed, , and These are symmetric positive matrices representing virtual mass, tilt, and stiffness, respectively. , These represent the Jacobian matrices of the robotic gripper and the camera, respectively. Indicates the gain coefficient; The Cartesian motion control module is used to adjust angular acceleration based on the PD algorithm. To ensure that the actual motion state of the robotic gripper follows the desired motion state, which includes the desired velocity. and expected acceleration Specifically, this includes: obtaining the desired position of the mechanical gripper. Expected speed and expected acceleration Actual location and actual speed According to the PD algorithm Adjusting the actual acceleration of the mechanical gripper to make the actual position and actual speed Approaching the desired position Expected speed Then, based on the actual acceleration... Determine angular acceleration In the formula, It is a proportionality coefficient. These are differential coefficients; Inverse kinematics module, used to determine joint angles angular velocity angular acceleration and interaction forces Determine the expected torque ; The environment interaction module is used to interact with the environment based on the expected torque. Update the joint angle of the robotic gripper angular velocity and interaction forces ; Positive kinematics module, used for the latest joint angles of the robotic gripper and angular velocity Determine the actual position of the robotic gripper and actual speed ; A vision processing module, comprising: Pixel coordinate unit: used to take pictures of the robotic gripper and the target part using a camera to obtain the origin of the robotic gripper's coordinate system. Origin of the base coordinate system The pixel coordinates of each feature point in the same pixel coordinate system; Pixel coordinate set determination unit: used to calculate the pixel coordinate set of each feature point in the base coordinate system based on the origin of the base coordinate system and the pixel coordinates of each feature point in the pixel coordinate system. And based on the origin of the mechanical gripper coordinate system and the pixel coordinates of each feature point in the pixel coordinate system, the set of pixel coordinates of each feature point in the mechanical gripper coordinate system is calculated. ,in, Indicates the first The pixel coordinates of the feature point in the base coordinate system. Indicates the first The pixel coordinates of the feature point in the robotic gripper coordinate system. The number of feature points, In the formula, Indicates the first The pixel coordinates of the feature point in the pixel coordinate system This represents the pixel coordinates of the origin of the base coordinate system in the pixel coordinate system. This represents the pixel coordinates of the origin of the robotic gripper's coordinate system in the pixel coordinate system. The visual processing module also includes: The coordinate transformation unit is used to convert pixel coordinates into coordinates in the world coordinate system based on a coordinate transformation model, wherein the coordinate transformation model is: in, This represents the distance of the camera's optical center in the z-direction of the camera coordinate system, u and v represent the pixel coordinates in the pixel coordinate system, and dx and dy represent the width and length of the pixel. and represents the x-coordinate and y-coordinate of the origin of the image coordinate system in the pixel coordinate system, respectively; f represents the focal length; R and t represent the rotation matrix and translation parameters of the homogeneous coordinate transformation, respectively; and X, Y, and Z represent the coordinates in the world coordinate system.

2. The visual impedance control system for automated product assembly as described in claim 1, characterized in that, The number of feature points is greater than or equal to 3, so that when the origin of the mechanical claw coordinate system reaches the origin of the base coordinate system, the mechanical claw coordinate system coincides with the base coordinate system.

3. The visual impedance control system for automated product assembly as described in claim 1, characterized in that, The mechanical gripper is equipped with a clamp, which holds the component to be assembled and moves it to the target component for assembly.

4. The visual impedance control system for automated product assembly as described in claim 1, characterized in that, The product in question is a 3C product.

5. A visual impedance control method for automated product assembly based on the visual impedance control system as described in any one of claims 1 to 4, characterized in that, include: Step S1: Obtain the pixel coordinate set of multiple feature points in the base coordinate system and the set of pixel coordinates in the robotic gripper coordinate system And the interaction force between the mechanical claw and the external environment. Based on control law Determine the desired speed of the robotic gripper. and expected acceleration In the formula, the feature point is located on the target component to be reached, the base coordinate system is defined on the target component and the base coordinate system is the world coordinate system, and the mechanical claw coordinate system is defined on the mechanical claw. Indicates the rate of change of image error. , Represents pixel coordinate set The rate of change of each coordinate position in the graph Represents pixel coordinate set The rate of change of each coordinate position in the graph For camera movement speed, , and These are symmetric positive matrices representing virtual mass, tilt, and stiffness, respectively. , These represent the Jacobian matrices of the robotic gripper and the camera, respectively. Indicates the gain coefficient; Step S2: Adjust angular acceleration based on PD algorithm To ensure that the actual motion state of the robotic gripper follows the desired motion state, which includes the desired velocity. and expected acceleration ; Step S3: Based on joint angles angular velocity angular acceleration and interaction forces Determine the expected torque ; Step S4: Based on the expected torque Update the joint angle of the robotic gripper angular velocity and interaction forces ; Step S5: Based on the latest joint angle of the robotic gripper and angular velocity Determine the actual position of the robotic gripper and actual speed ; Step S6: Determine whether the base coordinate system and the mechanical claw coordinate system coincide. If yes, end the movement; otherwise, go to step S1. In step S1, the pixel coordinate set of multiple feature points in the base coordinate system is obtained. and the set of pixel coordinates in the robotic gripper coordinate system ,include: The origin of the robotic gripper's coordinate system is obtained by taking pictures of the robotic gripper and the target component using a camera. Origin of the base coordinate system The pixel coordinates of each feature point in the same pixel coordinate system; Calculate the pixel coordinate set of each feature point in the base coordinate system based on the origin of the base coordinate system and the coordinates of each feature point in the pixel coordinate system. And based on the origin of the robotic gripper coordinate system and the coordinates of each feature point in the pixel coordinate system, the pixel coordinate set of each feature point in the robotic gripper coordinate system is calculated. ,in, Indicates the first The pixel coordinates of the feature point in the base coordinate system. Indicates the first The pixel coordinates of the feature point in the robotic gripper coordinate system. The number of feature points, In the formula, Indicates the first The pixel coordinates of the feature point in the pixel coordinate system This represents the pixel coordinates of the origin of the base coordinate system in the pixel coordinate system. This represents the pixel coordinates of the origin of the mechanical gripper's coordinate system in the pixel coordinate system.

6. An automated product assembly robot, characterized in that, The invention includes a robot body, a mechanical gripper located at the end of the robot body, and a visual impedance control system for automatic assembly of the product according to any one of claims 1 to 4, wherein the visual impedance control system is used to control the mechanical gripper to hold the part to be assembled and move it to the target part for assembly.

Images