Dynamic assembly method, apparatus and system

Abstract

The application provides a dynamic assembly method, device and system, comprising the following steps: S1, positioning a reference point on a CAD model of an assembly object according to an assembly operation of dynamic assembly, and matching a mechanical arm and a camera according to the assembly operation of dynamic assembly and an operation space thereof; S2, synchronizing an AGV and the clamping device, and synchronizing a mechanical arm end and an assembly workpiece on the assembly object with reference to the reference point; S3, correcting position error and angle error of the reference point and the assembly workpiece through a proportional integral differential algorithm to execute synchronization of the mechanical arm end and the assembly workpiece by visual tracking of the reference point; and S4, performing Kalman filter multi-sensor fusion based on visual tracking of the reference point and adjustment of adaptive force control to calculate a speed and a position of the mechanical arm end to execute an assembly operation on the assembly workpiece. The application is particularly suitable for automobile final assembly and has low and accurate cost.

Description

Technical Field

[0001] This invention relates to the field of robot control, and more particularly to a dynamic assembly method, apparatus, and system. Background Technology

[0002] With the development of robotics technology, more and more robotic workshops are being used in automobile production. Automobile production can be divided into four main workshops: pressing, welding, painting, and assembly. Robots are already widely used in the pressing, welding, painting, and assembly processes.

[0003] At the final assembly station, most tasks are more flexible and complex compared to the other three stations. One of the complex tasks is assembling automotive parts onto the vehicle body, where the body is moving. This type of task is typically done manually. Workers stand on AGVs to perform the assembly work, which is difficult and strenuous for manual labor. During assembly, engineers must look outwards to observe obstacles and face the bottom of the vehicle body, which is very strenuous. Furthermore, assembly requires accuracy and responsiveness to changing environments. To increase throughput, AGVs move along mobile clamping devices. Summary of the Invention

[0004] The first aspect of the present invention provides a dynamic assembly method, wherein the dynamic assembly method is performed by a robot placed on an AGV (Automated Guided Vehicle), the robot having a robotic arm equipped with an assembly tool, and the assembly object being dynamically assembled being fixed by a clamping tool, comprising the following steps: S1, locating a reference point on the CAD model of the assembly object according to the assembly operation of the dynamic assembly, and matching the robotic arm and a camera according to the assembly operation and its operating space; S2, synchronizing the AGV and the clamping device, and synchronizing the end effector of the robotic arm and the assembly workpiece on the assembly object with reference to the reference point; S3, visually tracking the reference point, and correcting the positional and angular errors of the reference point and the assembly workpiece using a proportional-integral-differential algorithm, thereby synchronizing the end effector of the robotic arm and the assembly workpiece; S4, performing Kalman filter multi-sensor fusion based on visual tracking of the reference point and adaptive force control adjustment, thereby calculating the speed and position of the end effector of the robotic arm to perform the assembly operation on the assembly workpiece.

[0005] Furthermore, the assembly tools mounted on the robotic arm are screwdrivers and electric drills, and the assembly operation is a screw-tightening operation.

[0006] Furthermore, when it is impossible to locate a reference point on the CAD model of the assembly object based on the assembly operation of the dynamic assembly, the dynamic assembly method further includes the following step: setting a laser projector on the AGV or the robot to project a laser reference point onto the assembly object using a laser.

[0007] Furthermore, the AGV and the clamping device perform synchronization based on magnetic navigation, QR code indication, or laser navigation.

[0008] Furthermore, the positional deviation is:

[0009] e x,y =x 0,xy -x x,y ,

[0010] Where, x 0,xy x represents the coordinate position of the assembled workpiece. x,y The position of the assembled workpiece is calculated based on visual recognition of the relative position of the reference point and the assembled workpiece.

[0011] Furthermore, the angular deviation is:

[0012] e rz =x 0,rz -x rz ,

[0013] Where, x 0,rz For the angle of the assembled workpiece, x rz The angle of the assembled workpiece is calculated based on the visual recognition of the relative angle between the reference point and the assembled workpiece.

[0014] Furthermore, the speed of the robotic arm's end effector is:

[0015]

[0016]

[0017] The rotational angular velocity of the robotic arm along the Z-axis is:

[0018]

[0019] Among them, the Mat rotation Let K be the rotation matrix of the current state variables. p K represents the proportional coefficient in the proportional-integral-differential algorithm. i is the integral coefficient.

[0020] Furthermore, a torque sensor is installed on the robotic arm, and the external force obtained by the torque sensor is:

[0021]

[0022] Among them, M d D d and K d This is the preset amount.

[0023] Furthermore, the Kalman filter multi-sensor fusion is as follows:

[0024]

[0025]

[0026] Where x is the target position, and i is the index of different sensors, x i Let A(k) represent the estimate from the vision or force control sensor, and k be the Kalman gain. Here, A(k) and C(i,k+1) are both identity matrices.

[0027] A second aspect of the present invention provides a dynamic assembly system, the dynamic assembly system comprising an AGV (Automated Guided Vehicle) and a robot placed on the AGV, the robot having a robotic arm equipped with an assembly tool, the assembly object being dynamically assembled being fixed by a clamping tool, wherein the system includes: a processor; and a memory coupled to the processor, the memory having instructions stored therein, the instructions causing the electronic device to perform actions when executed by the processor, the actions including: S1, locating a reference point on a CAD model of the assembly object according to the assembly operation of the dynamic assembly, and according to the dynamic assembly... S1. The assembly operation and its operating space are matched with the robotic arm and camera; S2. The AGV and the clamping device are synchronized, and the end effector of the robotic arm and the assembly workpiece on the assembly object are synchronized with reference to the reference point; S3. The reference point is tracked visually, and the position and angle errors of the reference point and the assembly workpiece are corrected by proportional-integral-differential algorithm, so as to perform synchronization between the end effector of the robotic arm and the assembly workpiece; S4. Based on the visual tracking of the reference point and the adjustment of adaptive force control, Kalman filter multi-sensor fusion is performed to calculate the speed and position of the end effector of the robotic arm to perform the assembly operation on the assembly workpiece.

[0028] Furthermore, the assembly tools mounted on the robotic arm are screwdrivers and electric drills, and the assembly operation is a screw-tightening operation.

[0029] Furthermore, when it is impossible to locate a reference point on the CAD model of the assembly object based on the assembly operation of dynamic assembly, the action also includes: setting a laser projector on the AGV or the robot to project a laser reference point onto the assembly object using a laser.

[0030] Furthermore, the AGV and the clamping device perform synchronization based on magnetic navigation, QR code indication, or laser navigation.

[0031] Furthermore, the positional deviation is:

[0032] e x,y =x 0,xy -xx,y ,

[0033] Where, x 0,xy x represents the coordinate position of the assembled workpiece. x,y The position of the assembled workpiece is calculated based on visual recognition of the relative position of the reference point and the assembled workpiece.

[0034] Furthermore, the angular deviation is:

[0035] e rz =x 0,rz -x rz ,

[0036] Where, x 0,rz For the angle of the assembled workpiece, x rz The angle of the assembled workpiece is calculated based on the visual recognition of the relative angle between the reference point and the assembled workpiece.

[0037] Furthermore, the speed of the robotic arm's end effector is:

[0038]

[0039]

[0040] The rotational angular velocity of the robotic arm along the Z-axis is:

[0041]

[0042] Among them, the Mat rotation Let K be the rotation matrix of the current state variables. p K represents the proportional coefficient in the proportional-integral-differential algorithm. i is the integral coefficient.

[0043] Furthermore, a torque sensor is installed on the robotic arm, and the external force obtained by the torque sensor is:

[0044]

[0045] Among them, M d D d and K d This is the preset amount.

[0046] Furthermore, the Kalman filter multi-sensor fusion is as follows:

[0047]

[0048]

[0049] Where x is the target position, and i is the index of different sensors, x iLet A(k) represent the estimate from the vision or force control sensor, and k be the Kalman gain. Here, A(k) and C(i,k+1) are both identity matrices.

[0050] A third aspect of the present invention provides a dynamic assembly device, wherein the dynamic assembly device is used to control a robot placed on an AGV (Automated Guided Vehicle) to perform an assembly operation. The robot has a robotic arm equipped with an assembly tool. The assembly object of the dynamic assembly is fixed by a clamping tool. The dynamic assembly device includes: a positioning and configuration device, which positions a reference point on the CAD model of the assembly object according to the assembly operation of the dynamic assembly, and matches the robotic arm and a camera according to the assembly operation and its operating space; a synchronization device, which synchronizes the AGV and the clamping device, and synchronizes the end effector of the robotic arm and the assembly workpiece on the assembly object with reference to the reference point; a vision tracking device, which tracks the reference point visually and corrects the position and angle errors of the reference point and the assembly workpiece using a proportional-integral-differential algorithm, thereby synchronizing the end effector of the robotic arm and the assembly workpiece; and a fusion device, which performs Kalman filter multi-sensor fusion based on the visual tracking of the reference point and the adjustment of adaptive force control, thereby calculating the speed and position of the end effector of the robotic arm to perform the assembly operation on the assembly workpiece.

[0051] A fourth aspect of the present invention provides a computer program product tangibly stored on a computer-readable medium and comprising computer-executable instructions that, when executed, cause at least one processor to perform the method according to a first aspect of the present invention.

[0052] A fifth aspect of the present invention provides a computer-readable medium having computer-executable instructions stored thereon, which, when executed, cause at least one processor to perform the method according to the first aspect of the present invention.

[0053] The dynamic assembly mechanism provided by this invention is particularly suitable for automotive final assembly. It utilizes AGVs, robots, and vision technology to perform dynamic assembly at a low cost. Furthermore, the accuracy and speed of the dynamic assembly meet requirements. The synchronized operation of the AGV and the gripping device ensures minimal changes in position and speed. Vision tracking enables the robot equipped with manipulators to accurately approach the assembly object, even with very small changes in position and speed. The fusion of adaptive force control and vision tracking further enhances the precision of the dynamic assembly operation. Attached Figure Description

[0054] Figure 1 This is a schematic diagram of reference point positioning according to a specific embodiment of the present invention;

[0055] Figure 2This is a flowchart of the steps of a dynamic assembly method according to a specific embodiment of the present invention;

[0056] Figure 3 This is a schematic diagram of Kalman filter multi-sensor fusion using a dynamic assembly method according to a specific embodiment of the present invention. Detailed Implementation

[0057] The specific embodiments of the present invention will be described below with reference to the accompanying drawings.

[0058] This invention provides a flexible and customizable dynamic assembly solution, particularly suitable for automotive assembly stations. The invention utilizes an AGV (Automated Guided Vehicle) with a robotic arm, and uses visual recognition to determine the relative position between the robotic arm's end effector and the workpiece. Furthermore, it employs a Kalman filter sensor fusion method combining visual and adaptive force control to execute the corresponding assembly operations.

[0059] The architecture of this invention comprises three parts: a gripping device, an AGV (Automated Guided Vehicle) trolley housing a robot, and auxiliary devices. Specifically, the gripping device is a clamping device; in an automotive production line, the assembly plant has a clamping track on top, allowing the clamping device to hold and slide the car to be assembled along the track. The AGV trolley houses a robot with a robotic arm. The end of the robotic arm is equipped with tools required for assembly, such as screwdrivers and drills, which can move under the control of the robotic arm. Auxiliary devices are optional, including a laser projector.

[0060] The clamping device is used to support the upper body of the vehicle. To replace workers on the AGV, this invention uses a human on the AGV to perform this task. A camera on the robot is used to locate the target assembly component at the robot's end effector, which performs the final assembly. If necessary, auxiliary devices assist the camera in locating the target. These auxiliary devices can be laser projectors and electronic tags, etc. Sometimes, features on the vehicle body are difficult to locate using a camera. With the help of auxiliary devices, the location of the target assembly component is more accurate and faster.

[0061] The first aspect of this invention provides a dynamic assembly method, wherein the dynamic assembly method is performed by a robot placed on an AGV (Automated Guided Vehicle), the robot having a robotic arm equipped with an assembly tool, and the object to be assembled being fixed by a clamping tool. Preferably, the assembly tool mounted on the robotic arm is a screwdriver or electric drill, and the assembly operation is a screw-tightening operation.

[0062] First, step S1 is executed. A reference point is located on the CAD model of the assembly object based on the assembly operation of the dynamic assembly. A robotic arm and camera are then matched according to the assembly operation and its operating space. Specifically, in this embodiment, the assembly object is a car, and the assembly operation involves tightening screws in a certain area of ​​the car's base. Therefore, we need to locate the screw holes. If another screw hole or other significant reference point can be located around the screw hole in the CAD model, it can serve as a positional reference for subsequent assembly operations. Furthermore, since the assembly operation involves tightening screws within the operating space of the car's base, a robotic arm and camera of the appropriate size are required to successfully complete the screw-tightening operation within that space. That is, appropriate tools are selected based on the details of the task. In addition, this invention also needs to consider visual requirements for automatically calculating camera selection and installation, such as field, depth, and resolution.

[0063] Furthermore, when it is impossible to locate a reference point on the CAD model of the assembly object based on the assembly operation of the dynamic assembly, the dynamic assembly method further includes the following step: setting a laser projector on the AGV or the robot to project a laser reference point onto the assembly object using a laser. In this invention, it is necessary to find obvious features for the camera to track. This invention must utilize an auxiliary device fixed to the clamping device to add features to the vehicle body for virtual camera tracking. The auxiliary device can be a laser marking projector, which can project a designed laser mark onto the vehicle body. The laser mark is easy to track virtually. The auxiliary device can be an electronic monitor. Before the camera begins tracking the electronic monitor, it needs to measure the deviation between the electronic monitor and the target screw hole. Once the deviation is determined, the camera tracks the electronic monitor to ensure that the tool can properly reach the target screw hole.

[0064] like Figure 1 As shown, the assembly object is a car base 100, the assembly workpiece is a screw hole 100a, and the assembly operation is tightening the screw. Therefore, when the end of the screwdriver 210 of the robotic arm on the AGV approaches the screw hole 100a where the tightening operation is to be performed, the camera 220 next to the screwdriver 210 cannot directly perform visual recognition on the screw hole 100a. Therefore, we select another nearby screw hole 100b as a reference point for tracking, wherein the reference screw hole 100b and the target screw hole 100a have a fixed deviation and can remain relatively stationary. Therefore, the end of the screwdriver 210 of the robotic arm uses the camera 220 to track the screw hole 100b to ensure that the screwdriver 210 can approach the screw hole 100a and perform the assembly operation in subsequent operations. In addition, once the assembly object is determined, the system can recommend such nearby features as reference points.

[0065] Then, step S2 is executed to synchronize the AGV and the gripping device, synchronizing the end effector of the robotic arm and the assembly workpiece on the assembly object with reference to the reference point. Preferably, the AGV and the gripping device perform synchronization based on magnetic navigation, QR code indication, or laser navigation.

[0066] Specifically, in this embodiment, the application scenario is an automobile assembly workshop. The clamping device is a gripper that can move along a predetermined track on the top of the assembly workshop, and can clamp the car to be assembled. The track is based on magnetic strips, rails, laser navigation, or QR codes. For example, the AGV (Automated Guided Vehicle) moves on the assembly workshop floor along the direction indicated by the QR code. Therefore, the movement trajectory of the AGV and the clamping tool can be located based on this, and the AGV and the clamping device can be synchronized accordingly.

[0067] Magnetic navigation cannot directly obtain position information. Therefore, the AGV moves at the same speed as the clamping tool under the guidance of this navigation. QR codes can directly obtain position information, making them better for the synchronous execution of dynamic assembly tasks. However, the AGV and clamping tool should move synchronously under system control during the dynamic assembly process.

[0068] Figure 2 This is a flowchart of the steps of a dynamic assembly method according to a specific embodiment of the present invention, as follows: Figure 2 As shown, step S3 is then executed, whereby the reference point is visually tracked, and a proportional-integral-differential algorithm is used to correct the positional and angular errors of the reference point and the assembly workpiece, thereby synchronizing the robotic arm end effector and the assembly workpiece. That is, in this embodiment, the screwdriver tip of the robotic arm finger is synchronized with the screw hole without contact, and the two remain relatively stationary. Here, screw hole 100a is the assembly workpiece, and screw hole 100b, which serves as the reference point, is the object being tracked visually.

[0069] Specifically, camera 220 continuously acquires images and utilizes asynchronous image acquisition to increase speed, where the speed needs to reach at least 50 frames per second. The reference points acting as the visual tracking object have angular information and are easily identifiable. Different screw holes in the camera image have different harmonics. In this embodiment, rotations within 90 degrees can be corrected during robot movement.

[0070] Preferably, in this embodiment, we employ a proportional-integral-derivative (PID) algorithm, wherein the position deviation is:

[0071] e x,y =x 0,xy -x x,y ,

[0072] Where, x 0,xyx is the coordinate position of the assembled workpiece. x,y The position of the assembled workpiece is calculated based on visual recognition of the relative position of the reference point and the assembled workpiece. In this embodiment, x 0,xy Let x be the coordinate position of screw hole 100a. x,y The position of screw hole 100a is calculated based on the visual identification of the relative positions of screw hole 100b and screw hole 100a, which serve as a reference point. Optionally, the reference point may also be a laser mark projected by a laser projector.

[0073] Furthermore, the angular deviation is:

[0074] e rz =x 0,rz -x rz ,

[0075] Where, x 0,rz For the angle of the assembled workpiece, x rz The angle of the assembled workpiece is calculated based on visual recognition of the relative angle between the reference point and the workpiece. In this embodiment, x 0,xy For the angle of screw hole 100a, x rz The angle of screw hole 100a is calculated based on the visual identification of the relative angle between screw hole 100b and screw hole 100a, which serve as a reference point. Optionally, the reference point may also be a laser mark projected by a laser projector.

[0076] Furthermore, the speed at which the robot's end effector moves horizontally is:

[0077]

[0078]

[0079] Furthermore, the rotational angular velocity of the robotic arm along the Z-axis is:

[0080]

[0081] Among them, the Mat rotation Let K be the rotation matrix of the current state variables. p K represents the proportional coefficient in the proportional-integral-differential algorithm. i is the integral coefficient.

[0082] This invention adjusts the speed at which the robotic arm approaches the screw hole 100a based on positional deviation.

[0083] Finally, step S4 is executed, where Kalman filter sensor fusion is performed based on the reference point tracked by vision and the adjustment of adaptive force control to calculate the speed and position of the robotic arm's end effector for performing assembly operations on the workpiece. Here, vision tracking is equivalent to a vision sensor, and the adjustment of adaptive force control is equivalent to a force sensor. Therefore, this invention, based on visual recognition and adaptive force control adjustment, can accurately perform assembly operations on the workpiece.

[0084] Taking a bolt connection as an example, when the screwdriver approaches the bolt hole, the tool and robot will block the camera from seeing the target. We need to select a nearby feature for the camera to track. This nearby feature has a fixed offset relative to the target bolt hole. The robot uses the camera to track the nearby feature to ensure the screwdriver can reach the target bolt hole.

[0085] The nearby features can be screw holes, patterns on components, etc. When the tool touches the target screw hole, the nearby feature should be identified by a camera, and its deviation relative to the target screw hole can be measured by CAD. Since the deviation between the nearby feature and the target screw hole is fixed, the deviation in the camera frame is also fixed. We use the fixed deviation in the camera frame to track the feature. Then, the tool can accurately touch the target screw hole. Furthermore, once the target is selected, nearby features can be automatically recommended by the system.

[0086] Specifically, for the adjustment of adaptive force control, this invention employs admittance control, wherein the external force sensed by the torque sensor is:

[0087]

[0088] Among them, M d D d and K d This is the preset amount.

[0089] Specifically, the Kalman filter multi-sensor fusion is as follows:

[0090]

[0091]

[0092] Where x is the target position, and i is the index of different sensors, x i Let A(k) represent the estimate from the vision or force control sensor, and k be the Kalman gain. Here, A(k) and C(i,k+1) are both identity matrices.

[0093] The above formula can be transformed into:

[0094]

[0095]

[0096] The Kalman gain update formula is as follows:

[0097]

[0098] P i (k+1|k+1)=(IK(i,k+1))·P i (k+1|k)

[0099] P i (k+1|k)=P i (k|k)+Q

[0100] Among them, P i (k+1|k) is the prior estimate covariance matrix;

[0101] P i (k+1|k+1) and P i (k|k) is the posterior estimated covariance matrix;

[0102] R and Q are noise estimates;

[0103] Therefore, after obtaining the target position estimates from the two sensors respectively, the weights of the sensor estimates are calculated:

[0104]

[0105] The speed control values ​​obtained from the two sensors are weighted and averaged to perform a fusion operation on visual tracking and adaptive force control adjustment.

[0106] like Figure 3 As shown, after performing visual tracking and adaptive force control adjustment, a test is performed. If the test is passed, the state and covariance matrix are initialized; if the test fails and the state is negative, a prediction operation is performed. Then, visual tracking or adaptive force control adjustment is performed to establish the visual tracking matrix and the adaptive force controllable matrix, and then an update is performed based on these.

[0107] A second aspect of the present invention provides a dynamic assembly system, the dynamic assembly system comprising an AGV (Automated Guided Vehicle) and a robot placed on the AGV, the robot having a robotic arm equipped with an assembly tool, the assembly object being dynamically assembled being fixed by a clamping tool, wherein the system includes: a processor; and a memory coupled to the processor, the memory having instructions stored therein, the instructions causing the electronic device to perform actions when executed by the processor, the actions including: S1, locating a reference point on a CAD model of the assembly object according to the assembly operation of the dynamic assembly, and according to the dynamic assembly... S1. The assembly operation and its operating space are matched with the robotic arm and camera; S2. The AGV and the clamping device are synchronized, and the end effector of the robotic arm and the assembly workpiece on the assembly object are synchronized with reference to the reference point; S3. The reference point is tracked visually, and the position and angle errors of the reference point and the assembly workpiece are corrected by proportional-integral-differential algorithm, so as to perform synchronization between the end effector of the robotic arm and the assembly workpiece; S4. Based on the visual tracking of the reference point and the adjustment of adaptive force control, Kalman filter multi-sensor fusion is performed to calculate the speed and position of the end effector of the robotic arm to perform the assembly operation on the assembly workpiece.

[0108] Furthermore, the assembly tools mounted on the robotic arm are screwdrivers and electric drills, and the assembly operation is a screw-tightening operation.

[0109] Furthermore, when it is impossible to locate a reference point on the CAD model of the assembly object based on the assembly operation of dynamic assembly, the action also includes: setting a laser projector on the AGV or the robot to project a laser reference point onto the assembly object using a laser.

[0110] Furthermore, the AGV and the clamping device perform synchronization based on magnetic navigation, QR code indication, or laser navigation.

[0111] Furthermore, the positional deviation is:

[0112] e x,y =x 0,xy -x x,y ,

[0113] Where, x 0,xy x represents the coordinate position of the assembled workpiece. x,y The position of the assembled workpiece is calculated based on visual recognition of the relative position of the reference point and the assembled workpiece.

[0114] Furthermore, the angular deviation is:

[0115] e rz =x 0,rz -x rz ,

[0116] Where, x 0,rz For the angle of the assembled workpiece, x rz The angle of the assembled workpiece is calculated based on the visual recognition of the relative angle between the reference point and the assembled workpiece.

[0117] Furthermore, the speed of the robotic arm's end effector is:

[0118]

[0119]

[0120] The rotational angular velocity of the robotic arm along the Z-axis is:

[0121]

[0122] Among them, the Mat rotation Let K be the rotation matrix of the current state variables. p K represents the proportional coefficient in the proportional-integral-differential algorithm. i is the integral coefficient.

[0123] Furthermore, a torque sensor is installed on the robotic arm, and the external force obtained by the torque sensor is:

[0124]

[0125] Among them, M d D d and K d This is the preset amount.

[0126] Furthermore, the Kalman filter multi-sensor fusion is as follows:

[0127]

[0128]

[0129] Where x is the target position, and i is the index of different sensors, x i Let A(k) represent the estimate from the vision or force control sensor, and k be the Kalman gain. Here, A(k) and C(i,k+1) are both identity matrices.

[0130] A third aspect of the present invention provides a dynamic assembly device, wherein the dynamic assembly device is used to control a robot placed on an AGV (Automated Guided Vehicle) to perform an assembly operation. The robot has a robotic arm equipped with an assembly tool. The assembly object of the dynamic assembly is fixed by a clamping tool. The dynamic assembly device includes: a positioning and configuration device, which positions a reference point on the CAD model of the assembly object according to the assembly operation of the dynamic assembly, and matches the robotic arm and a camera according to the assembly operation and its operating space; a synchronization device, which synchronizes the AGV and the clamping device, and synchronizes the end effector of the robotic arm and the assembly workpiece on the assembly object with reference to the reference point; a vision tracking device, which tracks the reference point visually and corrects the position and angle errors of the reference point and the assembly workpiece using a proportional-integral-differential algorithm, thereby synchronizing the end effector of the robotic arm and the assembly workpiece; and a fusion device, which performs Kalman filter multi-sensor fusion based on the visual tracking of the reference point and the adjustment of adaptive force control, thereby calculating the speed and position of the end effector of the robotic arm to perform the assembly operation on the assembly workpiece.

[0131] A fourth aspect of the present invention provides a computer program product tangibly stored on a computer-readable medium and comprising computer-executable instructions that, when executed, cause at least one processor to perform the method according to a first aspect of the present invention.

[0132] A fifth aspect of the present invention provides a computer-readable medium having computer-executable instructions stored thereon, which, when executed, cause at least one processor to perform the method according to the first aspect of the present invention.

[0133] The dynamic assembly mechanism provided by this invention is particularly suitable for automotive final assembly. It utilizes AGVs, robots, and vision technology to perform dynamic assembly at a low cost. Furthermore, the accuracy and speed of the dynamic assembly meet requirements. The synchronized operation of the AGV and the gripping device ensures minimal changes in position and speed. Vision tracking enables the robot equipped with manipulators to accurately approach the assembly object, even with very small changes in position and speed. The fusion of adaptive force control and vision tracking further enhances the precision of the dynamic assembly operation.

[0134] Although the present invention has been described in detail through the preferred embodiments above, it should be understood that the above description should not be considered as a limitation of the present invention. Various modifications and substitutions to the present invention will be apparent to those skilled in the art after reading the above. Therefore, the scope of protection of the present invention should be defined by the appended claims. Furthermore, no reference numerals in the claims should be construed as limiting the scope of the claims; the word "comprising" does not exclude any means or steps not listed in other claims or the specification; the words "first," "second," etc., are used only to indicate names and do not indicate any specific order.

Claims

1. Dynamic assembly method, wherein, The dynamic assembly method is performed by a robot placed on an AGV (Automated Guided Vehicle). The robot has a robotic arm equipped with assembly tools. The object to be assembled is fixed by a clamping tool. The method includes the following steps: S1, locate a reference point on the CAD model of the assembly object according to the assembly operation of the dynamic assembly, and match the robotic arm and camera according to the assembly operation of the dynamic assembly and its operating space. S2, synchronize the AGV and the clamping tool, and synchronize the end effector of the robotic arm and the assembly workpiece on the assembly object with reference to the reference point; S3, by visually tracking the reference point, the positional and angular errors of the reference point and the assembly workpiece are corrected by a proportional-integral-differential algorithm, thereby synchronizing the end effector of the robotic arm and the assembly workpiece; S4, based on visual tracking of the reference point and adaptive force control adjustment, Kalman filter multi-sensor fusion is performed to calculate the speed and position of the robotic arm end to perform assembly operations on the assembly workpiece; In step S1, locating the reference point includes: in the CAD model of the assembly object, selecting another feature adjacent to the target assembly workpiece as the reference point, wherein the reference point has a fixed deviation from the target assembly workpiece; In step S2, the AGV and the clamping tool synchronize based on magnetic navigation, QR code indication, or laser navigation. In step S4, the Kalman filter multi-sensor fusion is performed by using weights to weight the speed control quantities obtained from visual tracking and adaptive force control adjustment, thereby performing a fusion operation on visual tracking and adaptive force control adjustment.

2. The dynamic assembly method according to claim 1, characterized in that, The assembly tools mounted on the robotic arm are screwdrivers and electric drills, and the assembly operation is a screw-tightening operation.

3. The dynamic assembly method according to claim 1, characterized in that, When it is impossible to locate a reference point on the CAD model of the assembly object based on the assembly operation of the dynamic assembly, the dynamic assembly method further includes the following steps: A laser projector is then installed on the AGV or the robot to project a laser reference point onto the assembly object.

4. The dynamic assembly method according to claim 1, characterized in that, The position error is: ， in, The coordinate position of the assembled workpiece. The position of the assembled workpiece is calculated based on visual recognition of the relative position of the reference point and the assembled workpiece.

5. The dynamic assembly method according to claim 1, characterized in that, The angle error is: ， in, For the angle of the assembled workpiece, The angle of the assembled workpiece is calculated based on the visual recognition of the relative angle between the reference point and the assembled workpiece.

6. The dynamic assembly method according to claim 1, characterized in that, The speed at the end of the robotic arm is: ， ， The rotational angular velocity of the robotic arm along the Z-axis is: ， Among them, the The rotation matrix is ​​the current state variable. The proportional coefficient is the proportional coefficient in the proportional-integral-differential algorithm. is the integral coefficient.

7. The dynamic assembly method according to claim 1, characterized in that, The robotic arm is equipped with a torque sensor, and the external force obtained by the torque sensor is: in, , and This is the preset amount.

8. The dynamic assembly method according to claim 1, characterized in that, The Kalman filter multi-sensor fusion is as follows: in, For the target location, Subscripts for different sensors Represents the estimate from vision or force control sensors. For Kalman gain, where, and All are identity matrices.

9. A dynamic assembly system, comprising an AGV (Automated Guided Vehicle) and a robot placed on the AGV, the robot having a robotic arm equipped with assembly tools, the assembly object being dynamically assembled being fixed by a clamping tool, wherein... include: processor; as well as A memory coupled to the processor, the memory having instructions stored therein, the instructions causing the electronic device to perform actions when executed by the processor, the actions including: S1, locate a reference point on the CAD model of the assembly object according to the assembly operation of the dynamic assembly, and match the robotic arm and camera according to the assembly operation of the dynamic assembly and its operating space. S2, synchronize the AGV and the clamping tool, and synchronize the end effector of the robotic arm and the assembly workpiece on the assembly object with reference to the reference point; S3, by visually tracking the reference point, the positional and angular errors of the reference point and the assembly workpiece are corrected by a proportional-integral-differential algorithm, thereby synchronizing the end effector of the robotic arm and the assembly workpiece; S4, based on visual tracking of the reference point and adaptive force control adjustment, Kalman filter multi-sensor fusion is performed to calculate the speed and position of the robotic arm end to perform assembly operations on the assembly workpiece; In step S1, locating the reference point includes: in the CAD model of the assembly object, selecting another feature adjacent to the target assembly workpiece as the reference point, wherein the reference point has a fixed deviation from the target assembly workpiece; In step S2, the AGV and the clamping tool synchronize based on magnetic navigation, QR code indication, or laser navigation. In step S4, the Kalman filter multi-sensor fusion is performed by using weights to weight the speed control quantities obtained from visual tracking and adaptive force control adjustment, thereby performing a fusion operation on visual tracking and adaptive force control adjustment.

10. The dynamic assembly system according to claim 9, characterized in that, The assembly tools mounted on the robotic arm are screwdrivers and electric drills, and the assembly operation is a screw-tightening operation.

11. The dynamic assembly system according to claim 9, characterized in that, When it is impossible to locate a reference point on the CAD model of the assembly object based on the assembly operation of dynamic assembly, the action further includes: A laser projector is installed on the AGV or the robot to project a laser reference point onto the assembly object.

12. The dynamic assembly system according to claim 9, characterized in that, The position error is: ， in, The coordinate position of the assembled workpiece. The position of the assembled workpiece is calculated based on visual recognition of the relative position of the reference point and the assembled workpiece.

13. The dynamic assembly system according to claim 9, characterized in that, The angle error is: ， in, For the angle of the assembled workpiece, The angle of the assembled workpiece is calculated based on the visual recognition of the relative angle between the reference point and the assembled workpiece.

14. The dynamic assembly system according to claim 9, characterized in that, The speed at the end of the robotic arm is: ， ， The rotational angular velocity of the robotic arm along the Z-axis is: ， Among them, the The rotation matrix is ​​the current state variable. The proportional coefficient is the proportional coefficient in the proportional-integral-differential algorithm. is the integral coefficient.

15. The dynamic assembly system according to claim 9, characterized in that, The robotic arm is equipped with a torque sensor, and the external force obtained by the torque sensor is: in, , and This is the preset amount.

16. The dynamic assembly system according to claim 9, characterized in that, The Kalman filter multi-sensor fusion is as follows: in, For the target location, Subscripts for different sensors Represents the estimate from vision or force control sensors. For Kalman gain, where, and All are identity matrices.

17. Dynamic assembly device, wherein, The dynamic assembly device is used to control a robot placed on an AGV to perform assembly operations. The robot has a robotic arm equipped with assembly tools. The assembly object is fixed by a clamping tool. The dynamic assembly device includes: The positioning and configuration device locates a reference point on the CAD model of the assembly object according to the assembly operation of the dynamic assembly, and matches the robotic arm and camera according to the assembly operation of the dynamic assembly and its operating space. A synchronization device that synchronizes the AGV trolley and the clamping tool, and synchronizes the end effector of the robotic arm and the assembly workpiece on the assembly object with reference to the reference point; A vision tracking device tracks the reference point visually and corrects the position and angle errors of the reference point and the assembly workpiece using a proportional-integral-differential algorithm, thereby synchronizing the end effector of the robotic arm with the assembly workpiece. The fusion device, based on visual tracking of the reference point and adaptive force control adjustment, performs multi-sensor fusion using a Kalman filter to calculate the speed and position of the robotic arm end effector to perform assembly operations on the assembly workpiece. The positioning of the reference point includes: in the CAD model of the assembly object, selecting another feature that is adjacent to the target assembly workpiece as the reference point, wherein the reference point has a fixed deviation from the target assembly workpiece; The AGV and the clamping tool are synchronized based on magnetic navigation, QR code indication, or laser navigation. The Kalman filter multi-sensor fusion method utilizes weights to perform a weighted average of the speed control quantities obtained from visual tracking and adaptive force control adjustment, thereby performing a fusion operation on visual tracking and adaptive force control adjustment.

18. A computer program product tangibly stored on a computer-readable medium and comprising computer-executable instructions that, when executed, cause at least one processor to perform the method according to any one of claims 1 to 8.

19. A computer-readable medium having stored thereon computer-executable instructions, which, when executed, cause at least one processor to perform the method according to any one of claims 1 to 8.

Images