Machine vision screwing system, control method thereof, screwing device and medium

Abstract

The application discloses a kind of machine vision screwing system and its control method, screwing equipment and medium, method includes: in the process that actuator follows the movement of screwing workpiece to be waited for, the image information of screwing workpiece to be waited for is acquired;According to the pose information of the thread hole of screwing workpiece to be waited for is determined according to image information, and the pose information of actuator, the pose information of conveying mechanism and the pose information of the thread hole of screwing workpiece to be waited for are tracked dynamically to actuator;When screwing mechanism and the thread hole of screwing workpiece to be waited for are synchronized, target contact force is determined using force-position hybrid control mode, and screwing workpiece to be waited for is controlled by actuator according to target contact force and is screwed.The screwing operation can be completed under the premise that conveying mechanism does not stop, and the contact force of screwing operation is controlled using force-position hybrid control mode, the efficiency and positioning accuracy of visual guidance screwing are optimized, and the automation level is improved.

Description

Technical Field

[0001] This invention relates to the field of screw tightening technology, and more particularly to a control method for a machine vision screw tightening system, a machine vision screw tightening system, a computer-readable storage medium, and a screw tightening device. Background Technology

[0002] Currently, vision-guided screw tightening scenarios are all step-by-step scenarios, which require the conveyor belt to stop frequently, resulting in low screw tightening efficiency. Furthermore, these scenarios are only suitable for static scenarios or scenarios with few disturbances, and the positioning accuracy requirements are not high. Summary of the Invention

[0003] This invention aims to at least partially solve one of the technical problems in related technologies. To this end, the first objective of this invention is to propose a control method for a machine vision-guided screw-tightening system. This method determines the pose information of the threaded hole of the workpiece to be screwed based on its image information. The actuator is then dynamically tracked based on the pose information of the actuator, the conveying mechanism, and the threaded hole of the workpiece until the screw-tightening mechanism is synchronized with the threaded hole. A force-position hybrid control method is then used to determine the target contact force, and the actuator is controlled to perform the screw-tightening operation based on this target contact force. This allows the screw-tightening operation to be completed without stopping the conveying mechanism. Furthermore, the force-position hybrid control method optimizes the efficiency and positioning accuracy of vision-guided screw-tightening, and improves the level of automation.

[0004] The second objective of this invention is to provide a machine vision screw tightening system.

[0005] A third objective of this invention is to provide a computer-readable storage medium.

[0006] The fourth objective of this invention is to provide a screw-tightening device.

[0007] To achieve the above objectives, a first aspect of the present invention provides a control method for a machine vision screw-tightening system. The system includes an actuator, an imaging mechanism, a conveying mechanism, and a screw-tightening mechanism. The imaging mechanism and the screw-tightening mechanism are detachably mounted on the actuator. The imaging mechanism is configured to acquire image information, and the conveying mechanism is configured to convey a workpiece to be screwed. The method includes: acquiring image information of the workpiece to be screwed while the actuator follows the workpiece; determining the pose information of the threaded hole of the workpiece to be screwed based on the image information, and dynamically tracking the actuator based on the pose information of the actuator, the conveying mechanism, and the threaded hole of the workpiece to be screwed; and determining the target contact force using a force-position hybrid control method when the screw-tightening mechanism is synchronized with the threaded hole of the workpiece to be screwed, and controlling the actuator to perform a screw-tightening operation on the workpiece to be screwed based on the target contact force.

[0008] According to one embodiment of the present invention, the target contact force is determined by a force-position hybrid control method, including: when the screw-tightening mechanism is synchronized with the threaded hole of the workpiece to be tightened, acquiring the actual position, actual speed and actual acceleration of the screw-tightening mechanism; acquiring the actual contact force of the screw to be tightened; and determining the target contact force by a force-position hybrid control method based on the actual position, actual speed and actual acceleration of the screw-tightening mechanism and the actual contact force of the screw to be tightened.

[0009] According to one embodiment of the present invention, the target contact force is determined using the following formula:

[0010]

[0011] Among them, F d F represents the contact force of the target. e M represents the actual contact force of the screw to be tightened. d Let C represent the desired inertia matrix. d Let K represent the desired damping matrix. d Let S represent the desired stiffness matrix, I represent the selection matrix, and x represent the actual position of the screw-tightening mechanism. This indicates the actual speed of the screw-tightening mechanism. The actual acceleration of the screw-tightening mechanism is represented by x. d Indicates the desired position of the screw-tightening mechanism. This indicates the desired speed of the screw-tightening mechanism. This represents the desired acceleration of the screw-tightening mechanism.

[0012] According to one embodiment of the present invention, dynamic tracking of the actuator based on the pose information of the actuator, the pose information of the conveying mechanism, and the pose information of the threaded hole of the workpiece to be tightened includes: determining the pose error information between the threaded hole of the workpiece to be tightened and the screw tightening mechanism based on the pose information of the actuator, the pose information of the conveying mechanism, and the pose information of the threaded hole of the workpiece to be tightened; generating motion control commands based on the pose error information, the pose information of the actuator, and the pose information of the conveying mechanism; and controlling the actuator to perform dynamic tracking based on the motion control commands.

[0013] According to one embodiment of the present invention, the positional error information between the threaded hole of the workpiece to be screwed and the screw-tightening mechanism is determined by the following formula:

[0014] errDist k =L0+Δy t +(l 2k -l 2t )-(l 1k -l 1t )

[0015] Among them, errDist k L0 represents the pose error information of the threaded hole of the workpiece to be tightened and the screw tightening mechanism along the conveying direction at time k, and L0 represents the installation position deviation of the center of the imaging mechanism and the center of the screw tightening mechanism along the conveying direction. t This represents the pose error information along the conveying direction between the center of the imaging mechanism and the center of the threaded hole of the workpiece to be tightened at time t. 2k This represents the pose information of the actuator at time k, l 2t This represents the pose information of the actuator at time t, l 1k L represents the pose information of the transmission mechanism at time k. 1t This represents the pose information of the transmission mechanism at time t.

[0016] According to an embodiment of the present invention, the control method further includes: filtering the pose information of the threaded hole of the workpiece to be screwed; and / or filtering the pose error information between the threaded hole of the workpiece to be screwed and the screw-tightening mechanism.

[0017] According to an embodiment of the present invention, the control method further includes: acquiring the pose information of the threaded hole of the workpiece to be tightened at a first preset servo frequency, and acquiring the pose information of the actuator and the pose information of the conveying mechanism at a second preset servo frequency; wherein the first preset servo frequency is less than the second preset servo frequency.

[0018] According to one embodiment of the present invention, when the error between the position information of the screw tightening mechanism and the position information of the threaded hole of the workpiece to be tightened is less than a preset error threshold, and the moving speed of the actuator is the same as the moving speed of the workpiece to be tightened, it is determined that the screw tightening mechanism and the threaded hole of the workpiece to be tightened are synchronized.

[0019] To achieve the above objectives, a second aspect of the present invention provides a machine vision screw-tightening system, comprising an actuator, an imaging mechanism, a conveying mechanism, a screw-tightening mechanism, and a controller. The imaging mechanism and the screw-tightening mechanism are detachably mounted on the actuator. The conveying mechanism is configured to convey the workpiece to be screwed. The imaging mechanism is used to acquire image information of the workpiece to be screwed while the actuator is moving along with the workpiece. The controller is used to determine the pose information of the threaded hole of the workpiece to be screwed based on the image information, and to dynamically track the actuator based on the pose information of the actuator, the conveying mechanism, and the threaded hole of the workpiece to be screwed. When the screw-tightening mechanism is synchronized with the threaded hole of the workpiece to be screwed, a force-position hybrid control method is used to determine the target contact force, and the actuator is controlled to perform screw-tightening operation on the workpiece to be screwed based on the target contact force.

[0020] To achieve the above objectives, a third aspect of the present invention provides a computer-readable storage medium storing a control program for a machine vision screw-tightening system, which, when executed by a processor, implements the aforementioned control method for the machine vision screw-tightening system.

[0021] To achieve the above objectives, a fourth aspect of the present invention provides a screw-tightening device, including the aforementioned machine vision screw-tightening system.

[0022] According to the control method of the machine vision screw-tightening system of the present invention, the pose information of the threaded hole of the workpiece to be screwed is determined based on the image information of the workpiece to be screwed. Then, the screw-tightening mechanism is dynamically tracked based on the pose information of the actuator, the conveying mechanism, and the threaded hole of the workpiece to be screwed until they are synchronized. Next, a force-position hybrid control method is used to determine the target contact force, and the actuator is controlled to perform the screw-tightening operation based on the target contact force. This method enables the screw-tightening operation to be completed without stopping the conveying mechanism. Furthermore, the force-position hybrid control method controls the contact force of the screw-tightening operation, optimizing the efficiency and positioning accuracy of vision-guided screw-tightening and improving the level of automation. Attached Figure Description

[0023] Figure 1 This is a schematic diagram of a machine vision screw-tightening system according to an embodiment of the present invention;

[0024] Figure 2 This is a flowchart illustrating a control method for a machine vision screw-tightening system according to an embodiment of the present invention.

[0025] Figure 3 This is a flowchart illustrating a control method for a machine vision screw-tightening system according to another embodiment of the present invention. Detailed Implementation

[0026] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0027] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0028] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this application described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0029] The following describes in detail, with reference to the accompanying drawings, the machine vision screw-tightening system and its control method, screw-tightening equipment and medium of the present invention.

[0030] According to an embodiment of this application, a control method for a machine vision screw tightening system is provided.

[0031] Figure 1 This is a schematic diagram of a machine vision screw-tightening system according to an embodiment of the present invention, with reference to... Figure 1The machine vision screw-tightening system includes an actuator 11, an imaging mechanism 12, a conveying mechanism 13, and a screw-tightening mechanism 14. The imaging mechanism 12 and the screw-tightening mechanism 14 are detachably mounted on the actuator 11. The imaging mechanism 12 is configured to acquire image information, and the conveying mechanism 13 is configured to convey the workpiece to be tightened. The actuator 11 may include an actuator body 111 and an actuator end effector 112.

[0032] In some embodiments, continue to refer to Figure 1 The conveying mechanism 13 includes a conveyor belt. The machine vision screw-tightening system may include at least one photoelectric sensor 17, which is disposed on the conveyor belt. When there are two photoelectric sensors 17, they can be symmetrically disposed on both sides of the conveyor belt to detect whether there is a workpiece to be tightened on the conveying mechanism 13. The machine vision screw-tightening system may also include a vision motion controller 15 and a servo driver 16.

[0033] Specifically, the conveying mechanism 13, such as a conveyor belt, is configured to convey the workpiece to be tightened (e.g., a screw), and can move smoothly in the horizontal direction after activation. The conveying mechanism 13 is also equipped with photoelectric sensors 17, which can be one or more. The actuator 11, such as a gantry crane or a SCARA (Selective Compliance Assembly Robot Arm) robot, is used to dynamically track the workpiece to be tightened, with the tracking direction being the same as the movement direction of the conveying mechanism 13. The imaging mechanism 12, such as an industrial camera, is used to acquire image information of the workpiece to be tightened as the actuator 11 follows the workpiece. The screw-tightening mechanism 14 is detachably mounted on the actuator 11, specifically on the end effector 112 of the actuator, and can move up and down in the vertical direction to perform the screw-tightening operation synchronously with the threaded hole of the workpiece. There is a height difference between the imaging mechanism 12 and the screw-tightening mechanism 14, with the screw-tightening mechanism 14 mounted closer to the conveying mechanism 13.

[0034] The actuator 11 is in a servo state at the initial position, waiting for the arrival of the screw to be tightened. When the photoelectric sensor 17 detects the screw, it outputs a trigger signal to the vision motion controller 15. The vision motion controller 15 generates periodic trigger signals to the imaging mechanism 12, triggering the imaging mechanism 12 to capture images. The vision motion controller 15 acquires the image stream from the imaging mechanism 12 and performs processing operations on the acquired images to obtain the 6D pose of the threaded hole of the screw to be tightened. At the same time, it identifies the type of threaded hole of the screw to be tightened online, such as circular threaded hole, polygonal threaded hole, etc., and counts the number of screws to be tightened. The vision motion controller 15 obtains the pose information of the actuator 11 and the conveying mechanism 13 from the servo driver 16 via the EtherCAT (EtherNet Control Automation Technology, real-time Ethernet technology) bus. Based on the 6D pose of the threaded hole of the screw to be tightened and the pose information of the actuator 11 and the conveying mechanism 13, it performs trajectory planning, generates motion control information, and drives the end effector 112 of the actuator to drive the screw tightening mechanism 14 to perform the screw tightening operation through the servo driver 16.

[0035] Figure 2 This is a schematic flowchart illustrating a control method for a machine vision screw-tightening system according to an embodiment of the present invention. (Refer to...) Figure 2 The control method for this machine vision screw-tightening system may include:

[0036] S110: During the process of the actuator following the movement of the workpiece to be tightened, image information of the workpiece to be tightened is acquired.

[0037] Specifically, image information of the workpiece to be screwed is acquired through a camera, such as an industrial camera.

[0038] S120: Determine the pose information of the threaded hole of the workpiece to be tightened based on the image information, and dynamically track the actuator based on the pose information of the actuator, the pose information of the conveying mechanism, and the pose information of the threaded hole of the workpiece to be tightened.

[0039] Specifically, the image information of the workpiece to be screwed is processed, and the pose information of the threaded hole of the workpiece to be screwed can be obtained based on the image processing results. For example, the pose information may include the 6D pose information of the threaded hole of the workpiece to be screwed. At the same time, the type of threaded hole on the workpiece to be screwed can be determined based on the image information of the threaded hole, such as a circular threaded hole or a polygonal threaded hole.

[0040] Specifically, an encoder can be installed on the conveying mechanism side. By reading the encoder readings, the pose information of the conveying mechanism can be obtained. For example, the pose information may include the 6D pose information of the conveying mechanism. Similarly, an encoder can be installed on the end effector of the actuator. By reading the encoder readings, the pose information of the actuator can be obtained. For example, the pose information may include the 6D pose information of the actuator. The pose information of the actuator and the conveying mechanism can be acquired in real time via the EtherCAT bus. Simultaneously, trajectory planning is performed by combining the pose information of the threaded hole of the workpiece to be screwed. The planned trajectory includes the running speed and direction of the actuator. The actuator, carrying the screw-tightening mechanism, dynamically tracks the threaded hole of the workpiece to be screwed according to the planned motion trajectory until the screw-tightening mechanism and the workpiece to be screwed are synchronized.

[0041] S130, when the screw-tightening mechanism is synchronized with the threaded hole of the workpiece to be screwed, a force-position hybrid control method is used to determine the target contact force, and the execution structure is controlled according to the target contact force to perform the screw-tightening operation on the workpiece to be screwed.

[0042] Specifically, the actuator, along with the screw-tightening mechanism, dynamically tracks the threaded hole of the workpiece to be screwed according to the planned motion trajectory, gradually reducing the positional error between the screw-tightening mechanism and the threaded hole of the workpiece until the screw-tightening mechanism and the threaded hole of the workpiece are synchronized, that is, the screw-tightening mechanism and the threaded hole of the workpiece are relatively stationary. At this point, the actuator can be controlled to perform the screw-tightening operation on the workpiece.

[0043] Specifically, the target contact force refers to the external force applied to the actuator in the direction perpendicular to the movement of the workpiece to be tightened, used to tighten the screw into the threaded hole of the workpiece. When the screw-tightening mechanism is synchronized with the threaded hole of the workpiece to be tightened, a force-position hybrid control method is used to determine the target contact force, and this target contact force is applied to the actuator in the direction perpendicular to the movement of the workpiece to be tightened, controlling the screw-tightening mechanism to perform the screw-tightening operation, so that the screw-tightening mechanism can quickly and accurately tighten the screw into the threaded hole of the workpiece to be tightened.

[0044] In this way, screw tightening operations can be completed without stopping the conveyor mechanism, optimizing the efficiency and positioning accuracy of vision-guided screw tightening, improving the accuracy and precision of screw tightening, and enhancing the level of automation.

[0045] In some embodiments, the target contact force is determined using a force-position hybrid control method, including: acquiring the actual position, actual velocity, and actual acceleration of the screw-tightening mechanism when the screw-tightening mechanism is synchronized with the threaded hole of the workpiece to be tightened; acquiring the actual contact force of the screw to be tightened; and determining the target contact force using a force-position hybrid control method based on the actual position, actual velocity, and actual acceleration of the screw-tightening mechanism and the actual contact force of the screw to be tightened.

[0046] Specifically, when the screw-tightening mechanism is synchronized with the threaded hole on the workpiece to be tightened, the actual position of the screw-tightening mechanism is obtained by reading the encoder reading set on the end effector of the actuator. The actual speed and actual acceleration of the screw-tightening mechanism are obtained by the speed sensor and acceleration sensor set on the end effector of the actuator. The actual contact force of the screw is obtained by the force sensor set at the end of the screw-tightening mechanism. Based on the actual position, actual speed and actual acceleration of the screw-tightening mechanism, and the actual contact force of the screw to be tightened, the target contact force is determined by a force-position hybrid control method.

[0047] In some embodiments, the target contact force is determined using the following formula:

[0048]

[0049] Among them, F d F represents the contact force of the target. e M represents the actual contact force of the screw to be tightened. d Let C represent the desired inertia matrix. d Let K represent the desired damping matrix. d Let S represent the desired stiffness matrix, I represent the selection matrix, and x represent the actual position of the screw-tightening mechanism. This indicates the actual speed of the screw-tightening mechanism. The actual acceleration of the screw-tightening mechanism is represented by x. d Indicates the desired position of the screw-tightening mechanism. This indicates the desired speed of the screw-tightening mechanism. This represents the desired acceleration of the screw-tightening mechanism.

[0050] Specifically, after the screw-tightening mechanism is synchronized with the threaded hole of the workpiece to be screwed, the actual position x of the screw-tightening mechanism and the actual speed of the screw-tightening mechanism are... The actual acceleration of the screw tightening mechanism Desired position x of the screw tightening mechanism d Desired speed of screw tightening mechanism Desired acceleration of screw tightening mechanism and the actual contact force F of the screw to be tightened e Calculate the target contact force F using the following formula. d :

[0051]

[0052] Among them, the desired position x of the screw-tightening mechanism d Desired speed of screw tightening mechanism Desired acceleration of screw tightening mechanism Determined based on actual working conditions.

[0053] In this way, the target contact force is determined by force-position hybrid control, and the target contact force is applied to the actuator in the direction of movement perpendicular to the workpiece to be tightened. This enables the screw tightening mechanism to quickly and accurately tighten the screw into the threaded hole of the workpiece, improving the efficiency of screw tightening and, to a certain extent, preventing the screw from being tightened too loosely or too tightly.

[0054] In some embodiments, dynamic tracking of the actuator based on the pose information of the actuator, the pose information of the conveying mechanism, and the pose information of the threaded hole of the workpiece to be tightened includes: determining the pose error information between the threaded hole of the workpiece to be tightened and the screw tightening mechanism based on the pose information of the actuator, the pose information of the conveying mechanism, and the pose information of the threaded hole of the workpiece to be tightened; generating motion control commands based on the pose error information, the pose information of the actuator, and the pose information of the conveying mechanism; and controlling the actuator to perform dynamic tracking based on the motion control commands.

[0055] Specifically, when the pose error between the threaded hole of the workpiece to be tightened and the screw-tightening mechanism is close to 0, it means that the center of the screw-tightening mechanism is basically perpendicularly aligned with the center of the threaded hole of the workpiece to be tightened. Therefore, the pose error information between the threaded hole of the workpiece to be tightened and the screw-tightening mechanism is determined based on the pose information of the actuator, the transmission mechanism, and the threaded hole of the workpiece to be tightened. Based on the pose error information, motion control commands are generated by combining the pose information of the actuator and the transmission mechanism. These commands control the actuator to adjust the pose and speed of the screw-tightening mechanism in multiple degrees of freedom, continuously reducing the pose error between the threaded hole of the workpiece to be tightened and the screw-tightening mechanism, thus achieving tracking of the threaded hole of the workpiece to be tightened.

[0056] For example, when the positional error information between the threaded hole of the workpiece to be screwed and the screw-tightening mechanism indicates a certain difference in the horizontal direction, the moving speed of the screw-tightening mechanism can be adjusted so that the center of the screw-tightening mechanism and the center of the threaded hole of the workpiece to be screwed are on the same vertical line; when the positional error information between the threaded hole of the workpiece to be screwed and the screw-tightening mechanism indicates that the screw-tightening mechanism and the threaded hole of the workpiece to be screwed are at a certain angle to the horizontal plane, the angle between the screw-tightening mechanism and the horizontal plane can be adjusted, that is, the movement direction of the screw-tightening mechanism can be adjusted so that the center of the screw-tightening mechanism and the threaded hole of the workpiece to be screwed are parallel.

[0057] In some embodiments, the control method further includes: acquiring the pose information of the threaded hole of the workpiece to be tightened at a first preset servo frequency, and acquiring the pose information of the actuator and the pose information of the conveying mechanism at a second preset servo frequency; wherein the first preset servo frequency is less than the second preset servo frequency.

[0058] For example, during the tracking process, the imaging mechanism captures image information of the moving screw to be tightened workpiece. This image information is processed, and the pose information of the threaded hole of the screw to be tightened is calculated at a first preset servo frequency. Simultaneously, the pose information of the actuator and the conveying mechanism is acquired in real time via the EtherCAT bus at a second preset servo frequency. The pose information of the threaded hole of the screw to be tightened calculated at the first preset servo frequency, and the pose information of the actuator and the conveying mechanism acquired at the second preset servo frequency are fused together, and the pose error information between the threaded hole of the screw to be tightened and the screw-tightening mechanism is output at the second preset servo frequency. At this point, based on the pose information of the threaded hole of the screw to be tightened, the pose information of the actuator, and the pose information of the conveying mechanism, motion control commands are generated at the second preset servo frequency. The actuator, carrying the screw-tightening mechanism, adjusts in multiple degrees of freedom based on the motion control commands, dynamically tracking the threaded hole of the screw to be tightened until the pose error between the threaded hole of the screw to be tightened and the screw-tightening mechanism approaches zero.

[0059] It should be noted that the first preset servo frequency is the frequency at which the imaging mechanism captures images; this sampling frequency is low, resulting in accurate image acquisition. The second preset servo frequency is the data acquired by the encoder; this sampling frequency is high, resulting in less data measurement. The two data sets are fused to obtain higher-frequency and more accurate pose error information. Typically, the first preset servo frequency is 30Hz, and the second preset servo frequency is 1kHz.

[0060] In some embodiments, the positional error information between the threaded hole of the workpiece to be screwed and the screw-tightening mechanism is determined by the following formula:

[0061] errDist k =L0+Δy t +(l 2k -l 2t )-(l 1k -l 1t )

[0062] Among them, errDist k L0 represents the pose error information of the threaded hole of the workpiece to be tightened and the screw tightening mechanism along the conveying direction at time k, and L0 represents the installation position deviation of the center of the imaging mechanism and the center of the screw tightening mechanism along the conveying direction. tThis represents the pose error information along the conveying direction between the center of the imaging mechanism and the center of the threaded hole of the workpiece to be tightened at time t. 2k This represents the pose information of the actuator at time k, l 2t This represents the pose information of the actuator at time t, l 1k L represents the pose information of the transmission mechanism at time k. 1t This represents the pose information of the transmission mechanism at time t.

[0063] Specifically, based on the foregoing analysis, the pose error information between the threaded hole of the workpiece to be tightened and the screw-tightening mechanism is obtained by fusing the pose information of the threaded hole of the workpiece to be tightened (calculated at a first preset servo frequency), the pose information of the actuator (obtained at a second preset servo frequency), and the pose information of the conveying mechanism (fused at a second preset servo frequency). Since the first preset servo frequency is lower than the second preset servo frequency, the time interval between obtaining the pose information of the actuator and the conveying mechanism at the second preset servo frequency is shorter than the time interval between calculating the pose information of the threaded hole of the workpiece to be tightened at the first preset servo frequency.

[0064] For example, during dynamic tracking, the imaging mechanism captures the i-th frame of the workpiece to be tightened at time t, and calculates the pose information of the threaded hole of the workpiece at time t using a first preset servo frequency. Since time k falls between time t and time t+1, when the pose information of the threaded hole of the workpiece at time t is obtained using the first preset servo frequency, the pose information of the actuator at time t can be obtained using the second preset servo frequency. 2t and the pose information of the transmission mechanism 1t Before obtaining the pose information of the threaded hole of the workpiece to be tightened at frame i+1 (time t+1), the pose information of the actuator at time k can also be obtained. 2k and the pose information of the transmission mechanism 1k The installation position deviation L0 between the center of the shooting mechanism and the center of the screw-tightening mechanism along the conveying direction, and the pose error information Δy between the center of the shooting mechanism and the center of the threaded hole of the workpiece to be tightened at time t along the conveying direction. t The pose information of the actuator at time k 2k The pose information of the actuator at time t 2t The pose information of the transmission mechanism at time k 1k The pose information of the transmission mechanism at time t 1t Enter the following formula to calculate the positional error information between the threaded hole of the workpiece to be screwed and the screw-tightening mechanism:

[0065] errDist k =L0+Δy t +(l 2k -l2t )-(l 1k -l 1t )

[0066] In this way, the threaded hole of the screw to be tightened is aligned with the timestamps of the conveying mechanism and the actuator.

[0067] In some embodiments, the control method described above further includes: filtering the orientation information of the threaded hole of the workpiece to be screwed; and / or filtering the orientation error information between the threaded hole of the workpiece to be screwed and the screw-tightening mechanism.

[0068] Specifically, during dynamic tracking, various external disturbances exist. For example, the workpiece to be tightened may be moved on the conveyor mechanism, subjected to high-frequency or specific-frequency disturbances. To eliminate the influence of these external disturbances, filtering is required on the pose information of the threaded hole of the workpiece to be tightened, the pose error information between the threaded hole and the tightening mechanism, or both. For example, Kalman filtering is performed when the workpiece is moved on the conveyor mechanism, low-pass filtering is performed when it is subjected to high-frequency disturbances, and notch filtering is performed when it is subjected to specific-frequency disturbances. The following explanation uses the filtering of the pose error information between the threaded hole and the tightening mechanism as an example, but this is not intended to limit the invention.

[0069] For example, when it is detected that the workpiece to be tightened is moved on the conveyor mechanism, the pose error information between the threaded hole of the workpiece and the screw tightening mechanism is processed by Kalman filtering using the following formula:

[0070]

[0071] Where x represents the system state, This represents the estimated positional error between the threaded hole of the workpiece to be tightened and the screw-tightening mechanism at time k+1 after error correction. W(k+1) represents the estimated positional error information between the threaded hole of the workpiece to be screwed and the screw-tightening mechanism at time k, where W(k+1) represents the Kalman gain. This represents the estimated error value of the positional error information between the threaded hole of the workpiece to be tightened and the screw tightening mechanism at time k+1.

[0072] When a high-frequency disturbance is detected on the conveying mechanism of the workpiece to be tightened, the positional error information between the threaded hole of the workpiece and the screw tightening mechanism is low-pass filtered using the following formula:

[0073] y i =α*x i +(1-α)*y i-1

[0074] Among them, y i y represents the filtered value indicating the pose error information between the threaded hole of the workpiece to be screwed and the screw-tightening mechanism at the current moment. i-1 The filtered value, x, represents the pose error information between the threaded hole of the workpiece to be tightened and the screw-tightening mechanism at the previous moment. i This represents the detected value of the pose error information between the threaded hole of the workpiece to be screwed and the screw-tightening mechanism at the current moment, where α represents the low-pass filter coefficient.

[0075] When a disturbance of a specific frequency is detected on the conveyor mechanism of the workpiece to be tightened, the orientation error information between the threaded hole of the workpiece and the screw tightening mechanism is processed by notch filtering using the following formula:

[0076] y i =a0*x i +a1*x i-1 +a2*x i-2 +b1*y i-1 +b2*y i-2

[0077] Among them, y i y represents the filtered value indicating the pose error information between the threaded hole of the workpiece to be screwed and the screw-tightening mechanism at the current moment. i-1 y represents the filtered value indicating the pose error information between the threaded hole of the workpiece to be tightened and the screw-tightening mechanism at the previous moment. i-2 The filtered value, x, represents the positional error information between the threaded hole of the workpiece to be screwed and the screw-tightening mechanism at the previous time step. i The detected value, x, represents the positional error information between the threaded hole of the workpiece to be screwed and the screw-tightening mechanism at the current moment. i-1 The detected value, x, represents the positional error information between the threaded hole of the workpiece to be tightened and the screw-tightening mechanism at the previous moment. i-2 The value represents the detected positional error information between the threaded hole of the workpiece to be screwed and the screw-tightening mechanism at the previous time. a0, a1, a2, b1 and b2 represent the notch filter coefficients.

[0078] Thus, the orientation information of the threaded hole of the workpiece to be screwed, and / or the orientation error information between the threaded hole of the workpiece to be screwed and the screw-tightening mechanism are filtered to eliminate the influence of external disturbance factors and improve positioning accuracy.

[0079] In some embodiments, if the error between the pose information of the screw tightening mechanism and the pose information of the threaded hole of the workpiece to be tightened is less than a preset error threshold, and the moving speed of the actuator is the same as the moving speed of the workpiece to be tightened, it is determined that the screw tightening mechanism and the threaded hole of the workpiece to be tightened are synchronized.

[0080] Specifically, the dynamic screw-tightening process is divided into three stages: dynamic tracking, synchronous tracking screw-tightening, and reset to zero. During dynamic tracking, motion control commands are generated based on the error information between the screw-tightening mechanism's pose and the pose information of the threaded hole of the workpiece to be screwed, as well as the pose information of the actuator and the transmission mechanism. The actuator, carrying the screw-tightening mechanism, continuously adjusts its pose and speed in multiple degrees of freedom based on these motion control commands. When the error information between the screw-tightening mechanism's pose and the threaded hole of the workpiece to be screwed, and the speed information of the actuator, meet preset conditions, the dynamic tracking stage ends, and the synchronous tracking screw-tightening stage begins. After performing the screw-tightening operation, the process enters the reset to zero stage, awaiting the arrival of the next workpiece to be screwed.

[0081] For example, if the error between the position information of the screw-tightening mechanism and the position information of the threaded hole of the workpiece to be screwed is less than a preset error threshold, and the moving speed of the actuator is the same as the moving speed of the workpiece to be screwed, then the synchronous tracking screw-tightening stage is entered. At this time, the target contact force is determined by force-position hybrid control, and the target contact force is applied to the actuator in the direction perpendicular to the movement of the workpiece to be screwed, so as to control the screw-tightening mechanism to perform screw-tightening operation on the workpiece to be screwed; otherwise, the actuator continues to adjust the position and speed with the screw-tightening mechanism.

[0082] It should be noted that when there are multiple threaded holes on the workpiece to be screwed, during the dynamic tracking phase, the actuator sequentially tracks the threaded holes on the workpiece. When the speeds of the actuator and the workpiece are synchronized, and the positional errors between the screw-tightening mechanism and each threaded hole on the workpiece are less than a preset error threshold, the synchronous tracking screw-tightening phase begins. At this point, a force-position hybrid control method is used to determine the target contact force, and this target contact force is applied to the actuator in the direction perpendicular to the movement of the workpiece, controlling the screw-tightening mechanism to sequentially screw in all the threaded holes on the workpiece in a specified order.

[0083] In summary, a force-position hybrid control method is employed to determine the target contact force. This target contact force is then applied to the actuator in the direction perpendicular to the movement of the workpiece to be tightened. After the screw-tightening mechanism synchronizes with the threaded hole of the workpiece, it enables the screw-tightening mechanism to quickly and accurately tighten the screw into the threaded hole, improving screw-tightening efficiency and, to some extent, preventing the screw from being tightened too loosely or too tightly. To eliminate the influence of external disturbances, the pose information of the threaded hole of the workpiece to be tightened, and / or the pose error information between the threaded hole of the workpiece and the screw-tightening mechanism, is filtered to improve positioning accuracy. Thus, screw-tightening can be completed without stopping the conveyor mechanism. Furthermore, the force-position hybrid control method optimizes the efficiency and positioning accuracy of vision-guided screw-tightening, improving the level of automation.

[0084] In some embodiments, refer to Figure 1 Furthermore, a machine vision screw-tightening system is proposed, including an actuator 11, an imaging mechanism 12, a conveying mechanism 13, a screw-tightening mechanism 14, and a controller 15. The imaging mechanism 12 and the screw-tightening mechanism 14 are detachably mounted on the actuator 11. The conveying mechanism 13 is configured to convey the workpiece to be screwed. The imaging mechanism 12 is used to acquire image information of the workpiece to be screwed while the actuator 11 is moving along with the workpiece. The controller 15 is used to determine the pose information of the threaded hole of the workpiece to be screwed based on the image information, and to dynamically track the actuator 11 based on the pose information of the actuator 11, the conveying mechanism 13, and the threaded hole of the workpiece to be screwed. When the screw-tightening mechanism 14 is synchronized with the threaded hole of the workpiece to be screwed, a force-position hybrid control method is used to determine the target contact force, and the actuator is controlled to perform screw-tightening operation on the workpiece to be screwed based on the target contact force.

[0085] Specifically, the conveying mechanism 13, such as a conveyor belt, is configured to convey the workpiece to be tightened (the screw), and can move smoothly in the horizontal direction after activation. The conveying mechanism 13 is also equipped with photoelectric sensors 17, one or more of which are used to detect the presence of the workpiece to be tightened on the conveying mechanism 13. The actuator 11, such as a gantry crane or a SCARA robot, includes an actuator body 111 and an end effector 112, used to dynamically track the workpiece to be tightened, with the tracking direction being the same as the movement direction of the conveying mechanism 13. The imaging mechanism 12 is used to acquire image information of the workpiece to be tightened during the movement of the actuator 11 following the workpiece, and upload it to the controller 15. The screw-tightening mechanism 14 can move vertically, controlled by the actuator 11, and is used to perform the screw-tightening operation when the screw-tightening mechanism 14 is synchronized with the threaded hole of the workpiece to be tightened. The shooting mechanism 12 and the screw-tightening mechanism 14 are detachably mounted on the actuator 11, specifically on the end effector 112 of the actuator. There is a height difference between the shooting mechanism 12 and the screw-tightening mechanism 14. The screw-tightening mechanism 14 is mounted closer to the conveying mechanism 13, while the vertical distance between the shooting mechanism 12 and the workpiece to be screwed remains unchanged.

[0086] Specifically, the controller 15 integrates a vision processing function block and a motion control function block. The two function blocks interact with each other in real time through shared memory. The vision processing function block includes a pose fusion module, and the motion control function block includes a dynamic tracking module.

[0087] For example, the vision processing function block is used to obtain the pose information of the threaded hole of the workpiece to be tightened based on the image information of the screw to be tightened and transmit it to the pose fusion module. The controller 15 obtains the pose information of the actuator 11 and the transmission mechanism 13 in real time from the servo driver 16 via the EtherCAT bus at the second servo frequency, and writes it into the shared memory. The pose fusion module reads the pose information of the actuator 11 and the transmission mechanism 13 from the shared memory, and fuses them with the pose information of the threaded hole of the workpiece to be tightened. It outputs the pose error information of the threaded hole of the workpiece to be tightened and the screw tightening mechanism 14 and writes it into the shared memory. The dynamic tracking module reads the pose error information of the threaded hole of the workpiece to be tightened and the screw tightening mechanism 14 from the shared memory. At the same time, it generates motion control commands by combining the pose information of the actuator 11 and the transmission mechanism 13, and uses the PID (Proportional Integral Differential) control method to track the threaded hole of the workpiece to be tightened.

[0088] For example, the positional error information of the threaded hole of the workpiece to be screwed and the screw-tightening mechanism 14, and the scaling factor K are used. p Differential coefficient T dand integral coefficient T i The input is sent to the dynamic tracking module, which, based on various coefficients and the positional error information between the threaded hole of the workpiece to be tightened and the screw tightening mechanism 14, calculates the control quantity for the current control cycle using the following formula:

[0089]

[0090] Where u(t) is the control quantity for the current control cycle, and errDist t K represents the pose error curve between the threaded hole of the workpiece to be screwed and the screw-tightening mechanism 14. p T is the proportionality coefficient. d T is the differential coefficient. i The integral coefficient is used. The dynamic tracking module outputs motion control commands to the servo driver 16 based on the pose information of the actuator 11, the pose information of the transmission mechanism 13, and the control quantity of the current control cycle. The actuator 11 performs dynamic tracking based on the motion control commands issued by the servo driver 16. When the screw tightening mechanism 14 is synchronized with the threaded hole of the screw to be tightened, the target contact force is determined by the force-position hybrid control method. The target contact force is applied to the actuator in the direction perpendicular to the movement of the workpiece to be tightened, and the screw tightening mechanism 14 is controlled to perform screw tightening operation on the workpiece to be tightened.

[0091] As a concrete example, refer to Figure 3 The control methods for a machine vision screw-tightening system may include:

[0092] S201, Begin.

[0093] S202, the actuator is in servo mode at the starting position.

[0094] The actuator is in servo mode at the starting position, waiting for the arrival of the workpiece to be tightened.

[0095] S203, photoelectric sensor detection.

[0096] S204: Determine whether the threaded hole of the workpiece to be tightened has been detected. If yes, execute S205; otherwise, execute S202.

[0097] After the photoelectric sensor detects the workpiece to be tightened, it outputs a trigger signal to the controller. The controller generates a periodic hardware trigger signal and transmits it to the imaging mechanism, which triggers the imaging mechanism to acquire image information of the workpiece to be tightened and uploads the image information to the controller. At the same time, it identifies the type of thread hole of the workpiece to be tightened online and counts the number of workpieces to be tightened.

[0098] S205, Start tracking.

[0099] S206, Dynamic Tracking.

[0100] After acquiring and processing the image information of the workpiece to be tightened, the controller obtains the pose information of the threaded hole of the workpiece at a first preset servo frequency. Simultaneously, the controller obtains real-time feedback pose information of the actuator and the conveying mechanism from the servo driver via the EtherCAT bus at a second servo frequency. The pose information of the threaded hole of the workpiece to be tightened, calculated at the first preset servo frequency, along with the pose information of the actuator and the conveying mechanism obtained at the second servo frequency, are transmitted to the pose fusion module for fusion. The module outputs the pose error information between the threaded hole of the workpiece to be tightened and the screw-tightening mechanism at the second preset servo frequency. The dynamic tracking module generates motion control commands at the second preset servo frequency based on the pose error information of the threaded hole of the workpiece to be tightened and the screw-tightening mechanism, as well as the pose information of the actuator and the conveying mechanism, and transmits these commands to the servo driver. The actuator dynamically tracks the workpiece to be tightened based on the motion control commands issued by the servo driver.

[0101] In addition, various external disturbances exist during dynamic tracking, such as the workpiece to be tightened being moved on the conveyor mechanism, or being subjected to high-frequency or specific-frequency disturbances. To eliminate the influence of external disturbances, it is necessary to filter the pose information of the threaded hole of the workpiece to be tightened, the pose error information between the threaded hole of the workpiece to be tightened and the screw tightening mechanism, or the pose information of the threaded hole of the workpiece to be tightened and the pose error information between the threaded hole of the workpiece to be tightened and the screw tightening mechanism. For example, Kalman filtering is performed when the workpiece to be tightened is moved on the conveyor mechanism, low-pass filtering is performed when the workpiece to be tightened is subjected to high-frequency disturbances on the conveyor mechanism, and notch filtering is performed when the workpiece to be tightened is subjected to specific-frequency disturbances on the conveyor mechanism.

[0102] S207, determine whether the motion parameters are synchronized. If they are, proceed to S208; otherwise, continue to S206.

[0103] If the error between the position information of the screw tightening mechanism and the position information of the threaded hole of the workpiece to be tightened is less than the preset error threshold, and the moving speed of the actuator is the same as the moving speed of the workpiece to be tightened, it is determined that the screw tightening mechanism and the threaded hole of the workpiece to be tightened are synchronized, then the dynamic tracking stage ends and the synchronous tracking screw tightening stage begins.

[0104] S208, synchronously tracks the threaded hole of the workpiece to be tightened.

[0105] The target contact force is determined by a force-position hybrid control method based on the actual position, actual speed, and actual acceleration of the screw-tightening mechanism, as well as the actual contact force of the screw to be tightened.

[0106] S209, Perform the screw tightening operation.

[0107] In the direction of movement perpendicular to the workpiece to be tightened, a target contact force is applied to the actuator, so that the screw tightening mechanism can quickly and accurately tighten the screw into the threaded hole of the workpiece.

[0108] S210, End.

[0109] It should be noted that the above explanation of the embodiments and beneficial effects of the control method for the machine vision screw tightening system also applies to the machine vision screw tightening system of the present invention. To avoid redundancy, it will not be elaborated in detail here.

[0110] In some embodiments, a computer-readable storage medium is also provided, on which a control program for a machine vision screw-tightening system is stored, which, when executed by a processor, implements the aforementioned control method for the machine vision screw-tightening system.

[0111] It should be noted that the above explanation of the embodiments and beneficial effects of the control method for the machine vision screw tightening system is also applicable to the computer-readable storage medium of the embodiments of the present invention. To avoid redundancy, it will not be elaborated in detail here.

[0112] In some embodiments, a screw-tightening device is also provided, including the aforementioned machine vision screw-tightening system.

[0113] It should be noted that the above explanation of the embodiments and beneficial effects of the control method for the machine vision screw tightening system also applies to the screw tightening device of the present invention. To avoid redundancy, it will not be elaborated in detail here.

[0114] It should be noted that the logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0115] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0116] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0117] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0118] In this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components, unless otherwise explicitly limited. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0119] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A control method of a machine vision screwing system, characterized by, The system includes an actuator, an imaging mechanism, a conveying mechanism, and a screw-tightening mechanism. The imaging mechanism and the screw-tightening mechanism are detachably mounted on the actuator. The imaging mechanism is configured to acquire image information, and the conveying mechanism is configured to convey the workpiece to be tightened. The method includes: During the process of the actuator following the movement of the workpiece to be tightened, image information of the workpiece to be tightened is acquired; The position and orientation information of the threaded hole of the workpiece to be tightened is determined based on the image information, and the actuator is dynamically tracked based on the position and orientation information of the actuator, the conveying mechanism, and the threaded hole of the workpiece to be tightened. When the screw-tightening mechanism is synchronized with the threaded hole of the workpiece to be screwed, a force-position hybrid control method is used to determine the target contact force, and the execution structure is controlled to perform screw-tightening operation on the workpiece to be screwed according to the target contact force; Specifically, if the error between the position information of the screw-tightening mechanism and the position information of the threaded hole of the workpiece to be tightened is less than a preset error threshold, and the moving speed of the actuator is the same as the moving speed of the workpiece to be tightened, then the screw-tightening mechanism is determined to be synchronized with the threaded hole of the workpiece to be tightened.

2. The control method according to claim 1, characterized by, The target contact force is determined using a force-position hybrid control method, including: When the screw-tightening mechanism is synchronized with the threaded hole of the workpiece to be tightened, the actual position, actual speed and actual acceleration of the screw-tightening mechanism are obtained; Obtain the actual contact force of the screw to be tightened; The target contact force is determined by a force-position hybrid control method based on the actual position, actual speed, and actual acceleration of the screw-tightening mechanism, as well as the actual contact force of the screw to be tightened.

3. The control method according to claim 2, characterized by, The target contact force is determined using the following formula: in, This represents the target contact force. This indicates the actual contact force of the screw to be tightened. Represents the desired inertia matrix. Represents the desired damping matrix. Represents the desired stiffness matrix. Represents the selection matrix. Represents the identity matrix. This indicates the actual position of the screw-tightening mechanism. This indicates the actual speed of the screw-tightening mechanism. This indicates the actual acceleration of the screw-tightening mechanism. This indicates the desired position of the screw-tightening mechanism. This indicates the desired speed of the screw-tightening mechanism. This represents the desired acceleration of the screw-tightening mechanism.

4. The control method according to claim 1, characterized by, Dynamic tracking of the actuator is performed based on the pose information of the actuator, the pose information of the conveying mechanism, and the pose information of the threaded hole of the workpiece to be tightened, including: The positional error information between the threaded hole of the workpiece to be tightened and the screw tightening mechanism is determined based on the positional information of the actuator, the positional information of the conveying mechanism, and the positional information of the threaded hole of the workpiece to be tightened. Motion control commands are generated based on the pose error information, the pose information of the actuator, and the pose information of the transmission mechanism. The actuator is controlled to perform dynamic tracking according to the motion control command.

5. The control method according to claim 4, characterized by, The positional error information between the threaded hole of the workpiece to be tightened and the screw-tightening mechanism is determined by the following formula: in, This represents the pose error information of the threaded hole of the workpiece to be tightened and the screw tightening mechanism along the conveying direction at time k. This indicates the installation position deviation between the center of the shooting mechanism and the center of the screw-tightening mechanism along the conveying direction. This represents the pose error information of the center of the shooting mechanism and the center of the threaded hole of the workpiece to be tightened along the conveying direction at time t. This represents the pose information of the actuator at time k. This represents the pose information of the actuator at time t. This represents the pose information of the transmission mechanism at time k. This represents the pose information of the transmission mechanism at time t.

6. The control method according to claim 4, characterized in that, Also includes: The orientation information of the threaded hole of the workpiece to be tightened is filtered. and / or The positional error information between the threaded hole of the workpiece to be tightened and the screw tightening mechanism is filtered.

7. The control method according to any one of claims 1-6, characterized in that, Also includes: The position and orientation information of the threaded hole of the workpiece to be tightened is obtained at a first preset servo frequency, and the position and orientation information of the actuator and the conveying mechanism are obtained at a second preset servo frequency. Wherein, the first preset servo frequency is less than the second preset servo frequency.

8. A machine vision screw-tightening system, characterized in that, The system includes an actuator, a camera, a conveyor, a screw-tightening mechanism, and a controller. The camera and screw-tightening mechanisms are detachably mounted on the actuator. The conveyor is configured to convey the workpiece to be screwed. The imaging mechanism is used to acquire image information of the screw to be tightened workpiece during the movement of the actuator following the movement of the screw to be tightened workpiece; The controller is configured to determine the pose information of the threaded hole of the workpiece to be screwed based on the image information, and to dynamically track the actuator based on the pose information of the actuator, the pose information of the conveying mechanism, and the pose information of the threaded hole of the workpiece to be screwed. When the screw-tightening mechanism is synchronized with the threaded hole of the workpiece to be screwed, a force-position hybrid control method is used to determine the target contact force, and the actuator is controlled to perform a screw-tightening operation on the workpiece based on the target contact force. Specifically, if the error between the pose information of the screw-tightening mechanism and the pose information of the threaded hole of the workpiece to be screwed is less than a preset error threshold, and the moving speed of the actuator is the same as the moving speed of the workpiece to be screwed, then the screw-tightening mechanism is determined to be synchronized with the threaded hole of the workpiece to be screwed.

9. A computer-readable storage medium, characterized in that, It stores a control program for a machine vision screw tightening system, which, when executed by a processor, implements the control method for the machine vision screw tightening system according to any one of claims 1-7.

10. A screw-tightening device, characterized in that, Includes the machine vision screw tightening system as described in claim 8.

Images