Method, device, controller, robot and screwing apparatus

Abstract

The application provides a screw driving method, device, controller, manipulator and screw driving equipment. The screw driving method obtains drawing information of a workpiece, obtains a preset positioning point position of a positioning point and a preset screw hole position of a screw hole, and the positioning point and the screw hole are located on the workpiece; a current workpiece image of the workpiece is obtained, and a current positioning point position is obtained; a current screw hole deviation is obtained according to the current positioning point position and the preset positioning point position; a screw driving position is obtained according to the current screw hole deviation and the preset screw hole position; and the manipulator is controlled to drive a screw on the workpiece according to the screw driving position. Thus, efficient screw driving is realized.

Description

Technical Field

[0001] This invention relates to the field of automation technology, and in particular to a method, apparatus, controller, robot, and screw-driving equipment for screw driving. Background Technology

[0002] Electrical components typically involve many parts, such as PCBs (Printed Circuit Boards). These components often require screwing to assemble. Before screwing, screw holes are usually pre-drilled on the components according to the assembly method; during screwing, screws are driven into each hole one by one, following the location of the existing screw holes.

[0003] For thin and lightweight workpieces like PCBs (Printed Circuit Boards), the initial screw-driving process can easily cause instability. Therefore, screws are initially left untightened, and tightened only after all screws are mostly in place. However, this method is slow and inefficient, necessitating the development of automated and intelligent screw-driving technology. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of the prior art and provide a screw-driving method, apparatus, controller, robot, and screw-driving equipment that can efficiently drive screws.

[0005] The objective of this invention is achieved through the following technical solution:

[0006] A method for driving screws, comprising:

[0007] Obtain the workpiece drawing information to obtain the preset positioning point position of the positioning point and the preset screw hole position of the screw hole, wherein the positioning point and the screw hole are located on the workpiece;

[0008] Obtain the current workpiece image to get the current positioning point position;

[0009] The current screw hole deviation is obtained based on the current positioning point position and the preset positioning point position;

[0010] The screw-driving position is obtained based on the current screw hole deviation and the preset screw hole position;

[0011] Based on the screw location, the robot arm is controlled to screw the screws onto the workpiece.

[0012] A screw-driving device, comprising:

[0013] Drawing information acquisition module: used to acquire drawing information of workpiece, obtain the preset positioning point position of positioning point and the preset screw hole position of screw hole, wherein the positioning point and the screw hole are located on the workpiece;

[0014] Current workpiece acquisition module: used to acquire the current workpiece image and obtain the current positioning point position.

[0015] The position deviation calculation module is used to calculate the current screw hole deviation based on the current positioning point position obtained by the current workpiece acquisition module and the preset positioning point position obtained by the drawing information acquisition module.

[0016] The working position calculation module is used to determine the screw driving position based on the current screw hole deviation obtained by the position deviation calculation module and the preset screw hole position obtained by the drawing information acquisition module.

[0017] Screw driving module: Used to control the robot to drive screws onto the workpiece based on the screw driving position obtained by the calculation working position module.

[0018] A controller comprising the screw-driving device described in any of the above embodiments.

[0019] A robotic arm that screws into the workpiece according to the screw-driving method described in any of the above embodiments.

[0020] A screw-driving device comprising the robotic arm described in any of the above embodiments.

[0021] An electronic device comprising:

[0022] Processor; and

[0023] A memory that stores executable code, which, when executed by the processor, causes the processor to perform the method described above.

[0024] A computer-readable storage medium having executable code stored thereon, which, when executed by a processor of an electronic device, causes the processor to perform the method described above.

[0025] Compared with the prior art, the present invention has at least the following advantages:

[0026] By acquiring the workpiece drawing information, the preset positioning point position and the preset screw hole position of each screw hole are obtained. Then, the current positioning point position is obtained by acquiring the current workpiece image. Based on the current positioning point position and the preset positioning point position, the current screw hole deviation is obtained. Then, based on the preset screw hole position and the current screw hole deviation, the screw position of each screw hole is determined. Thus, the screw hole on the workpiece can be fixed by controlling the robot arm to drive screws, which improves the speed of screw driving and makes the screw driving technology more automated and intelligent. Attached Figure Description

[0027] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0028] Figure 1 This is a flowchart illustrating a screw-driving method in one embodiment; Figure 2 This is a flowchart illustrating the screw-driving method in another embodiment; Figure 3 This is a schematic diagram of the current calibration sample image; Figure 4 This is a schematic diagram of the current screwdriver image; Figure 5 This is a schematic diagram of the current workpiece image; Figure 6 This is a schematic diagram of a screw-driving device in one embodiment; Figure 7 This is a schematic diagram of the screw-driving device in another embodiment; Figure 8 This is a schematic diagram of the robotic arm in one embodiment; Figure 9 This is a schematic diagram of the robotic arm in another embodiment; Figure 10 This is a schematic diagram of the robotic arm in another embodiment; Figure 11 This is a schematic diagram of the robotic arm in another embodiment; Figure 12 This is a schematic diagram of the robotic arm in another embodiment; Figure 13 This is a schematic diagram of the robotic arm in another embodiment; Figure 14 This is a schematic diagram of the screw-driving device in one embodiment; Figure 15 This is a schematic diagram of the screw-driving device in another embodiment; Figure 16 This is a schematic diagram of the screw-driving device in another embodiment; Figure 17 This is a schematic diagram of the screw-driving device in one embodiment; Figure 18 This is a schematic diagram of the screw-driving device in one embodiment; Figure 19 This is a schematic diagram of the electronic device.

[0029] Figure descriptions: 10. Robotic arm; 100. Electric screwdriver assembly; 200. Upper camera; 300. Retracting cylinder; 400. Straightening gripper; 500. Sleeve; 600. Lifting cylinder; 700. Lower camera; 800. Machine base; 900. Torque calibrator; 20. Screw-driving device; 21. Workpiece handling mechanism; 211. Drive motor; 212. Carrier; 213. First belt drive assembly; 214. Second belt drive assembly; 22. Grease application mechanism; 221. Glue dispensing module; 222. Glue dispensing cylinder; 23. Feeding mechanism; 231. Carrier tray. Detailed Implementation

[0030] To facilitate understanding of the present invention, a more complete description will be given below with reference to the accompanying drawings. Preferred embodiments of the invention are shown in the drawings. However, the invention can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided to provide a thorough and complete understanding of the disclosure of the invention.

[0031] It should be noted that when an element is referred to as being "fixed to" another element, it can be directly attached to the other element or there may be an intervening element. When an element is referred to as being "connected to" another element, it can be directly connected to the other element or there may be an intervening element. The terms "vertical," "horizontal," "left," "right," and similar expressions used herein are for illustrative purposes only and do not represent the only possible implementation.

[0032] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0033] See Figure 1 In one embodiment, a method for driving screws includes:

[0034] Step 110: Obtain the workpiece drawing information, and obtain the preset positioning point position of the positioning point and the preset screw hole position of the screw hole, wherein the positioning point and the screw hole are located on the workpiece.

[0035] The workpiece can be an electrical component, a hardware component, or a plastic component. The drawing information includes the workpiece's structure and dimensions, as well as the position and dimensions of each component. Based on the positional information of each component, the relative positional information between them can be obtained. This relative positional information includes the spacing between components and their relative orientation.

[0036] Locating points on the workpiece are used to position the workpiece during screw driving, and the position of the locating points determines whether the workpiece is correctly positioned. Threaded holes on the workpiece can be used for screw driving or for assembly in other processes. In one embodiment, for threaded holes requiring screw driving, the corresponding threaded holes can be marked in the drawing information for screw driving.

[0037] Step 120: Obtain the current workpiece image and get the current positioning point position.

[0038] The current workpiece image records the workpiece, and the current positioning point's location can be obtained based on the positioning points on the workpiece in the current workpiece image. The current workpiece image can be acquired through an image acquisition device, such as a camera or video recorder, which can capture image or video information of the workpiece. When the image acquisition device is capturing video information, the video information can be selected to obtain the image information containing the workpiece.

[0039] Step 130: Obtain the current screw hole deviation based on the current positioning point position and the preset positioning point position.

[0040] Since the relative positions of the locating point and the screw hole on the workpiece are fixed, the position of the screw hole can be determined based on the position of the locating point. The deviation direction and distance of the current locating point position relative to the preset locating point position are equivalent to the deviation direction and distance of the screw hole in the current workpiece image relative to the preset screw hole position. The current screw hole deviation includes the deviation in the deviation direction and / or the deviation in the deviation distance of the screw hole in the current workpiece image relative to the preset screw hole position. When both the deviation in the deviation direction and the deviation in the deviation distance are 0, the workpiece is correctly positioned and located at the preset workpiece position.

[0041] Step 140: Obtain the screw position based on the current screw hole deviation and the preset screw hole position.

[0042] Based on the current screw hole deviation and the preset screw hole position, the current position of the screw hole in the current workpiece image can be calculated, which is the position where the screw needs to be driven.

[0043] Step 150: Control the robot arm to screw the screws onto the workpiece according to the screw position.

[0044] Once the screw-driving position is determined, a corresponding screw-driving command is generated based on the position and sent to the robot arm. The robot arm then moves to the screw-driving position according to the command and controls the screw-driving device on the robot arm to drive the screw at the screw-driving position on the workpiece. The screw can be a slotted screw or a Phillips head screw, or a standard screw of M2.5-M4 or a screw with a spring washer, such as a slotted pan head screw with a spring washer.

[0045] In summary, the process involves first acquiring drawing information, then obtaining preset positioning point positions and preset screw hole positions based on the workpiece's positioning points and screw holes recorded in the drawing information; next, acquiring the current workpiece image in real time using an image acquisition device, and obtaining the current positioning point position based on the positioning points recorded in the current workpiece image; comparing the current positioning point position with the preset positioning point position to obtain the deviation of the positioning point, and then obtaining the positional deviation of the screw hole in the current workpiece image relative to the preset screw hole position, i.e., the current screw hole deviation; finally, based on the preset screw hole position and the current screw hole deviation, obtaining the current position of the screw hole in the current workpiece image, which is the screw-driving position, allows the robot arm to be controlled to move towards the screw-driving position and drive the screw at the screw-driving position on the workpiece.

[0046] See Figure 2 A method for driving screws, comprising:

[0047] Step 210: Obtain the drawing information of the workpiece and obtain the relative positional relationship between the screw holes on the workpiece; determine the preset positioning point position and the preset screw hole position based on the relative positional relationship of the screw holes.

[0048] The preset positioning point position includes the coordinates of the positioning point in the robot coordinate system, and the preset screw hole position includes the coordinates of the screw hole in the robot coordinate system. Since the relative positions of the positioning point and the screw hole are determined, after specifying the position of the positioning point in the robot coordinate system, that is, after specifying the preset positioning point position in the robot coordinate system, the coordinates of the screw hole, i.e., the preset screw hole position, can be determined in the robot coordinate system based on the relative positional relationship between the screw hole and the positioning point, and the relative positional relationship between the screw holes.

[0049] To obtain the preset positioning point position and preset screw hole position more flexibly, in one embodiment, the relative positional relationship of the screw hole positioning points is obtained according to the drawing information, wherein the relative positional relationship of the screw hole positioning points includes the relative positional information between the screw hole and the positioning point; based on the relative positional relationship of the screw hole positioning points and the relative positional relationship of the screw hole, the preset positioning point position and the preset screw hole position are determined. After determining the preset positioning point position, the preset screw hole position can be determined based on the relative positional relationship of the screw hole positioning points, or it can be determined based on the relative positional relationship of the screw hole.

[0050] The drawing information can be workpiece drawings in Pro / E, Solidworks, Inventor, or CAD software. To facilitate the acquisition of the screw hole location information (i.e., the preset screw hole location) from the drawing information, in one embodiment, the drawing information from Pro / E and / or Solidworks and / or Inventor software is converted into drawing information from CAD software. The hole location production data package is then sent to the equipment's host computer system via the CAD software.

[0051] Step 220: Send calibration motion information to the robot arm to move the robot arm to the preset calibration position; acquire the current calibration part image to obtain the current calibration part position; compare the current calibration part position with the preset calibration part position to confirm whether the vision system is normal.

[0052] The vision system acquires images via an image acquisition device on the robotic arm and sends these images to a host computer. A calibration piece is mounted on the machine tool and is fixed in place. The preset calibration piece position is determined based on its center point. When the image acquisition device is disassembled and reinstalled, its center will inevitably shift slightly. The proper functioning of the vision system depends on the image acquisition device on the robotic arm. If the image acquisition device shifts, and the robotic arm moves to the preset calibration position, the calibration piece in the vision system may not be in its intended position, requiring adjustment of the image acquisition device on the robotic arm.

[0053] In one embodiment, the current image of the calibration piece is acquired by an image acquisition device on a robotic arm at a preset calibration position. The image acquisition device on the robotic arm moves with the robotic arm, and the object recorded by the image acquisition device can be controlled by controlling the movement of the robotic arm.

[0054] To determine whether the vision system is functioning correctly, in one embodiment, the current calibration component position is compared with a preset calibration component position to confirm whether the vision system is functioning correctly. This includes: obtaining the current calibration component deviation based on the current and preset calibration component positions; comparing the current deviation with the preset deviation to confirm whether the vision system is functioning correctly. Further, comparing the current deviation with the preset deviation to confirm whether the vision system is functioning correctly includes: confirming the vision system is functioning correctly when the current deviation is less than or equal to the preset deviation; and confirming the vision system is malfunctioning when the current deviation is greater than the preset deviation. When the vision system is functioning correctly, the subsequent screw-driving step can be performed. When the vision system is malfunctioning, the positions of the robotic arm and the image acquisition device on the robotic arm are checked to confirm whether the malfunction is caused by an abnormal position of the image acquisition device on the robotic arm. Further, when the malfunction is caused by an abnormal position of the image acquisition device on the robotic arm, the image acquisition position of the image acquisition device on the robotic arm is calibrated to ensure that the current calibration component deviation is less than or equal to the preset deviation. Furthermore, when the current calibration deviation is less than or equal to the preset calibration deviation, the calibration of the image acquisition device's photographing position on the robotic arm is stopped.

[0055] The process involves moving the image acquisition device above the calibration piece for position calibration, calculating the current deviation of the calibration piece, and then combining this with the preset calibration piece position to obtain the actual position of the center of the image acquisition device.

[0056] To determine the current calibrator position based on the current calibrator image, in one embodiment, acquiring the current calibrator image and obtaining the current calibrator position includes: acquiring the current calibrator image; performing binarization processing on the current calibrator image to obtain a binarized calibrator image; and determining the calibrator center on the binarized calibrator image to obtain the current calibrator position. After binarization processing, the current calibrator image can be used to obtain a binarized calibrator image recording the outline of the calibrator. Therefore, the calibrator center can be determined based on the shape of the calibrator outline on the binarized calibrator image. To determine the calibrator center on the binarized calibrator image, in one embodiment, the binarized calibrator image is processed according to a center point function to obtain the center of the calibrator outline as the current calibrator position. The center point function is used to determine the center of the calibrator in the binarized calibrator image based on the calibrator outline. Since the shape of the calibrator can be various, to flexibly determine the center of the calibrator outline, in one embodiment, the center point function is determined based on the shape of the calibrator. For example, when the calibration piece is cylindrical and its outline is circular in the current calibration piece image, the corresponding center point function can be the Find Circle function. The Find Circle function determines the center of the calibration piece in the current calibration piece image and uses the center of the calibration piece in the current calibration piece image as the center of the calibration piece, which is the current position of the calibration piece.

[0057] In this process, the number of objects recorded in the binarized image of the calibration component is less than or equal to a preset number of objects, and the objects recorded in the binarized image of the calibration component include the calibration component. To more accurately determine the current location of the calibration component and avoid interference from objects other than the calibration component in the current calibration component image, the preset number of objects is further set to 1, meaning that the binarized image of the calibration component only records the calibration component.

[0058] To calculate the current calibration deviation, in one embodiment, the formula for calculating the calibration deviation is as shown in equation (1):

[0059]

[0060] In equation (1), d1 is the current calibration deviation, β1 is the pixel ratio of the image acquisition device on the robot arm, that is, the pixel ratio of the image acquisition device used to acquire the image of the current calibration part, and X 11 Y represents the first coordinate value of the current calibrator position in the image coordinate system of the current calibrator image. 11 Let (X) be the second coordinate value of the current calibrator position in the image coordinate system of the current calibrator image. 11 Y 11 X represents the coordinates of the current calibrator position in the image coordinate system of the current calibrator image. 10To preset the first coordinate value of the calibrator position in the image coordinate system of the current calibrator image, Y 10 To preset the second coordinate value of the calibrator position in the image coordinate system of the current calibrator image, (X 10 Y 10 ) represents the coordinates of the preset calibration piece position in the image coordinate system of the current calibration piece image. Wherein,

[0061] See Figure 3 In the image coordinate system of the current calibration part image, the origin is O1, the first coordinate axis is the X1 axis, and the second coordinate axis is the Y1 axis. The image coordinate system of the current calibration part image is also the robot's coordinate system. An origin sensor is located near the bit; when the bit rotates, it obscures the origin sensor, and this location is considered the bit's rotation origin, which is also the origin of the image coordinate system of the current calibration part image.

[0062] If the current calibration deviation d1 is less than or equal to the preset calibration deviation, it is determined that the current calibration position is consistent with the preset calibration position, the current calibration position meets the requirements, and the vision system is confirmed to be normal.

[0063] Because disassembly and maintenance are involved during screw-driving, components of the image acquisition device on the robotic arm may shift, such as the lens. To promptly calibrate the vision system, in one embodiment, if the current calibration deviation d1 is greater than a preset calibration deviation, the image acquisition device on the robotic arm is calibrated to make the current calibration deviation d1 less than or equal to the preset calibration deviation. This calibration can involve calibrating the installation position of the image acquisition device, the lens placement, or the image capture position. To confirm whether the vision system is functioning correctly after calibration, the image of the current calibration component is acquired again after calibration to recalculate the current calibration deviation.

[0064] Step 230: Acquire the current image of the screwdriver to obtain its current position. Compare the current screwdriver position with the preset screwdriver position to confirm whether the screwdriver's position is correct. The preset screwdriver position is a pre-selected fixed coordinate position for the lower camera of the robotic arm. During calibration, the screwdriver bit is moved to the center point of the lower camera. When the screwdriver bit is disassembled and reinstalled, it will take another picture at the camera position, that is, the operation of step 230 will be repeated to determine whether the screwdriver bit has shifted relative to the camera center.

[0065] Since the screwdriver is mounted on a robotic arm, the image acquisition device for capturing the current screwdriver image (hereinafter referred to as the lower camera) is different from the image acquisition device for capturing the current calibration part image, to facilitate image acquisition of the screwdriver on the robotic arm. Furthermore, due to the screwdriver tip on the robotic arm, the lower camera is further positioned on the machine tool to capture the current screwdriver image. The preset screwdriver position is pre-selected and stored in the host computer. When detecting the screwdriver position, the robotic arm is moved above the lower camera, causing the screwdriver bit to move to the center point of the lower camera. After the screwdriver bit is disassembled and reinstalled, a new image is taken at the center point of the lower camera to determine if the screwdriver bit has shifted relative to the camera center.

[0066] To confirm whether the screwdriver position is correct, in one embodiment, the current screwdriver position is compared with a preset screwdriver position to confirm whether the screwdriver position is correct. This includes: obtaining the current screwdriver deviation based on the current screwdriver position and the preset screwdriver position; and comparing the current screwdriver deviation with the preset screwdriver deviation to confirm whether the screwdriver position is correct. Further, comparing the current screwdriver deviation with the preset screwdriver deviation to confirm whether the screwdriver position is correct includes: when the current screwdriver deviation is less than or equal to the preset screwdriver deviation, the screwdriver position is confirmed to be correct; when the current screwdriver deviation is greater than the preset screwdriver deviation, the screwdriver position is confirmed to be abnormal.

[0067] To determine the current screwdriver position from a current screwdriver image, in one embodiment, obtaining the current screwdriver position by acquiring the current screwdriver image includes: acquiring the current screwdriver image; performing binarization processing on the current screwdriver image to obtain a binary image of the screwdriver; and determining the center of the screwdriver on the binary image of the screwdriver to obtain the current screwdriver position. Specifically, after binarization processing, a binary image of the screwdriver containing its outline can be obtained. Therefore, the center of the screwdriver can be determined based on the shape of the screwdriver's outline on the binary image. Since a sleeve is installed near the screwdriver by the robotic arm to attract the screw, the current screwdriver image may contain the sleeve. However, by binarizing the current screwdriver image, the interference from the sleeve in the current screwdriver image can be reduced, resulting in a smaller number of objects recorded in the obtained binary image of the screwdriver, and the objects recorded in the binary image of the screwdriver include the screwdriver itself. To determine the current screwdriver position, in one embodiment, the screwdriver center is determined on the binary image of the screwdriver to obtain the current screwdriver position. This includes: determining the center of the screwdriver's outline in the binary image using a center point function, and using the center of the screwdriver's outline as the current screwdriver position. Further, the center of the screwdriver's outline is determined in the binary image using the Find Circle function. Find Circle is a built-in LabVIEW function. It identifies the center of the screwdriver's outline by accurately locating the center of the circumcircle or incircle of the screwdriver tip.

[0068] It is important to understand that the center point function in step 230 and the center point function in step 220 can be the same function, but the objects processed in steps 230 and 220 are different, resulting in different results. Step 230 uses the center point function to process the current screwdriver image to determine the current screwdriver position, while step 220 uses the center point function to process the current calibration part image to determine the current calibration part position.

[0069] To calculate the current screwdriver deviation, in one embodiment, the formula for calculating the screwdriver deviation is as shown in equation (2):

[0070]

[0071] In equation (2), d2 is the current screwdriver deviation, β2 is the pixel ratio of the lower camera, and X 21 Y represents the first coordinate value of the current screwdriver position in the image coordinate system of the current screwdriver image. 21 Let X be the second coordinate value of the current screwdriver position in the image coordinate system of the current screwdriver image, (X... 21 Y 21 (x) represents the coordinates of the current screwdriver position in the image coordinate system of the current screwdriver image.20 To preset the first coordinate value of the screwdriver position in the image coordinate system of the current screwdriver image, Y 20 To preset the second coordinate value of the screwdriver position in the image coordinate system of the current screwdriver image, (X 20 Y 20 () represents the coordinates of the preset screwdriver position in the image coordinate system of the current screwdriver image.

[0072] See Figure 4 In the image coordinate system of the previous screwdriver image, the origin of the image coordinate system of the current screwdriver image is the origin O2, the first coordinate axis of the image coordinate system of the current screwdriver image is the X2 axis, and the second coordinate axis of the image coordinate system of the current screwdriver image is the Y2 axis.

[0073] If the current screwdriver deviation d2 is less than or equal to the preset screwdriver deviation, the current screwdriver position is determined to be consistent with the preset screwdriver position, and the current screwdriver position meets the requirements, confirming that the screwdriver position is normal. If the current screwdriver deviation d2 is greater than the preset screwdriver deviation, the screwdriver position is determined to be abnormal. Further, when the screwdriver position is abnormal, the screwdriver position is calibrated to ensure that the current screwdriver deviation d2 is less than or equal to the preset screwdriver deviation.

[0074] Step 240: Based on the current screwdriver image, obtain the current screwdriver deflection angle; compare the current screwdriver deflection angle with the preset screwdriver deflection angle to obtain screwdriver angle compensation. A sensor is installed on the side of the screwdriver as its origin. Each time it returns to the origin, the screwdriver tip position is always at a fixed angle. When the screwdriver tip is changed, the installation angle of the tip inside the electric screwdriver may deviate; when the tip returns to the origin position, there is a deflection angle, which is then captured by a camera.

[0075] The screwdriver angle compensation is used in the nut recognition algorithm to ensure the screwdriver is inserted into the nut slot. The current screwdriver deflection angle includes the angle between the screwdriver in the current screwdriver image and the preset calibration line. The screwdriver tip itself has a deflection angle, and the angle at which the nut slot is placed into the rectangular feed tray is random, also resulting in a certain deflection angle. Combining these two deflection angles allows for the calculation of the screwdriver tip's deflection angle relative to the nut slot.

[0076] To calculate the current screwdriver deflection angle, in one embodiment, the current screwdriver deflection angle is obtained based on the current screwdriver image, including: determining an deflection angle calculation line based on the current screwdriver image; and determining screwdriver angle compensation based on the angle between the deflection angle calculation line and a preset calibration line, as well as a preset screwdriver deflection angle. The preset calibration line can be a coordinate axis in the image coordinate system of the current screwdriver image. The deflection angle calculation line can be the centerline of the screwdriver or the edge line of the screwdriver's contour.

[0077] In one embodiment, determining the deflection angle calculation line based on the current screwdriver image includes: detecting the current screwdriver image using a boundary detection function to obtain a first screwdriver boundary line and a second screwdriver boundary line, wherein the first screwdriver boundary line and the second screwdriver boundary line are the boundaries of the screwdriver in the current screwdriver image, respectively; calculating the deflection angle calculation line based on the first screwdriver boundary line and the second screwdriver boundary line, wherein the deflection angle calculation line includes the center line between the first screwdriver boundary line and the second screwdriver boundary line. The boundary detection function can be the Find Straight Edge function. The center line between the first screwdriver boundary line and the second screwdriver boundary line, i.e., the deflection angle calculation line, can be obtained by processing the first screwdriver boundary line and the second screwdriver boundary line using the Bisecting Line function. Bisecting Line is also a LabVIEW function used to calculate the bisector between two specified lines.

[0078] See Figure 4 In the current image coordinate system of the screwdriver image, the Find Straight Edge function determines the first screwdriver boundary line as line AB and the second screwdriver boundary line as line CD, where the equation of line AB is (y B -y A )x+(x A -x B )y+(x A y B -y A x B ) = 0, the equation of line CD is (y D -y C )x+(x C -x D )y+(x C y D -y C x D =0. Then, the Bisecting Line function is used to determine the deflection angle calculation line as the straight line EF, where the equation of the straight line EF is (y = 0). F -y E )x+(x E -x F )y+(x E y F -y E x F ) = 0.

[0079] To calculate screwdriver angle compensation, in one embodiment, the screwdriver angle compensation is determined based on the angle between the angle calculation line and the preset calibration line. This includes: determining two angle calculation points on the angle calculation line and obtaining the coordinates of the two angle calculation points, where the angle calculation points are located on the angle calculation line; calculating the current screwdriver angle between the angle calculation line and the preset calibration line based on the coordinates of the two angle calculation points; and comparing the current screwdriver angle with the preset screwdriver angle to obtain the screwdriver angle compensation. The current screwdriver angle can be obtained by processing the coordinates of the two angle calculation points using inverse trigonometric functions. For example, the current screwdriver angle can be obtained using the formula α = actan((y F -y E ) / (x F -x E The calculation yields α, where α represents the current screwdriver deflection angle, (x... E ,y E (x) represents the coordinates of one of the two angle calculation points in the image coordinate system of the current screwdriver image. E This represents the coordinate value of one of the two deflection angle calculation points on the first coordinate axis in the image coordinate system of the current screwdriver image. E This represents the coordinate value of one of the two deflection angle calculation points on the second coordinate axis in the image coordinate system of the current screwdriver image, (x F ,y F (x) represents the coordinates of the other angle calculation point among the two angle calculation points in the image coordinate system of the current screwdriver image. F This represents the coordinate value of the other angle calculation point among the two angle calculation points in the image coordinate system of the current screwdriver image, on the first coordinate axis. F This represents the coordinate value of the other angle calculation point on the second coordinate axis in the image coordinate system of the current screwdriver image.

[0080] Step 250: Obtain the current workpiece image and get the current positioning point position; based on the current positioning point position and the preset positioning point position, get the current screw hole deviation; based on the current screw hole deviation and the preset screw hole position, calculate the screw driving position.

[0081] In step 250, the image acquisition device on the robot arm acquires an image of the workpiece to obtain the current workpiece image, and then sends the current workpiece image to the controller.

[0082] In one embodiment, the positioning point includes a Mark hole. Further, there are at least two Mark holes.

[0083] To facilitate the acquisition of the current workpiece image by the image acquisition device on the robotic arm, in one embodiment, after obtaining the screwdriver angle compensation and before acquiring the current workpiece image, the screw-driving method further includes sending deviation motion information to the robotic arm so that the robotic arm moves toward the workpiece.

[0084] To obtain the current positioning point location, in one embodiment, the current workpiece image is acquired to obtain the current positioning point location. This includes acquiring the current workpiece image; performing binarization processing on the current workpiece image to obtain a binarized workpiece image; and obtaining the center of the positioning point in the binarized workpiece image using a center point function to obtain the current positioning point location. Binarization processing reduces interference from other objects in the current workpiece image, ensuring that the binarized workpiece image only records the positioning point. Then, the center of the positioning point in the binarized workpiece image can be obtained more accurately using the center point function. The center point function is used to calculate the center of the positioning point based on its contour. The center point function includes the Find Circle function.

[0085] To calculate the current screw hole deviation, in one embodiment, the calculation formula for the current screw hole deviation is as shown in equation (3):

[0086]

[0087] In equation (3), d3 is the current screw hole deviation, β3 is the pixel ratio of the image acquisition device used to acquire the current workpiece image, and X 31 Y represents the first coordinate value of the current positioning point in the image coordinate system of the current workpiece image. 31 Let (X) be the second coordinate value of the current positioning point in the image coordinate system of the current workpiece image. 31 Y 31 (X) represents the coordinates of the current positioning point in the image coordinate system of the current workpiece image. 30 Y is the first coordinate value of the preset positioning point in the image coordinate system of the current workpiece image. 30 The second coordinate value of the preset positioning point position in the image coordinate system of the current workpiece image, (X 30 Y 30 ) represents the coordinates of the preset positioning point in the image coordinate system of the current workpiece image.

[0088] See Figure 5 In the current workpiece image coordinate system, the origin of the current workpiece image coordinate system is the origin O3, the first coordinate axis of the current workpiece image coordinate system is the X3 axis, and the second coordinate axis of the current workpiece image coordinate system is the Y3 axis.

[0089] In this application, the current calibration part image, the current screwdriver image, and the current workpiece image are all acquired under the assumption that the mounting base of the automatic screw fastening machine is in an ideal horizontal state, ignoring the slight tilt of the camera and lens during installation.

[0090] To ensure accurate screw driving, in one embodiment, after obtaining the current screw hole deviation, the screw driving method further includes: comparing the current screw hole deviation with a preset screw hole deviation to determine whether to calculate the screw driving position. The comparison between the current screw hole deviation and the preset screw hole deviation is checked to ensure the current screw hole deviation is not too large, preventing excessive deviation in the calculated screw driving position from affecting accuracy. Further, determining whether to calculate the screw driving position includes: if the current screw hole deviation is less than or equal to the preset screw hole deviation, calculating the screw driving position based on the current screw hole deviation and the preset screw hole position; if the current screw hole deviation is greater than the preset screw hole deviation, adjusting the workpiece position. The current screw hole deviation is calculated using the current positioning point position and the preset positioning point position. The positioning point is used to check if the workpiece is placed correctly; incorrect placement or tilting of the workpiece will lead to screw driving failure. The deviation of other hole positions is only calculated when the current screw hole deviation is small. If the current screw hole deviation is less than or equal to the preset screw hole deviation, it indicates that the current positioning point position is consistent with the preset positioning point position, the workpiece placement meets the requirements, and screw driving can proceed.

[0091] Step 260: Send the nut recognition motion information to the robot arm so that the robot arm moves toward the screw feeding mechanism; obtain the current nut image and get the current nut slot angle.

[0092] The current nut image is acquired by the image acquisition device of the robot arm; the current nut slot angle includes the angle between the nut slot and the preset calibration line.

[0093] In one embodiment, the screw feeding information can be determined based on the current position of the robot and the position of the feeding mechanism. Based on the current position of the robot and the position of the feeding mechanism, the route for the robot to move from the current position to the feeding mechanism can be calculated, and the robot can move to the screw feeding mechanism according to the route.

[0094] Step 270: Based on the screwdriver angle compensation and the current nut slot offset angle, obtain the motion insertion information; send the motion insertion information to the robot arm so that the robot arm moves according to the motion insertion information and inserts the screwdriver set on the robot arm into the nut slot.

[0095] The motion insertion information is used to control the screwdriver rotation. The screwdriver rotates according to the screwdriver angle compensation, so that the screwdriver can rotate to the corresponding position of the current nut slot angle, so that the screwdriver tip is aligned with the nut slot, making it convenient for the screwdriver to be inserted into the nut slot.

[0096] Screwdriver angle compensation is calculated based on the screwdriver's own deflection angle and the deflection angle of the nut in the rectangular feed tray (i.e., the current nut slot deflection angle). Combining these two deflection angles allows us to calculate the screwdriver tip's deflection angle relative to the nut slot. When both the screwdriver's own deflection angle and the nut slot's deflection angle are determined according to preset calibration lines, the difference between the two deflection angles can be used to determine the corresponding screwdriver angle compensation angle.

[0097] Step 280: Based on the screw-driving position, control the robotic arm to drive screws onto the workpiece.

[0098] To ensure a more secure installation of the screw on the workpiece, in one embodiment, controlling the robot to screw the screw onto the workpiece based on the screwing position includes: sending grease application motion information to the robot to move the robot toward the grease application mechanism and to apply grease to the screw held by the robot.

[0099] To ensure timely insertion of the greased screws into the workpiece, in one embodiment, after the screw is coated with grease, a screw-driving motion signal is sent to the robot arm, causing the robot arm to move towards the screw-driving position. When the robot arm reaches the screw-driving position, it is controlled to drive the screw in. For better screw driving, in one embodiment, there are at least two screw-driving positions, each corresponding to a hole position number. The hole position number is used to control the order in which the robot arm drives the screws at each screw-driving position.

[0100] There are two modes for screwing in screws: one is a combination of pre-locking and double-locking, and the other is a normal locking mode.

[0101] In the combined pre-locking and re-locking mode, screwing is divided into two main steps: pre-locking and re-locking. Pre-locking includes the following steps:

[0102] Step 281A, the threaded section, controls the screw to rotate into the screw hole and controls the screwing angle.

[0103] Step 282A, quick section, control the screw to rotate rapidly into the screw hole, and control the screw-in angle.

[0104] Step 283A, final slow speed section, control the screw to rotate slowly into the screw hole.

[0105] After pre-locking, the screw is not fully engaged in the screw hole, and no torque is generated between the screw and the screw hole. During re-locking, the torque is controlled to tighten the screw, and the pre-locked screw is engaged in the screw hole according to the actual torque. The combined pre-locking and re-locking mode is suitable for products where the hole position may shift when tightening the screw.

[0106] In normal lock mode, the following steps are included:

[0107] Step 281B, Insertion section: Control the screw to rotate into the screw hole and control the screw-in angle.

[0108] Step 282B, quick section, control the screw to rotate rapidly into the screw hole, and control the screw-in angle.

[0109] Step 283B, final slow speed section, control the screw to rotate slowly into the screw hole.

[0110] In normal locking mode, the screw is driven into the screw hole in one go, with the final torque being the set torque. Normal locking mode is suitable for products where the hole position will not shift when tightening the screw.

[0111] To achieve more flexible screw driving, in one embodiment, the screw driving method further includes acquiring manual adjustment data through a human-computer interaction device to adjust the screwdriver's driving parameters. Specifically, to address the various parameters encountered in practical applications, such as multiple torque programs, multi-step tightening, and various tightening strategies, a custom-developed user-friendly human-computer interaction interface is required. This interface interprets and adjusts parameters, including but not limited to whether the tightening force is too large or too small, the speed is too fast or too slow, and the time for determining whether the tightening is complete is too long or too short. When tightening abnormalities occur, some parameters need to be changed, since theoretically calculated values ​​will always deviate from actual values.

[0112] To provide users with a reference for adjusting screw-driving parameters, in one embodiment, screw-driving feedback information is acquired through a torque calibrator and / or an intelligent servo electric screwdriver, and sent to a human-machine interface device to provide feedback to the user on the screw-driving status. The screw-driving feedback information may be at least one of the following: screwdriver torque, number of screwdriver rotations, and screwdriver rotation direction.

[0113] To save costs, in one embodiment, the screw-driving method further includes: tracing product quality information based on screwdriver tightening data to determine whether the manufactured products meet product quality requirements. Specifically, the screwdriver tightening data includes the screwdriver's turning angle and torque, and the traceability of product quality information includes whether the screw hole is stripped, whether it is stuck, and whether the screw is too long or too short. In other words, data acquisition and analysis uses the servo screwdriver to record tightening data, and through the traceability function of the data acquisition system, engineers can use this data to determine whether the manufactured products meet product quality requirements.

[0114] To select more cost-effective screws, in one embodiment, the method further includes: comparing the actual screwdriver insertion angle with a preset insertion angle, and comparing the actual screwdriver torque with a preset torque, to confirm whether there are any problems with the screw or the threaded hole. The actual screwdriver insertion angle and actual torque are collected after the screwdriver completes one tightening operation, and then packaged and sent to the PLC in one go. The preset insertion angle and preset torque are sufficient to tighten the screw to the end and achieve the required torque. However, if the actual insertion angle is significantly greater or less than the preset insertion angle, or the actual torque is significantly greater or less than the preset torque, it indicates a problem with the screw or the threaded hole.

[0115] To align with the development of smart manufacturing, in one embodiment, screwdriver adjustment data is acquired via Ethernet, and the screwdriver parameters are adjusted based on this data. Specifically, the acquisition of screwdriver adjustment data is performed by a PLC. The PLC interprets the adjustment parameters by collecting the screwdriver control results, including but not limited to whether the tightening force is too high or too low, the speed is too fast or too slow, and the time taken to determine when the screw is fully tightened is too long or too short. Based on this, the screwdriver's built-in controller adjusts its parameters. In other words, to support the development of smart manufacturing, the data acquisition system can also connect to Ethernet, allowing for parameter modification, adjustment, recording, reading, and operational control of the screwdriver via the controller.

[0116] To facilitate screw driving and image acquisition, in one embodiment, the robotic arm includes at least one of a four-axis, five-axis, or six-axis robotic arm. In other embodiments, the robotic arm can also be a four-axis mechanism composed of four single-axis modules. The four-axis robotic arm has multiple dimensions, such as (X,Y,Z,R), where (X,Y,Z) represents the coordinate values ​​of the robotic arm tool in the OXYZ coordinate system with the base of the robotic arm as the origin (0,0,0), and R represents the rotation axis of the robotic arm, used to adjust the screwdriver and straighten the gripper of the screw. When space is limited, not rotating the screw at the correct angle will cause interference with the product, requiring rotation along the R axis to avoid this interference. Therefore, the four-axis robotic arm has high degrees of freedom and high flexibility. Using the robotic arm to pick up, place, and tighten screws improves process reliability.

[0117] To facilitate screwdriver rotation, in one embodiment, the screwdriver includes an intelligent servo electric screwdriver. Using the intelligent servo electric screwdriver as the core component for performing tightening tasks, the torque accuracy can reach ±5%. The intelligent servo electric screwdriver features multi-tasking capabilities, high precision, long lifespan, adjustable torque, adjustable speed, adjustable number of turns, and support for various alarm outputs. The intelligent servo electric screwdriver will rotate freely for a short period during the initial screw insertion stage, automatically guiding the screw to a higher position, resulting in more accurate screw insertion and more efficient tightening.

[0118] This application achieves rapid program switching and flexible production by inputting relevant parameters into the host computer via data entry and distribution, and distributing program instructions to the robotic arm control system, screwdriver control system, and PLC control system via communication protocols. This application utilizes a rapid switching function for screw driving, which includes rapid import of product data and rapid addition of equipment parameters. Rapid import of product data involves sending workpiece drawings to the host computer. Rapid addition of equipment parameters includes adding parameters such as screw feeding and fastening information for each screw hole. These parameters can be selected in the maintenance interface according to actual conditions. Screw feeding refers to the rectangular feeding tray that provides the screws, and fastening information refers to whether the screw to be screwed into the hole is screwed in directly to the end or pre-tightened (with a pre-tightening force less than the final tightening force) before final tightening.

[0119] This application utilizes an image acquisition device on a mobile robotic arm to photograph and compensate for calculation errors above each screw hole, obtaining the photographed coordinates and screw-locking position coordinates for each screw hole. Since the screw holes on the workpiece may have slight positional errors, the preset screw hole positions obtained through CAD software are only theoretical values. Therefore, before the initial production of the workpiece, each screw hole needs to be photographed individually, and the offset of the screw hole compensated to the preset screw hole position to obtain the screw-locking position, thereby achieving precise screw-locking.

[0120] Corresponding to the aforementioned application function implementation method embodiments, this application also provides a screw-driving device, controller, robot, screw-driving equipment, and corresponding embodiments.

[0121] See Figure 6 This application also provides a screw-driving device 60, which includes:

[0122] Drawing information acquisition module 610: used to acquire drawing information of workpiece, obtain the preset positioning point position of positioning point and the preset screw hole position of screw hole, wherein the positioning point and screw hole are located on the workpiece;

[0123] The current workpiece acquisition module 621 is used to acquire the current workpiece image and obtain the current positioning point position;

[0124] Position deviation calculation module 622: used to obtain the current screw hole deviation based on the current positioning point position obtained by the current workpiece acquisition module and the preset positioning point position obtained by the drawing information acquisition module;

[0125] Calculate working position module 623: Used to obtain the screw driving position based on the current screw hole deviation obtained by the calculate position deviation module and the preset screw hole position obtained by the obtain drawing information module;

[0126] Screw-driving module 630: Used to control the robot to drive screws onto the workpiece based on the screw-driving position obtained from the calculation working position module.

[0127] See Figure 7 In one embodiment, the screw-driving device includes:

[0128] The drawing information acquisition module 610 acquires the drawing information of the workpiece and obtains the relative positional relationship between the screw holes on the workpiece; based on the relative positional relationship of the screw holes, it determines the preset positioning point position and the preset screw hole position.

[0129] The vision system inspection module 640 sends calibration motion information to the robot arm to move the robot arm to the preset calibration position; acquires the current calibration part image to obtain the current calibration part position; and compares the current calibration part position with the preset calibration part position to confirm whether the vision system is normal.

[0130] The screwdriver inspection module 650 acquires the current screwdriver image and obtains the current screwdriver position; it then compares the current screwdriver position with the preset screwdriver position to confirm whether the screwdriver position is normal.

[0131] The angle compensation calculation module 660 obtains the current screwdriver deflection angle based on the current screwdriver image; it compares the current screwdriver deflection angle with the preset screwdriver deflection angle to obtain the screwdriver angle compensation.

[0132] The screw driving position calculation module 620 acquires the current workpiece image and obtains the current positioning point position; based on the current positioning point position and the preset positioning point position, it obtains the current screw hole deviation; based on the current screw hole deviation and the preset screw hole position, it calculates the screw driving position.

[0133] Screw recognition module 670: Sends screw recognition motion information to the robot arm to move the robot arm toward the screw feeding mechanism; acquires the current image of the screw and obtains the current screw slot angle;

[0134] Material handling module 680: Based on screwdriver angle compensation and the current nut slot offset angle, it obtains rotation insertion information; it sends motion insertion information to the robot arm, so that the robot arm moves according to the motion insertion information to insert the screwdriver mounted on the robot arm into the nut slot.

[0135] Screw-driving module 630: Controls the robotic arm to drive screws onto the workpiece according to the screw-driving position.

[0136] The screw-driving position calculation module 620 includes a current workpiece acquisition module 621, a position deviation calculation module 622, and a working position calculation module 623. The current workpiece acquisition module 621 acquires the current workpiece image to obtain the current positioning point position. The position deviation calculation module 622 calculates the current screw hole deviation based on the current positioning point position obtained from the current workpiece acquisition module and the preset positioning point position from the drawing information acquisition module. The working position calculation module 623 calculates the screw-driving position based on the current screw hole deviation obtained from the position deviation calculation module and the preset screw hole position obtained from the drawing information acquisition module.

[0137] The screw-driving device of this application acquires the workpiece drawing information through the drawing information acquisition module 610. Based on the relative positional relationship between each screw hole and positioning point in the drawing information, the position of the positioning point is specified in the robot coordinate system, and the position of the screw hole in the robot coordinate system is determined. Then, the image acquisition device on the robot acquires an image of the calibration part, obtaining the current calibration part image. The vision system inspection module 640 determines the current calibration part position in the current calibration part image using the Find Circle function, and checks whether the current calibration part position deviates excessively from the preset calibration part position, thereby checking whether the vision system is normal. If the vision system is normal, the robot is controlled to move to the preset calibration position. The screwdriver inspection module 650 acquires the current screwdriver image on the robot, determines the current screwdriver position in the current screwdriver image using the Find Circle function, and checks whether the current screwdriver position deviates excessively from the preset screwdriver position, thereby checking whether the screwdriver position is normal. The angle compensation calculation module 660 calculates the angle between the screwdriver's deflection calculation line and the preset calibration line based on the current screwdriver image, and compares this angle with the preset screwdriver deflection angle to determine the screwdriver angle compensation. The screw driving position calculation module 620 determines the screw driving position based on the current workpiece image. Once the screw driving position is determined, the nut recognition module 670 determines the current nut slot deflection angle based on the acquired current nut image. The material handling module 680 controls the robot to insert the screwdriver into the nut slot based on the screwdriver angle compensation and the current nut slot deflection angle. The screw driving module 630 drives the screw held by the screwdriver into the screw driving position on the workpiece.

[0138] This application also provides a controller that includes the screw-driving device described in any of the above embodiments. For example, the controller can be a host computer. The host computer controls the vision system to take pictures and calculates the screwdriver angle compensation, while the PLC executes the position positioning of the robotic arm.

[0139] See 8 to Figure 13This application also provides a robotic arm 10, which performs screw-driving on the workpiece according to the screw-driving method described in any of the above embodiments. See also 8 to... Figure 9 The robotic arm 10 includes an electric screwdriver assembly 100, an upper camera 200, a retraction cylinder 300, a gripping cylinder, a sleeve 500, and a lifting cylinder 600. The lifting cylinder 600 is mounted on one side of the electric screwdriver assembly 100, and the upper camera 200 is mounted on the other side of the electric screwdriver assembly 100, with the lens of the upper camera 200 facing downwards. In this embodiment, the upper camera 200 and the sleeve 500 are arranged parallel to each other. The retraction cylinder 300 is mounted on the power output end of the lifting cylinder 600, causing the power output end of the lifting cylinder 600 to drive the retraction cylinder 300 to move up and down. The gripping cylinder is mounted on the power output end of the retraction cylinder 300, controlling the retraction cylinder 300 and the gripping cylinder to move up and down along the Z-axis via the lifting cylinder 600, and the retraction cylinder 300 drives the gripping cylinder to move along the Y-axis. The gripping cylinder drives the gripping jaws to loosen screws. The socket 500 is mounted on the electric screwdriver head. The socket 500 is used to limit the screw position so as to identify the screw position and quickly align the screw when the electric screwdriver is used to drive the screw.

[0140] Further, see 14 to Figure 15 The screw-driving device 20 also includes a lower camera 700, which is fixed in a preset position on the table of the machine base 800 with its lower lens facing upwards. Since the upper camera 200's lens is positioned downwards, and the lower camera 700 is fixed in a preset position on the table of the machine base 800 with its lower lens facing upwards, the combined imaging effect of the upper camera 200 and the lower camera 700 allows for rapid identification of the screw's position. It should be noted that the electric screwdriver assembly 100 is equipped with a torque calibrator 900, enabling the electric screwdriver assembly 100 to automatically calibrate the output screw-driving torque.

[0141] The electric screwdriver can control the tightening force; the number of rotations, torque, and time can be set via specialized software. The PLC controls the displacement and speed of the robotic arm 10 when screwing in screws. For example, if an M3 screw has a pitch of 0.5mm and is screwed in 10 turns at a screwdriver speed of 600 RPM, then the displacement of the robotic arm 10 is 5mm, and the speed is 5mm / s = 0.5 * 10 / [10 / (60 / 6)]. The electric screwdriver assembly 100 includes an electric screwdriver, a straightening gripper 400, and a sleeve 500. The electric screwdriver is used to align the slot of the nut; the slot can be a slotted flathead or a Phillips head. The straightening gripper 400 ensures the perpendicularity of the screw shaft, and the sleeve 500 restricts the nut to ensure it remains within the tightening range of the electric screwdriver bit. Existing robotic arms 10 rely solely on vacuum to hold the nut portion of the screw, which cannot guarantee the perpendicularity of the screw. Compared with existing technologies, the robotic arm 10 of this application can better restrict the position and perpendicularity of the screw, ensuring the stability of the screw insertion.

[0142] See Figure 14 and Figure 16 This application also provides a screw-driving device 20, which includes the robotic arm 10 described in any of the above embodiments. To better perform screw driving, in one embodiment, the screw-driving device 20 further includes a machine base 800, a workpiece transport mechanism 21, a grease application mechanism 22, and a feeding mechanism 23. The robotic arm 10, workpiece transport mechanism 21, grease application mechanism 22, and feeding mechanism 23 are all mounted on the machine base 800. The workpiece transport mechanism 21 is used to transport the workpiece to a designated position for image acquisition and screw driving. The grease application mechanism 22 is used to apply grease to the screws gripped by the robotic arm 10, and the feeding mechanism 23 is used to provide various types of screws. The workpiece transport mechanism 21, the grease application mechanism, and the feeding mechanism 23 are respectively mounted on the worktable.

[0143] The workpiece handling mechanism 21 includes a drive motor 211, a carrier 212, and a first belt drive assembly 213 and a second belt drive assembly 214 arranged parallel to each other on the machine base 800. The power output end of the drive motor 211 is connected to the first belt drive assembly 213 and the second belt drive assembly 214, respectively, so that the drive motor 211 drives the first belt drive assembly 213 and the second belt drive assembly 214 to run synchronously. The carrier 212 is movably mounted on the first belt drive assembly 213 and the second belt drive assembly 214, so that the first belt drive assembly 213 and the second belt drive assembly 214 drive the carrier 212 to move relative to the machine base 800. In this embodiment, the drive motor 211 is a common speed-regulating motor. The first belt drive assembly 213 and / or the second belt drive assembly 214 are equipped with a stop cylinder and a lifting cylinder. The stop cylinder is used to control the carrier 212 to stop at a streamline, and the lifting cylinder is used to limit the carrier 212.

[0144] See Figure 17 Furthermore, the grease application mechanism 22 includes a glue dispensing module 221 and a glue dispensing cylinder 222. The glue dispensing module 221 is used to apply threadlocker to the screw threads. The glue dispensing cylinder 222 is used to control the amount of glue dispensed from the glue dispensing module 221.

[0145] See Figure 18 Furthermore, the feeding mechanism 23 includes a pen-shaped cylinder and two carrier trays 231, namely an upper carrier tray 231 and a lower carrier tray 231. Each carrier tray 231 is provided with multiple screw carriers 212. The pen-shaped cylinder is used to control the movement of the carrier trays 231. When the upper carrier tray 231 needs to be fed with screws, the upper carrier tray 231 moves in and the lower carrier tray 231 moves out; conversely, the lower carrier tray 231 moves in and the upper carrier tray 231 moves out.

[0146] The feeding mechanism 23 includes several rectangular feeding trays, each used to hold a specific type of screw. Before each task is executed, the screws are placed into the feeding trays, and the corresponding tray number is written to the host computer. The host computer sends the tray number corresponding to the screws that need to be screwed into each screw hole to the PLC based on the product model. The PLC then controls the screw picking process.

[0147] After the product model, the location of each screw hole, the source of the screw material, and the screw driving mode are entered into the human-machine interface (HMI), the HMI can store this data. When a specific product model needs to be processed, the various data for that product can be directly retrieved and sent to the controller. The data entry process requires manual confirmation of the product model, screw source, screw driving mode, etc.

[0148] In one embodiment, the screw-driving device performs the following process:

[0149] First, the PLC controls the robot arm to move to the lower camera position, and uses the lower camera to determine whether the upper mechanism of the robot arm has deviated. If a deviation occurs, an alarm is triggered. If no deviation occurs, the PLC controls the robot arm to move to the carrier transport mechanism, and uses the upper camera to take a picture of the product mark positioning point on the carrier. The position coordinate deviation between the current product mark point and the mark point recorded in the upper computer coordinate system is measured, and the actual coordinates of the holes where screws need to be drilled on the product are calculated.

[0150] Next, the PLC controls the robotic arm to move to the screw feeding mechanism and uses a camera to photograph the nut, calculating the angle of the nut's notch. After calculating the notch angle (in this embodiment, the nut's notch is a slotted notch), the PLC controls a servo motor to rotate the screwdriver bit to the angle corresponding to the notch. The PLC then controls the robotic arm to move to the picking position. When the robotic arm reaches the picking position, the PLC controls the gripping cylinder to actuate, causing the gripping claws to clamp the screw and pick up the material. At this point, the screwdriver bit and the nut's notch angle are aligned, and they are tightly pressed together.

[0151] Next, the PLC controls the robot arm to move to the grease application mechanism. When the PLC controls the robot arm to move to the grease application mechanism, the thread side of the screw is in close contact with the grease brush. The grease brush rotates once, so that the thread of the screw is covered with grease.

[0152] Finally, the PLC controls the robotic arm to move to the screw feeding mechanism and, according to the pre-set screw hole number, locks the screws one by one.

[0153] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated further here.

[0154] Figure 19This is a schematic diagram of the structure of an electronic device shown in an embodiment of this application.

[0155] See Figure 19 The electronic device 1900 includes a memory 1910 and a processor 1920.

[0156] The processor 1920 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0157] Memory 1910 may include various types of storage units, such as system memory, read-only memory (ROM), and permanent storage devices. ROM may store static data or instructions required by processor 1920 or other modules of the computer. Permanent storage devices may be read-write storage devices. Permanent storage devices may be non-volatile storage devices that retain stored instructions and data even when the computer is powered off. In some embodiments, permanent storage devices use mass storage devices (e.g., magnetic or optical disks, flash memory) as permanent storage devices. In other embodiments, permanent storage devices may be removable storage devices (e.g., floppy disks, optical drives). System memory may be a read-write storage device or a volatile read-write storage device, such as dynamic random access memory. System memory may store some or all of the instructions and data required by the processor during operation. Furthermore, memory 1910 may include any combination of computer-readable storage media, including various types of semiconductor memory chips (e.g., DRAM, SRAM, SDRAM, flash memory, programmable read-only memory), and disks and / or optical disks may also be used. In some embodiments, the memory 1910 may include a removable storage device that is readable and / or writable, such as a laser disc (CD), a read-only digital versatile optical disc (e.g., DVD-ROM, dual-layer DVD-ROM), a read-only Blu-ray disc, an ultra-high density optical disc, a flash memory card (e.g., SD card, mini SD card, Micro-SD card, etc.), a magnetic floppy disk, etc. Computer-readable storage media do not contain carrier waves or transient electronic signals transmitted wirelessly or via wired connections.

[0158] The memory 1910 stores executable code, which, when processed by the processor 1920, can cause the processor 1920 to execute some or all of the methods described above.

[0159] Furthermore, the method according to this application can also be implemented as a computer program or computer program product, which includes computer program code instructions for performing some or all of the steps in the method described above.

[0160] Alternatively, this application may be implemented as a computer-readable storage medium (or a non-transitory machine-readable storage medium or a machine-readable storage medium) storing executable code (or computer program or computer instruction code) thereon, which, when executed by a processor of an electronic device (or server, etc.), causes the processor to perform part or all of the steps of the methods described above according to this application.

[0161] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this invention patent should be determined by the appended claims.

Claims

1. A method for driving screws, characterized in that, include: Obtain the workpiece drawing information to obtain the preset positioning point position of the positioning point and the preset screw hole position of the screw hole, wherein the positioning point and the screw hole are located on the workpiece; Send calibration motion information to the robotic arm so that the robotic arm moves to a preset calibration position; The current image of the calibration piece is obtained to determine the current position of the calibration piece, wherein the current image of the calibration piece is used to acquire the calibration piece at the preset calibration position through the image acquisition device of the robotic arm; The current calibration piece position is compared with the preset calibration piece position to confirm whether the vision system is functioning properly; Obtain the current screwdriver image and determine the current screwdriver position; Compare the current screwdriver position with the preset screwdriver position to confirm whether the screwdriver position is normal; Based on the current screwdriver image, the current screwdriver deflection angle is obtained; The current screwdriver angle is compared with the preset screwdriver angle to obtain screwdriver angle compensation; Obtain the current workpiece image to get the current positioning point position; The current screw hole deviation is obtained based on the current positioning point position and the preset positioning point position; The screw-driving position is obtained based on the current screw hole deviation and the preset screw hole position; Send a cap recognition movement message to the robotic arm so that the robotic arm moves toward the screw feeding mechanism; Obtain the current image of the nut and obtain the current nut slot angle; Based on the screwdriver angle compensation and the current nut slot deflection angle, the rotation into the slot information is obtained; Send motion insertion information to the robotic arm so that the robotic arm moves according to the motion insertion information and inserts the screwdriver mounted on the robotic arm into the slot of the nut; Based on the screw location, control the robotic arm to screw the screws onto the workpiece; The step of comparing the current calibration component position with the preset calibration component position to confirm whether the vision system is functioning properly includes: The current calibration component deviation is obtained based on the current calibration component position and the preset calibration component position; The current calibration deviation is compared with the preset calibration deviation to confirm whether the vision system is functioning correctly. The step of obtaining the current image of the calibration piece and obtaining the current location of the calibration piece includes: Acquire the image of the current calibration piece; The current calibration image is binarized to obtain a binarized calibration image; The center of the calibration piece is determined on the binary image of the calibration piece to obtain the current position of the calibration piece; the formula for calculating the deviation of the current calibration piece is shown in equation (1): （1） in, The deviation of the current calibration piece, The pixel ratio of the image acquisition device on the robotic arm, that is, the pixel ratio of the image acquisition device used to acquire the image of the current calibration piece. Let the first coordinate value be the position of the current calibration piece in the image coordinate system of the current calibration piece image. Let be the second coordinate value of the current calibrator position in the image coordinate system of the current calibrator image. , ( ) represents the coordinates of the current calibration component's position in the image coordinate system of the current calibration component image. Let the first coordinate value be the preset coordinate value of the position of the calibration piece in the image coordinate system of the current calibration piece image. The second coordinate value of the preset calibration piece position in the image coordinate system of the current calibration piece image, ( , () represents the coordinates of the preset calibration piece position in the image coordinate system of the current calibration piece image; If the current calibration deviation d1 is less than or equal to the preset calibration deviation, it is determined that the current calibration position is consistent with the preset calibration position, the current calibration position meets the requirements, and the vision system is confirmed to be normal. If the current calibration deviation d1 is greater than the preset calibration deviation, the image acquisition device on the robotic arm is calibrated so that the current calibration deviation d1 is less than or equal to the preset calibration deviation. The step of obtaining the workpiece drawing information to obtain the preset positioning point position of the positioning point and the preset screw hole position of the screw hole includes: Obtain the drawing information of the workpiece to obtain the relative positional relationship between the screw holes on the workpiece; Based on the relative positional relationship of the screw holes, the preset positioning point position and the preset screw hole position are determined, wherein the preset positioning point position includes the coordinate value of the positioning point in the robot coordinate system, and the preset screw hole position includes the coordinate value of the screw hole in the robot coordinate system; The step of obtaining the current screwdriver image and determining the current screwdriver position includes: Get the current screwdriver image; The current screwdriver image is binarized to obtain a binary image of the screwdriver; The center of the screwdriver is determined on the binary image of the screwdriver to obtain the current position of the screwdriver. Determining the screwdriver center on the binary image of the screwdriver to obtain the current screwdriver position includes: The center of the screwdriver's outline is determined in the binary image of the screwdriver using the center point function, and the center of the screwdriver's outline is taken as the current position of the screwdriver. The step of comparing the current screwdriver position with the preset screwdriver position to confirm whether the screwdriver position is normal includes: The current screwdriver deviation is obtained based on the current screwdriver position and the preset screwdriver position; Compare the current screwdriver deviation with the preset screwdriver deviation to confirm whether the screwdriver position is normal; The step of comparing the current screwdriver deviation with the preset screwdriver deviation to confirm whether the screwdriver position is normal includes: When the current screwdriver deviation is less than or equal to the preset screwdriver deviation, it is confirmed that the screwdriver position is normal. When the current screwdriver deviation is greater than the preset screwdriver deviation, the screwdriver position is confirmed to be abnormal.

2. A screw-driving device, characterized in that, The method for performing the screw driving as described in claim 1 includes: Drawing information acquisition module: used to acquire drawing information of workpiece, obtain the preset positioning point position of positioning point and the preset screw hole position of screw hole, wherein the positioning point and the screw hole are located on the workpiece; Current workpiece acquisition module: used to acquire the current workpiece image and obtain the current positioning point position; Position deviation calculation module: used to obtain the current screw hole deviation based on the current positioning point position obtained by the current workpiece acquisition module and the preset positioning point position obtained by the drawing information acquisition module; The working position calculation module is used to obtain the screw driving position based on the current screw hole deviation obtained by the position deviation calculation module and the preset screw hole position obtained by the drawing information acquisition module. Screw driving module: used to control the robot to drive screws onto the workpiece based on the screw driving position obtained by the calculation working position module; The vision system inspection module is used to send calibration motion information to the robot arm so that the robot arm moves to the preset calibration position. Obtain the current image of the calibration piece to determine its current position; Compare the current calibration piece position with the preset calibration piece position to confirm whether the vision system is functioning properly; Screwdriver inspection module: Used to acquire the current screwdriver image and obtain the current screwdriver position; compare the current screwdriver position with the preset screwdriver position to confirm whether the screwdriver position is normal; Angle compensation calculation module: used to obtain the current screwdriver deflection angle based on the current screwdriver image; compare the current screwdriver deflection angle with the preset screwdriver deflection angle to obtain screwdriver angle compensation; Screw recognition module: Used to send screw recognition motion information to the robot arm so that the robot arm moves towards the screw feeding mechanism; to obtain the current image of the screw and obtain the current screw slot angle; Material handling module: It is used to obtain rotation insertion information based on screwdriver angle compensation and the current nut slot angle; and send motion insertion information to the robot arm so that the robot arm moves according to the motion insertion information to insert the screwdriver set on the robot arm into the nut slot.

3. A controller, characterized in that, Includes the screw-driving device as described in claim 2.

4. A robotic arm, characterized in that, The robotic arm performs screw-driving on the workpiece according to the screw-driving method described in claim 1.

5. A screw-driving device, characterized in that, Including the robotic arm as described in claim 4.

6. An electronic device, characterized in that, include: processor; as well as A memory having executable code stored thereon, which, when executed by the processor, causes the processor to perform the method as described in claim 1.

7. A computer-readable storage medium having executable code stored thereon, which, when executed by a processor of an electronic device, causes the processor to perform the method of claim 1.

Images