Screw locking system and locking method thereof

Abstract

The invention relates to the technical field of mechanical assembly, in particular to a screw locking system and a locking method thereof, the system comprises a moving frame, a multi-axis mechanical arm, a locking device, a shooting device, a feeding device and a control processing device, and the locking device comprises a locking mechanism and a beak. The locking method comprises the following steps: S1, obtaining a training model; s2, calibrating the eyes outside the hand by the shooting device; s3, obtaining the position change from the tail end of the arm to the beak; s4, planning a moving path of the moving frame; s5, hole position prediction is conducted on the to-be-locked image obtained by the shooting device through the training model, and position movement information of the multi-axis mechanical arm is obtained; s6, the multi-axis mechanical arm is controlled to move, and the locking device is made to conduct locking operation at the current position; s7, if the movable frame is located at the designated ending position, all locking work is completed; otherwise, controlling the moving frame to move to the next position according to the planned path, and then returning to S5. According to the device, the locking operation efficiency can be improved, and the locking operation safety can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of mechanical assembly, and more specifically, to a screw locking system and a screw locking method thereof. Background Art

[0002] In the application of screw locking, most of the screw locking devices are used to tighten the screws in the preset holes of the product to complete the locking operation. In order to improve production efficiency, the prior art also provides a mobile device with vision to drive the screw locking device to move and control the screw locking device to lock according to the identified hole position, which can reduce the labor intensity of the operator and avoid the phenomenon of screw leakage. However, in the actual locking process, in addition to the preset hole positions for screw locking, the holes set on the product may also have hole positions with other shapes, or have hole positions that are different from the state to be locked. The existing mobile device with vision is difficult to identify different hole positions, so that the screw locking device may tighten the screws in the hole positions other than the preset hole positions, which may cause damage to the screw locking device or the product, which is not conducive to further improving the locking efficiency. Summary of the invention

[0003] The purpose of the present invention is to overcome the difficulty in identifying different hole positions during locking operations, and to provide a screw locking system and a locking method thereof, which can improve the efficiency of the locking operation and can improve the safety of the locking operation.

[0004] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0005] A screw locking system is provided, comprising a movable frame, wherein a plurality of multi-axis robotic arms are connected to the movable frame via a base, a locking device is connected to the end of each multi-axis robotic arm, and a shooting device and a feeding device which are arranged in a one-to-one correspondence with the locking devices are also connected to the movable frame; a control processing device is also provided, wherein the movable frame, the multi-axis robotic arms, the locking device, the shooting device and the feeding device are all communicatively connected to the control processing device; wherein the locking device comprises a locking mechanism and a bird's beak connected to the locking mechanism, the locking end of the locking mechanism extends into the bird's beak; and the discharge end of the feeding device is communicated with the bird's beak.

[0006] Preferably, the feeding device includes a mounting base installed on the mobile frame, the mounting base is connected to a feeding track, a transport channel is provided in the mounting base, and the discharge end of the transport channel is connected to a nozzle; it also includes a sliding mechanism, the sliding end of the sliding mechanism is connected to a transport part, and the sliding mechanism can drive the transport part to communicate with the feeding track and the transport channel; the nozzle and the bird's beak are connected through a pipe.

[0007] The present invention also provides a screw locking method, comprising the following steps:

[0008] S1. Obtain multiple sampling images of the area to be locked by sampling, and mark all the hole positions in all the sampling images to obtain a labeled data set and train it to obtain a training model;

[0009] S2. The camera is calibrated outside the hand by the corresponding locking device;

[0010] S3. placing the mobile frame in the area to be locked, and setting a fixed sharp point at any position in the area to be locked, and then controlling the multi-axis robot arm to drive the locking device to reach the position of the fixed sharp point multiple times in different positions, and calculating the position transformation from the end of the arm to the beak;

[0011] S4. Move the mobile rack to the initial designated position and plan the moving path of the mobile rack;

[0012] S5. The photographing device obtains the image to be locked at the current position of the area to be locked, and then uses the training model of step S1 to predict the hole position of the image to be locked, and performs mask processing to obtain a mask image with a specified hole position, and then obtains a circular hole information matrix of the specified hole position in the mask image; then, combined with the calibration of step S2, the circular hole information matrix is ​​converted to obtain the position movement information of the multi-axis robot arm;

[0013] S6. Control the multi-axis robot to move according to the position movement information of the multi-axis robot, so that the locking device moves to the top of the designated hole, the feeding device transmits the screw to the locking device, and the locking device performs the locking operation at the current position;

[0014] S7. If the current position of the mobile rack is the end designated position in the planned path, all locking operations are completed; otherwise, the mobile rack is controlled to move to the next position according to the planned path, and then returns to step S5.

[0015] Furthermore, in step S1, the Yolo algorithm is used to train the annotated data set; the annotated data set includes feature categories of each hole position and position information of each hole position; wherein the feature categories include normal holes, special-shaped holes, and locked holes; the position information includes the vertex coordinates of the rectangular box corresponding to each hole position in its corresponding image coordinate system.

[0016] Furthermore, in step S3, the position transformation formula from the end of the arm to the beak is:

[0017] Tip T TcpTcp T End End T Base = Tip T Base ;

[0018] In the formula, Tip T Tcp represents the homogeneous transformation matrix from the beak to the fixed point position, Tcp T End represents the homogeneous transformation matrix from the end of the arm to the beak position, End T Base represents the homogeneous transformation matrix from the base to the arm end position, Tip T Base Represents the homogeneous transformation matrix from the base to the fixed cusp position; wherein:

[0019]

[0020] In the formula, T represents the position information of the homogeneous transformation matrix, [x, y, z] represents the position of the origin of the coordinate system of the bird's beak in the coordinate system of the end of the arm;

[0021] Therefore, we can get:

[0022] Tcp T End End T Base = Tip T Base ;

[0023] By moving the multi-axis robotic arm multiple times, at least three different postures are obtained and a matrix equation group can be obtained, and the position transformation from the end of the arm to the beak can be obtained by the matrix equation group; wherein the matrix equation group is expressed as:

[0024]

[0025] In the formula, End T Base_i represents the ith position of the origin of the coordinate system of the end of the arm in the coordinate system of the base.

[0026] Furthermore, in step S2, the homogeneous transformation matrix from the base to the camera position is obtained by calibration of the eye outside the hand: Cam T Base , and get the internal parameter matrix Intr Cam ;in:

[0027]

[0028] In the formula, f x 、fy represent the focal length of the camera in the x direction and the y direction, respectively, x 、u y Represents the pixel density in the x direction and y direction respectively;

[0029] Step S5 includes the following steps:

[0030] S51. The shooting device obtains the image to be locked in the area to be locked, and then uses the training model to predict the hole position of the image to be locked, and obtains the corresponding rectangular frame of the feature category of the irregular hole and the normal hole in the image to be locked;

[0031] S52. Output the center points of the corresponding rectangular boxes of all the irregular holes to obtain the circular hole information matrix of the irregular holes;

[0032] S53. Mask the locked image so that the contents of the corresponding rectangular frame of the normal hole are retained and the rest are replaced with a solid color background to obtain the mask image and obtain the circular hole information matrix of all normal holes in the mask image;

[0033] S54. Convert the circular hole information matrix of the normal hole to obtain the position movement information of the multi-axis robot arm.

[0034] Further, in step S52, the circular hole information matrix of the irregular-shaped holes is a matrix of six columns and n rows, where n is the number of irregular-shaped holes identified, and each row is represented as [x, y, z, r, p, y];

[0035] In step S53, the circular hole information matrix of the normal hole is a matrix of three columns and N rows, and each row is in the form of:

[0036] [x px y px 1];

[0037] In the formula, x px is the horizontal coordinate of the normal hole in the image coordinate system to be locked, y px is the vertical coordinate of the normal hole in the coordinate system of the image to be locked; the minimum depth value depth inside the rectangular frame of each normal hole is taken as the distance between the center of the normal hole and the optical center of the shooting device, which is 1;

[0038] In step S54, each row P in the circular hole information matrix of the normal hole is obtained. Hole And the minimum depth value depth inside the rectangular frame of each normal hole, calculate the normal hole coordinates of each normal hole in the coordinate system of the camera:

[0039]

[0040] P Hole=[x px y px 1] T ;

[0042] The homogeneous transformation matrix from the camera coordinate system to the normal hole position is:

[0043]

[0044] The homogeneous transformation matrix from the coordinate system of the base to the normal hole position is:

[0045] Hole T Base = Hole T Cam Cam T Base ;

[0046] Then, the homogeneous transformation matrix from the coordinate system of the base to the normal hole position is converted into [x base ,y base ,z base ,r base ,p base ,y base ] format, which is the format of the position vector plus the rotation vector in the coordinate system of the base, that is, the movement information of the multi-axis robotic arm.

[0047] Further, step S4 includes the following steps:

[0048] S41. Setting the number of mobile racks: Select n mobile racks for operation, and then divide the area to be locked into S working areas, where: The working area of ​​the first mobile rack is the n working areas closest to the starting position, the working area of ​​the second mobile rack is the n-1 working areas closest to the starting position in the remaining working areas, and so on;

[0049] S42. The number and position distribution of the multi-axis robotic arms are set: the number of the multi-axis robotic arms is set to k, the working space diameter of the multi-axis robotic arms is set to d, and the width of the area to be locked is set to D, and the three satisfy the following relationship:

[0050]

[0051] S43. The mobile setting of the mobile rack: When the mobile rack is responsible for at least two working areas, the mobile rack first uses v 1 The moving frame moves to the working area farthest from the starting position at a speed of v. 2 Move towards the starting position at a speed of distance, and then perform the locking operation at the next position, and so on; where v 1 >v 2 .

[0052] Furthermore, in step S42, the movement setting of the multi-axis robot arm is also included: the multi-axis robot arm locks the designated hole positions in the working area one by one with a serpentine motion trajectory.

[0053] Further, in step S43, when the mobile rack completes all the work areas for which it is responsible, the mobile rack moves to v 1 Return to the starting position at a speed of

[0054] Compared with the prior art, the present invention has the following beneficial effects:

[0055] The present invention includes a screw locking system and a locking method thereof, wherein a shooting device can be used to shoot an image of an area to be locked, and a control processing device can be used to extract all hole position information from the image to be locked, and can identify a designated hole position to be locked from all hole position information, and then convert the information of the designated hole position into position movement information of a multi-axis mechanical arm, so that the locking device is located above the corresponding hole position, and then the screw is fed to the bird's mouth of the locking device through a feeding device, and the locking mechanism performs screw locking; by identifying the setting of the designated hole position to be locked, it is possible to avoid the locking device from performing screw locking on hole positions other than the designated hole position, thereby preventing the locking device or the area to be locked from being damaged; the setting of the mobile frame can be used by the screw locking system to perform screw locking operations on a larger area to be locked, thereby improving the efficiency of the locking operation, wherein, by planning the moving path of the mobile frame, it is possible to improve the utilization rate of the multi-axis mechanical arm, improve the efficiency of the locking operation, and improve the safety of the locking operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 It is a structural schematic diagram of a screw locking system of the present invention;

[0057] Figure 2 It is a structural schematic diagram of the locking device of the present invention;

[0058] Figure 3 It is a structural schematic diagram of the feeding device of the present invention;

[0059] Figure 4 A flowchart of a locking method of a screw locking system of the present invention;

[0060] Figure 5 A schematic diagram of the working area of ​​multiple mobile racks in step S41 of the present invention;

[0061] Figure 6Schematic diagram of the position distribution and activity of the multi-axis robot arm relative to the mobile frame in step S42 of the present invention.

[0062] The illustrations are as follows:

[0063] 1-mobile frame, 11-roller mechanism, 2-base, 3-multi-axis robotic arm, 31-arm end, 4-locking device, 41-bird's beak, 411-feeding port, 42-locking mechanism, 421-rotation mechanism, 422-slide rail and slider mechanism, 43-force sensor, 5-shooting device, 6-feeding device, 61-mounting seat, 611-first slide slot, 612-second slide slot, 62-feeding track, 63-nozzle, 64-sliding mechanism, 641-cylinder, 642-slide key, 65-conveying unit. DETAILED DESCRIPTION

[0064] The present invention is further described below in conjunction with specific implementation methods. The accompanying drawings are only used for exemplary descriptions and are only schematic diagrams, not actual drawings, and cannot be understood as limiting this patent; in order to better illustrate the embodiments of the present invention, some parts of the accompanying drawings may be omitted, enlarged or reduced, and do not represent the size of the actual product; for those skilled in the art, it is understandable that some well-known structures and their descriptions in the accompanying drawings may be omitted.

[0065] The same or similar numbers in the drawings of the embodiments of the present invention correspond to the same or similar parts; in the description of the present invention, it should be understood that if the terms "upper", "lower", "left", "right" and the like indicate directions or positional relationships based on the directions or positional relationships shown in the drawings, it is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific direction, be constructed and operated in a specific direction. Therefore, the terms describing the positional relationship in the drawings are only used for illustrative purposes and cannot be understood as limitations on this patent. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to specific circumstances.

[0066] Example 1

[0067] like Figures 1 to 3 The figure shows an embodiment of a screw locking system of the present invention, comprising a mobile frame 1, a plurality of multi-axis robotic arms 3 are connected to the mobile frame 1 through a base 2, the arm end 31 of each multi-axis robotic arm 3 is connected to a locking device 4, and the mobile frame 1 is also connected to a shooting device 5 and a feeding device 6 which are arranged one-to-one corresponding to the locking device 4; the figure also comprises a control processing device, and the mobile frame 1, the multi-axis robotic arms 3, the locking device 4, the shooting device 5, and the feeding device 6 are all communicatively connected to the control processing device; wherein the locking device 4 comprises a locking mechanism 42 and a bird's beak 41 connected to the locking mechanism 42, and the locking end of the locking mechanism 42 extends into the bird's beak 41; the discharge end of the feeding device 6 is connected to the bird's beak 41.

[0068] like Figure 1 As shown, the bottom of the mobile frame 1 is connected to a roller mechanism 11, which is connected to the switch for communication. The setting of the roller mechanism 11 can realize the movement of the mobile frame 1. The base 2 is fixedly connected to the bottom of the mobile platform of the mobile frame 1, and the multi-axis mechanical arm 3 can drive the locking device 4 to move in the horizontal direction. The shooting device 5 includes a depth camera and a mounting bracket. The depth camera is connected to the mobile platform of the mobile frame 1 through the mounting bracket. The shooting range of the depth camera can cover the activity range of the corresponding locking device 4.

[0069] like Figure 2 As shown, the locking mechanism 42 includes a rotating mechanism 421 and a slide rail slider mechanism 422 which are communicatively connected to the control processing device. The slide rail slider mechanism 422 is vertically arranged, and the fixed end of the rotating mechanism 421 is connected to the slider end of the slide rail slider mechanism 422. The rotating end of the rotating mechanism 421 extends into the bird's beak 41. The locking device 4 also includes a force sensor 43 connected to the slide rail end of the slide rail slider mechanism 422. The force sensor 43 can be used to detect the change in the force of the locking mechanism 42 acting on the arm end 31 during the locking process. The slide rail slider mechanism 422 can drive the rotating mechanism 421 to move up and down, and the rotation of the rotating mechanism 421 can drive the screw located in the bird's beak 41 to rotate and lock. It should be noted that the rotating mechanism 421 is a prior art and will not be described in detail here.

[0070] like Figure 1 and Figure 3 As shown, the feeding device 6 includes a mounting seat 61 mounted on the moving table of the moving frame 1, the top of the mounting seat 61 is connected to a feeding track 62, a conveying channel is provided in the mounting seat 61, and a nozzle 63 is connected to the discharge end of the conveying channel. The nozzle 63 is connected to the feed port 411 of the bird's mouth 41 through a pipeline, as shown in FIG. Figure 2 The feeding device 6 further comprises a sliding mechanism 64, a sliding end of the sliding mechanism 64 is connected to a conveying portion 65, and the sliding mechanism 64 can drive the conveying portion 65 to communicate with the feeding track 62 and the conveying channel.

[0071] In this embodiment, the sliding mechanism 64 includes a cylinder 641, a first slide groove 611 is provided on the top of the mounting seat 61, and a slide key 642 is slidably provided in the first slide groove 611; the cylinder seat of the cylinder 641 is fixedly connected to the mounting seat 61, and the piston rod of the cylinder 641 extends into the mounting seat 61 and is connected to the slide key 642. A second slide groove 612 is also provided on the top of the mounting seat 61, and the second slide groove 612 is connected to the feed end of the conveying channel, and the conveying part 65 is fixedly connected to the slide key 642, and the conveying part 65 is located at the second slide groove 612. In this embodiment, the conveying part 65 is provided with a slot for conveying screws.

[0072] Specifically, the first slide groove 611 includes a first sliding section and a second sliding section arranged at an angle to the first sliding section, the first end of the first sliding section is a material receiving end point, the second end of the first sliding section is connected to the first end of the second sliding section, and the second end of the second sliding section is a feeding end point. The second slide groove 612 is a slide groove extending along a straight line, the first sliding section and the second slide groove 612 are arranged parallel to each other, and the second sliding section is bent in a direction away from the second slide groove 612, so that when the sliding key 642 slides to the feeding end point, it gradually moves away from the second slide groove 612.

[0073] When the feeding device 6 needs to transport screws, the piston rod of the cylinder 641 retracts, and the sliding key 642 is located at the material receiving end point. The slot of the conveying part 65 overlaps with the projection of the second slide groove 612 in the vertical direction, and the slot of the conveying part 65 is connected to the feeding track 62, so the screw can be transported from the feeding track 62 to the slot of the conveying part 65. When the piston rod of the cylinder 641 is gradually extended, the sliding key 642 moves from the material receiving end point to the material feeding end point in the first slide groove 611. At the same time, the conveying part 65 also moves toward the direction of the conveying channel. During this movement, the slot of the conveying part 65 gradually no longer overlaps with the second slide groove 612. The screw located at the slot of the conveying part 65 can be detached from the slot and fall into the conveying channel, and then transmitted to the bird's mouth 41 through the nozzle 63 to complete the feeding.

[0074] Example 2

[0075] like Figures 4 to 5 The first embodiment of a screw locking method of the present invention is shown. The screw locking method comprises the following steps:

[0076] S1. Obtain multiple sampling images of the area to be locked by sampling, and mark all the hole positions in all the sampling images to obtain a marked data set, and then use the Yolo v8 algorithm to train the marked data set to obtain a training model;

[0077] S2. The depth camera is calibrated outside the hand through the corresponding locking device 4;

[0078] S3. Place the mobile frame 1 in the area to be locked, and set a fixed point at any position in the area to be locked, then control the multi-axis robot 3 to drive the locking device 4 to reach the position of the fixed point multiple times in different positions, and calculate the position change from the arm end 31 to the bird beak 41;

[0079] S4. Move the mobile rack 1 to the initial designated position and plan the moving path of the mobile rack 1; then execute step S5;

[0080] S5. The depth camera obtains the image to be locked at the current position of the area to be locked, and then uses the training model of step S1 to predict the hole position of the image to be locked, and performs mask processing to obtain a mask image with a specified hole position, and then obtains the circular hole information matrix of the specified hole position in the mask image; then, combined with the calibration of step S2, the circular hole information matrix is ​​converted to obtain the position movement information of the multi-axis robot 3; then step S6 is executed;

[0081] S6. According to the position movement information of the multi-axis robot 3, the multi-axis robot 3 is controlled to move, so that the locking device 4 is moved to the top of the specified hole, the feeding device 6 transmits the screw to the locking device 4, and the locking device 4 performs the locking operation at the current position;

[0082] S7. If the current position of the mobile rack 1 is the end designated position in the planned path, all locking operations are completed; otherwise, the mobile rack 1 is controlled to move to the next position according to the planned path, and then returns to step S5.

[0083] It should be noted that there is no order of precedence between steps S1 to S3.

[0084] The depth camera can be used to capture the image of the area to be locked, and the control processing device can extract all the hole position information of the image to be locked, and can identify the designated hole position to be locked from all the hole position information, and then convert the information of the designated hole position into the position movement information of the multi-axis robot 3, so that the locking device 4 is located above the corresponding hole position, and then the screw is fed to the bird mouth 41 of the locking device 4 through the feeding device 6, and the locking mechanism 42 performs screw locking; by identifying the setting of the designated hole position to be locked, it can be avoided that the locking device 4 performs screw locking on the hole position other than the designated hole position, which causes the locking device 4 or the area to be locked to be damaged; the setting of the mobile frame 1 can be used for the screw locking system to perform screw locking operations on a larger range of the area to be locked, thereby improving the efficiency of the locking operation, wherein, by planning the moving path of the mobile frame 1, the utilization rate of the multi-axis robot 3 can be improved, the locking operation efficiency can be improved, and the safety of the locking operation can be improved. The area to be locked can be the area of ​​the container floor.

[0085] Specifically, in step S1, when the area to be locked is a container floor, the sampled image is also an image of the container floor, and the sampled image contains holes of all feature categories; in this embodiment, the sampled image is a sampled two-dimensional picture. In addition, the annotated data set includes the feature category of each hole position, and also includes the position information of each hole position; wherein the feature categories include normal holes, special-shaped holes, and locked holes; the position information includes the coordinates of the vertices of the rectangular box corresponding to each hole position in its corresponding image coordinate system, and the image coordinate system takes the upper left corner of the sampled image as the origin. It should be noted that the designated hole position is a normal hole, which is used for the hole position to be locked.

[0086] Specifically, in step S2, the homogeneous transformation matrix from the base 2 to the depth camera position is obtained by calibration of the eye outside the hand: Cam T Base , and get the internal parameter matrix Intr Cam ;in,

[0087]

[0088] In the formula, f x 、f y Respectively represent the focal length of the depth camera in the x direction and the y direction, u x 、u y Represents the pixel density in the x direction and y direction respectively.

[0089] Specifically, in step S3, the position transformation formula from the arm end 31 to the bird beak 41 is:

[0090] Tip T Tcp Tcp T End End T Base = Tip T Base ;

[0091] In the formula, Tip T Tcp Represents the homogeneous transformation matrix from the beak 41 to the fixed point position, Tcp T End The homogeneous transformation matrix representing the position of the arm end 31 to the beak 41, End T Base Represents the homogeneous transformation matrix from the base 2 to the arm end 31 position, Tip T Base Represents the homogeneous transformation matrix from base 2 to the fixed cusp position; where:

[0092]

[0093] Wherein, T represents the position information of the homogeneous transformation matrix, [x, y, z] represents the position of the origin of the coordinate system of the bird's beak 41 in the coordinate system of the arm end 31;

[0094] Therefore, we can get:

[0095] Tcp T End End T Base = Tip T Base ;

[0096] By moving the multi-axis robot arm 3 multiple times, at least three different positions and postures are obtained and a matrix equation group can be obtained. The position transformation from the arm end 31 to the bird beak 41 can be obtained through the matrix equation group; wherein the matrix equation group is expressed as:

[0097]

[0098] In the formula, End T Base_i The ith position of the origin of the coordinate system of the arm tip 31 in the coordinate system of the base 2 is represented.

[0099] Specifically, step S5 includes the following steps:

[0100] S51. The depth camera obtains the image to be locked in the area to be locked, and then uses the training model of step S1 to predict the hole position of the image to be locked, and obtains the corresponding rectangular frames of the feature categories of irregular holes and normal holes in the image to be locked;

[0101] The image to be locked includes a two-dimensional image to be locked and a depth image to be locked aligned with the two-dimensional image to be locked. Alignment means that the two-dimensional image to be locked has the same pixel quantity, length and width as the depth image to be locked, and the depth image to be locked also includes depth information; and when the area to be locked is a container floor, the image to be locked is the container floor image to be locked;

[0102] S52. Output the center points of the corresponding rectangular frames of all the irregular holes to obtain the circular hole information matrix of the irregular holes; wherein the circular hole information matrix of the irregular holes is a matrix of six columns and n rows, n is the number of irregular holes identified, and each row is represented by [x, y, z, r, p, y]; it should be noted that the obtained circular hole information matrix of the irregular holes can facilitate the operator to quickly obtain the position information of the irregular holes and perform subsequent processing on the irregular holes;

[0103] S53. Masking the locked image so that the contents of the corresponding rectangular frame of the normal hole are retained and the rest are replaced with a solid color background to obtain a masked image; by adjusting various parameters in the circle center detection algorithm, the normal holes in the masked image can be accurately detected, and the circular hole information matrix of all normal holes in the masked image is obtained;

[0104] Among them, the circular hole information matrix of the normal hole is a matrix with three columns and N rows, and the form of each row is:

[0105] [x px y px 1];

[0106] In the formula, x px is the horizontal coordinate of the normal hole in the image coordinate system to be locked, y px is the ordinate of the normal hole in the coordinate system of the image to be locked; the minimum depth value depth inside the rectangular frame of each normal hole is taken as the distance between the center of the normal hole and the optical center of the depth camera, which is 1; since the two-dimensional image to be locked is aligned with the depth image to be locked, the corresponding depth information can be obtained through the pixel position information in the two-dimensional image to be locked;

[0107] S54. Convert the circular hole information matrix of the normal hole to obtain the position movement information of the multi-axis robot 3; wherein each row P in the circular hole information matrix of the normal hole is obtained Hole And the minimum depth value depth inside the rectangular frame of each normal hole, calculate the normal hole coordinates of each normal hole in the depth camera coordinate system:

[0108]

[0109]

[0110] P Hole =[x px y px 1] T ;

[0112] The homogeneous transformation matrix from the depth camera coordinate system to the normal hole position is:

[0113]

[0114] The homogeneous transformation matrix from the coordinate system of base 2 to the normal hole position is:

[0115] Hole T Base = Hole T Cam Cam T Base ;

[0116] Then, the homogeneous transformation matrix from the coordinate system of the base 2 to the normal hole position is converted into [x base ,y base ,z base ,r base ,p base ,y base ] format of the position vector plus the rotation vector in the coordinate system of the base 2, which is the position movement information of the multi-axis robot arm 3.

[0117] Example 3

[0118] This embodiment is similar to Embodiment 2, except that Figures 5 and 6 As shown, in this embodiment, step S4 includes the following steps:

[0119] S41. Setting the number of mobile racks 1: Select n mobile racks 1 for operation, and then divide the area to be locked into S working areas, where: The working area of ​​the first mobile rack 1 is the n working areas closest to the starting position, the working area of ​​the second mobile rack 1 is the n-1 working areas closest to the starting position in the remaining working areas, and so on; Figure 5 The figure shows the distribution of the waiting lock area when there are three mobile racks 1, where A 1 , A 2 , A 3 is the working area of ​​the first mobile rack 1, B 1 , B 2 A is the working area of ​​the second mobile rack 1 , and C is the working area of ​​the third mobile rack 1 .

[0120] S42. Setting the number and position distribution of the multi-axis robotic arms 3: Setting the number of the multi-axis robotic arms 3 to k, setting the working space diameter of the multi-axis robotic arms 3 to d, and setting the width of the area to be locked to D, the three satisfying the following relationship:

[0121]

[0122] The multi-axis robotic arms 3 are symmetrically distributed on the mobile frame 1, and the multi-axis robotic arms 3 adjacent to each other on the same side are kept The distance between the multi-axis robot arm 3 and the mobile frame 1 can maintain its working space. The width is outside the mobile frame 1; the working space range of the multi-axis robot arm 3 located at the adjacent position on the same side overlaps, specifically, maintaining The width overlap can complete the locking operation more efficiently, such as Figure 6 As shown; it should be noted that Figure 6In the circular area of ​​the multi-axis robot 3 with a diameter of d, the center of the circle is the location of the base 2, and the circular area is the locking operation range of the bird's beak 41.

[0123] In step S42 , the movement setting of the multi-axis robot arm 3 is also included: the multi-axis robot arm 3 locks the designated hole positions in the working area one by one with a serpentine motion trajectory, which can avoid collisions between the multi-axis robot arms 3 .

[0124] S43. Mobile setting of mobile rack 1: When the mobile rack 1 is responsible for at least two working areas, the mobile rack 1 first v 1 The speed moves to the working area farthest from the starting position. After completing the locking operation at this position, the mobile frame 1 moves at a speed of v 2 Move towards the starting position at a speed of distance, and then perform the locking operation at the next position, and so on; where v 1 >v 2 ;

[0125] In step S43, when the mobile rack 1 completes all the work areas it is responsible for, the mobile rack 1 is 1 Return to the starting position at a speed of

[0126] It should be noted that there is no order between steps S41 to S43. In step S4, the size setting of the mobile rack 1 is also included: the length of the mobile rack 1 can be set to be consistent with the width D of the container floor. This setting can ensure that the working space of the multi-axis manipulator 3 symmetrically arranged on the mobile rack 1 covers all the holes in the area as much as possible, thereby improving the utilization rate of the multi-axis manipulator 3; the width of the mobile rack 1 is consistent with the minimum unit width of the working area after the container floor is allocated. This setting can reduce the number of movements of the mobile rack 1.

[0127] Specifically, in this embodiment, the control processing device includes a host and a switch connected to the host for communication. The multi-axis robot arm 3 and the depth camera are both connected to the switch for communication. The host can simultaneously obtain data information of the depth camera and the multi-axis robot arm 3 through the switch, and perform data information processing and analysis. In step S6, according to the position movement information of the multi-axis robot arm 3 obtained in step S5, a signal is transmitted to the switch through the host, and then a signal is transmitted to the multi-axis robot arm 3 through the switch to control the multi-axis robot arm 3 to move, so that the bird's beak 41 of the locking device 4 located at the end 31 of the arm moves to the specified hole position, that is, just above the normal hole, and the feeding device 6 transmits the screw to the bird's beak 41 and performs the locking operation.

[0128] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the embodiments here. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the claims of the present invention.

Claims

1. A screw locking system, characterized in that: The invention comprises a mobile frame (1), wherein a plurality of multi-axis mechanical arms (3) are connected to the mobile frame (1) via a base (2), wherein the arm end (31) of each multi-axis mechanical arm (3) is connected to a locking device (4), and the mobile frame (1) is also connected to a shooting device (5) and a feeding device (6) which are arranged in a one-to-one correspondence with the locking device (4); and further comprises a control processing device, wherein the mobile frame (1), the multi-axis mechanical arms (3), the locking device (4), the shooting device (5), and the feeding device (6) are all connected to the control processing device in a communication manner; wherein the locking device (4) comprises a locking mechanism (42) and a bird's beak (41) connected to the locking mechanism (42), wherein the locking end of the locking mechanism (42) extends into the bird's beak (41); and the discharging end of the feeding device (6) is connected to the bird's beak (41).

2. The screw locking system according to claim 1, characterized in that: The feeding device (6) comprises a mounting seat (61) mounted on the mobile frame (1), the mounting seat (61) being connected to a feeding track (62), a conveying channel being arranged in the mounting seat (61), and a nozzle (63) being connected to the discharge end of the conveying channel; and further comprises a sliding mechanism (64), the sliding end of the sliding mechanism (64) being connected to a conveying part (65), and the sliding mechanism (64) being capable of driving the conveying part (65) to communicate with the feeding track (62) and the conveying channel; and the nozzle (63) is connected to the bird's beak (41) through a pipeline.

3. A locking method applied to the screw locking system according to claim 1 or 2, characterized in that: The steps include: S1. Obtain multiple sampling images of the area to be locked by sampling, and mark all the hole positions in all the sampling images to obtain a labeled data set and train it to obtain a training model; S2. The photographing device (5) performs calibration of the eye outside the hand through the corresponding locking device (4); S3. placing the mobile frame (1) in the area to be locked, and setting a fixed sharp point at any position in the area to be locked, then controlling the multi-axis mechanical arm (3) to drive the locking device (4) to reach the position of the fixed sharp point multiple times in different postures, and calculating the position transformation from the end of the arm (31) to the beak (41); S4. Move the mobile rack (1) to an initial designated position and plan a moving path of the mobile rack (1); S5. The photographing device (5) obtains the image to be locked at the current position of the area to be locked, and then uses the training model of step S1 to predict the hole position of the image to be locked, and performs mask processing to obtain a mask image with a specified hole position, and then obtains a circular hole information matrix of the specified hole position in the mask image; then, combined with the calibration of step S2, the circular hole information matrix is ​​converted to obtain the position movement information of the multi-axis robot (3); S6. Control the multi-axis robot (3) to move according to the position movement information of the multi-axis robot (3), so that the locking device (4) moves to the top of the designated hole position, the feeding device (6) transmits the screw to the locking device (4), and the locking device (4) performs a locking operation at the current position; S7. If the current position of the mobile rack (1) is the end designated position in the planned path, all locking operations are completed; otherwise, the mobile rack (1) is controlled to move to the next position according to the planned path, and then returns to step S5.

4. The method for locking a screw locking system according to claim 3, characterized in that: In step S1, the Yolo algorithm is used to train the annotated data set; the annotated data set includes a feature category for each hole position and position information of each hole position; wherein the feature category includes normal holes, irregular holes, and locked holes; the position information includes the coordinates of the vertices of the rectangular box corresponding to each hole position in its corresponding image coordinate system.

5. The screw locking method of claim 3, wherein: In step S3, the position transformation formula from the arm end (31) to the bird beak (41) is: Tip T Tcp Tcp T End End T Base = Tip T Base ; In the formula, Tip T Tcp represents the homogeneous transformation matrix from the beak (41) to the fixed cusp position, Tcp T End represents the homogeneous transformation matrix from the arm end (31) to the beak (41), End T Base A homogeneous transformation matrix representing the position of the base (2) to the arm end (31), Tip T Base A homogeneous transformation matrix representing the base (2) to the fixed cusp position; wherein: Wherein, T represents the position information of the homogeneous transformation matrix, [x, y, z] represents the position of the origin of the coordinate system of the bird's beak (41) in the coordinate system of the arm end (31); Therefore, we can get: Tcp T End End T Base = Tip T Base ; By moving the multi-axis robot arm (3) multiple times, at least three groups of different postures are obtained and a matrix equation group can be obtained, and the position transformation from the arm end (31) to the bird's beak (41) can be obtained through the matrix equation group; wherein the matrix equation group is expressed as: In the formula, End T Base_i Indicates the i-th position of the origin of the coordinate system of the arm end (31) in the coordinate system of the base (2).

6. The method for locking a screw locking system according to claim 3, characterized in that: In step S2, the homogeneous transformation matrix of the position of the base (2) to the camera (5) is obtained by calibration of the eye outside the hand. Cam T Base , and get the internal parameter matrix Intr Cam ; in: In the formula, f x 、f y represent the focal lengths of the camera (5) in the x direction and the y direction, respectively, x 、u y Represents the pixel density in the x direction and y direction respectively; Step S5 includes the following steps: S51. The shooting device (5) obtains the image to be locked in the area to be locked, and then uses the training model to predict the hole position of the image to be locked, and obtains the corresponding rectangular frame of the feature category of the irregular hole and the normal hole in the image to be locked; S52. Output the center points of the corresponding rectangular boxes of all the irregular holes to obtain the circular hole information matrix of the irregular holes; S53. Mask the locked image so that the contents of the corresponding rectangular frame of the normal hole are retained and the rest are replaced with a solid color background to obtain the mask image and obtain the circular hole information matrix of all normal holes in the mask image; S54. Convert the circular hole information matrix of the normal hole to obtain the position movement information of the multi-axis robot arm (3).

7. The screw locking method of claim 6, wherein: In step S52, the circular hole information matrix of the irregular-shaped holes is a matrix of six columns and n rows, where n is the number of irregular-shaped holes identified, and each row is represented as [x, y, z, r, p, y]; In step S53, the circular hole information matrix of the normal hole is a matrix of three columns and N rows, and each row is in the form of: [x px and px 1]; In the formula, x px is the horizontal coordinate of the normal hole in the image coordinate system to be locked, y px is the vertical coordinate of the normal hole in the coordinate system of the image to be locked; the minimum depth value depth inside the rectangular frame of each normal hole is taken as the distance between the center of the normal hole and the optical center of the shooting device (5), which is 1; In step S54, each row P in the circular hole information matrix of the normal hole is obtained. Hole And the minimum depth value depth inside the rectangular frame of each normal hole, calculate the normal hole coordinates of each normal hole in the coordinate system of the shooting device (5): P Hole =[x px y px 1] T ; The homogeneous transformation matrix from the coordinate system of the photographing device (5) to the normal hole position is: The homogeneous transformation matrix from the coordinate system of the base (2) to the normal hole position is: Hole T Base = Hole T Cam Cam T Base ; Then, the homogeneous transformation matrix from the coordinate system of the base (2) to the normal hole position is converted into [x base ,y base ,z base ,r base ,p base ,y base ] format of the position vector plus the rotation vector in the coordinate system of the base (2), which is the movement information of the multi-axis robotic arm (3).

8. The method for locking a screw locking system according to claim 3, characterized in that: Step S4 includes the following steps: S41. Setting the number of the mobile racks (1): Select n mobile racks (1) for operation, and then divide the area to be locked into S working areas, where: The working area of ​​the first mobile rack (1) is the n working areas closest to the starting position, the working area of ​​the second mobile rack (1) is the n-1 working areas closest to the starting position in the remaining working areas, and so on; S42. The number and position distribution of the multi-axis robotic arms (3) are set: the number of the multi-axis robotic arms (3) is set to k, the working space diameter of the multi-axis robotic arms (3) is set to d, and the width of the area to be locked is set to D, and the three satisfy the following relationship: S43. Movement setting of the mobile rack (1): When the mobile rack (1) is responsible for at least two working areas, the mobile rack (1) first moves to the working area farthest from the starting position at a speed of v1. After completing the locking operation at this position, the mobile rack (1) moves toward the starting position at a speed of v2. distance, and perform locking operation at the next position, and so on; wherein, v1>v2.

9. The screw locking method of claim 8, wherein: In step S42, the movement setting of the multi-axis robot arm (3) is also included: the multi-axis robot arm (3) locks the designated hole positions in the working area one by one with a serpentine motion trajectory.

10. The screw locking method of the screw locking system according to claim 8, characterized in that: In step S43, when the mobile frame (1) completes all the working areas for which it is responsible, the mobile frame (1) returns to the starting position at a speed of v1.

Images