A threaded hole recognition tracking positioning method, system, device and storage medium

Abstract

The present application relates to the technical field of threaded hole positioning, and particularly relates to a threaded hole identification, tracking and positioning method, system, device and storage medium, which comprises the following steps: taking a picture containing a workpiece and a tooling, visually identifying a positioning hole of the tooling clamping the workpiece, determining coarse positioning of the area where the tooling is located; determining the initial position of the tooling based on the coarse positioning of the area where the tooling is located, starting visual tracking, and realizing dynamic locking of the target tooling; based on visual tracking of the tooling positioning hole, the area where the tooling is located is cropped to obtain the main distribution area of the threaded hole; according to the main distribution area of the threaded hole, the threaded hole is identified and extracted to obtain mask information of the threaded hole and edge feature points of the threaded hole; and the 6D pose of the workpiece is determined based on the edge feature points of the threaded hole. The present method has simple requirements for the operation process of the equipment, is suitable for rapid large-scale replication of different screw locking production lines, and can lock and attach screws in a dynamic scene.

Description

Technical Field

[0001] This invention relates to the field of threaded hole positioning technology, specifically to a threaded hole identification, tracking, and positioning method, system, device, and storage medium. Background Technology

[0002] The production of electromechanical equipment is inseparable from assembly, and the most important method of assembly is screw fastening. These electromechanical devices range from small everyday items like mobile phones, computers, microwave ovens, and refrigerators to large engineering vehicles, high-speed trains, and airplanes, all requiring a large number of screws of different types to assemble parts and form the final product.

[0003] Each common device, such as a mobile phone or computer, requires dozens of screws; household appliances like washing machines and refrigerators require hundreds; and the assembly of large equipment or precision instruments requires thousands or even tens of thousands of screws. Traditional screw fastening relies primarily on manual installation, but this method is inefficient and heavily dependent on the operator's experience. Skilled workers can fasten over ten thousand screws per day with a high yield rate; while new employees can only achieve three to four thousand, with a very high defect rate. Furthermore, with the aging population, there will be an even greater shortage of workers capable of performing simple, repetitive tasks.

[0004] Automatic screw fastening devices have emerged to address this need. However, due to limitations in screw positioning and other technologies, existing automatic screw fastening equipment requires a static environment. This forces the production line to stop at the screw fastening station, increasing the overall production cycle time and reducing efficiency. Furthermore, existing automatic screw fastening mechanisms generally require pre-programming or teaching the position of each threaded hole, fixing the hole position beforehand. This makes the process cumbersome, and changes in component type necessitate reconfiguration of the equipment and program. Consequently, this type of automated equipment lacks versatility, existing only as a dedicated machine for a specific station and unable to be replicated on a large scale for production lines handling different types of components. Summary of the Invention

[0005] The first aspect of this invention proposes a threaded hole identification, tracking, and positioning method with good versatility and capable of positioning on dynamic production lines. The steps of the method are as follows:

[0006] S1: Take pictures containing the workpiece and the tooling, visually identify the positioning holes of the tooling that holds the workpiece, and determine the coarse positioning of the area where the tooling is located.

[0007] S2: Determine the initial position of the tooling based on the coarse positioning of the area where the tooling is located, enable visual tracking, and achieve dynamic locking of the target tooling;

[0008] S3: Based on visual tracking of the tooling positioning holes, the area where the tooling is located is clipped to obtain the main distribution area of ​​the threaded holes;

[0009] S4: Based on the main distribution area of ​​the threaded holes, identify and extract the threaded holes to obtain the mask information and edge feature points of the threaded holes;

[0010] S5: Determine the 6D pose of the workpiece based on the feature points at the edge of the threaded hole.

[0011] This method has simple requirements for equipment operation procedures, is suitable for rapid large-scale replication of different screw fastening production lines, and can fasten screws in dynamic scenarios, thereby greatly improving production line efficiency.

[0012] In some implementations of the first aspect, the dynamic locking includes:

[0013] Taking the initial position of the tooling as the target, the target's movement speed is calculated to obtain the target's movement speed coordinates. The specific calculation formula is as follows:

[0014]

[0015] in, These are the accelerations read in the x and y directions, respectively. These are the x and y coordinate values ​​read, respectively; ξ x ξ y , These are the counterweight parameters in the x and y directions, respectively; Q n The noise matrix is ​​Gaussian. The x-coordinate of the top-left corner is obtained by visual tracking at time n. The y-coordinate of the top-left corner is obtained by visual tracking at time n. The x-coordinate of the lower right corner is obtained by visual tracking at time n. The y-coordinate of the lower right corner is obtained by visual tracking at time n. Let be the velocity in the x direction at time n. Let y be the velocity at time n, and Δt be the time interval between each sampling update.

[0016] In some implementations of the first aspect, step S3 involves cropping the target region to obtain the cropping coordinates:

[0017]

[0018] in, To crop the image, we need the x and y coordinates of the top-left corner of the image at time n+1. The x and y coordinates of the bottom right corner of the cropped image at time n+1;

[0019] These are the coordinates of the top-left corner x at time n, obtained from visual tracking and visual recognition of the positioning hole, respectively. These are the coordinates of the top-left corner y at time n, obtained from visual tracking and visual recognition of the positioning hole, respectively. The coordinates of the lower right corner x at time n are obtained by visual tracking and visual recognition of the positioning hole, respectively. The coordinates of the lower right corner y at time n are obtained by visual tracking and visual recognition of the positioning hole, respectively. Let be the velocity in the x direction at time n. Let y be the velocity at time n, and Δt be the time interval between each sampling update.

[0020] In some implementations of the first aspect, the identification and extraction of threaded holes are performed in step S4, specifically as follows:

[0021] The cropped image is binarized and subjected to opening and closing operations to obtain the independent regions of each threaded hole on the workpiece, and the mask information of each threaded hole is obtained. After gradient processing of the area covered by the mask of each threaded hole, the feature points of the edge are obtained by the corner point extraction method. The feature points of the edge are clustered to remove noise, and the edge feature points of the threaded hole are obtained.

[0022] In some implementations of the first aspect, step S4 further includes repositioning of the threaded hole boundary. If the obtained threaded hole edge feature point is greater than or equal to the actual position offset error threshold, then step S3 is repeated to re-cut the area where the tooling is located.

[0023] In some implementations of the first aspect, determining the 6D pose of the workpiece based on the feature points of the threaded hole edge includes:

[0024] Define the effective feature points detected at the edge of the threaded hole as follows: The subscript i represents the i-th point among the edge points of the screw hole, and the superscript k represents the k-th screw hole;

[0025] Based on the model space projection of the pinhole camera, the homogeneous pixel coordinates of the edge points detected in the image are represented as follows:

[0026]

[0027] in, Let S be the homogeneous coordinates of the spatial edge point in the world coordinate system. cam Let K be the scale parameter matrix. cam R cam T cam These are the camera's intrinsic parameter matrix, rotation transformation matrix, and translation matrix;

[0028] The preset truth value of the feature point is The detection error of all points at the edge of a series of threaded holes can be written as:

[0029]

[0030] Define the 6D pose of the workpiece as X = [xyzabc] T , where x, y, z are the Cartesian coordinates of the workpiece in the robot coordinate system, and a, b, c are the rotation angles along the three axes x, y, and z;

[0031] The pose of the target workpiece can be solved as follows:

[0032] X = -(J T J+λI) -1 J T Err

[0033] Where X is the target 6D pose, λ is the introduced coefficient ranging from 0 to 1, I is the identity matrix, and J is the Jacobian matrix.

[0034]

[0035] Furthermore, the method also includes S6, which guides the robot to coordinates based on the 6D pose of the workpiece and controls the robot to perform the screw fastening operation.

[0036] The second aspect provides a threaded hole identification, tracking, and positioning system, including:

[0037] The coarse positioning module is used to capture images containing the workpiece and the tooling. By visually recognizing the positioning holes of the tooling that holds the workpiece, the area where the tooling is located is coarsely positioned.

[0038] The visual tracking module is used to determine the initial position of the tooling based on the coarse positioning of the area where the tooling is located, and to enable visual tracking to dynamically lock the target tooling.

[0039] The trimming module is used to trim the area where the tooling is located based on visual tracking of the tooling positioning holes, and to obtain the main distribution area of ​​the threaded holes;

[0040] The identification and extraction module is used to identify and extract threaded holes based on their main distribution areas, and obtain the mask information and edge feature points of the threaded holes. If the obtained edge feature points of the threaded holes are greater than or equal to the offset error threshold, the threaded hole boundary is repositioned and the area where the tooling is located is re-trimmed.

[0041] The 6D pose acquisition module is used to determine the 6D pose of the workpiece based on the feature points on the edge of the threaded hole.

[0042] The third aspect provides a threaded hole identification, tracking, and positioning device, including a processor and a memory, wherein the processor executes program data stored in the memory to implement a threaded hole identification, tracking, and positioning method as described above.

[0043] A fourth aspect provides a computer-readable storage medium, characterized in that it is used to store control program data, wherein the control program data, when executed by a processor, implements a threaded hole identification, tracking, and positioning method as described above.

[0044] The beneficial effects are:

[0045] (1) The threaded hole identification, tracking, and positioning method proposed in this invention enables online dynamic screw driving without interrupting the production line. Therefore, the production line does not need to be paused at the screw driving station, which greatly improves the overall production efficiency. The rapid identification and positioning of threaded holes and tooling positioning holes can achieve real-time positioning of the area to be screwed in a very short time. Furthermore, with the support of a visual tracking algorithm, it can achieve dynamic online tracking of the target area and control the tracking error within a very small range (usually less than 1 mm and the angle is less than 1 degree).

[0046] (2) The threaded hole identification, tracking, and positioning method proposed in this invention is applicable to different screw-driving scenarios and is easier to deploy quickly and widely. Using the identification and positioning of the circumferential edge allows for the identification and positioning of any number and distribution of circles. Therefore, during deployment, only the circular holes in the actual scenario need to be made into templates. The algorithm can then match the identified circles with the circular holes in the template, and the 6D pose of the threaded hole can be solved based on the matching result. This invention proposes the identification, positioning, and template matching of circular holes, and this algorithm can also be extended to other geometric shapes, such as rectangles, line segments, triangles, and other geometric shapes.

[0047] (3) The threaded hole recognition, tracking and positioning method proposed in this invention has high accuracy and can better guide the robot to align with the threaded hole, thereby greatly increasing the success rate of screw driving. By directly recognizing the circumferential pixels when positioning the circle, and finally calculating the position and attitude through these points, instead of fitting these circumferential points into a circle, the positioning error caused by the circular offset of the circular hole in the camera's field of view due to perspective transformation is eliminated. Attached Figure Description

[0048] Figure 1 A flowchart illustrating the threaded hole identification, tracking, and positioning method;

[0049] Figure 2 A schematic diagram of a threaded hole identification, tracking, and positioning system. Detailed Implementation

[0050] Exemplary embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings.

[0051] Example

[0052] See Figure 1 This invention proposes a method for identifying, tracking, and locating threaded holes, the specific steps of which are as follows:

[0053] S1: By capturing images containing the workpiece and the tooling, visual recognition is performed on the positioning holes of the tooling that holds the workpiece, and coarse positioning is performed on the area where the tooling is located.

[0054] When the conveyor belt transports the workpiece carrying the screws to be screwed into the camera's field of view, it triggers the photoelectric switch. The photoelectric switch sends a start signal to activate the camera to capture images of the workpiece and the fixture. By processing the captured images, the positioning holes of the fixture holding the workpiece are visually identified, achieving coarse positioning of the fixture area.

[0055] S2: Determine the initial position of the tooling based on the coarse positioning of the area where the tooling is located, enable visual tracking, and achieve dynamic locking of the target tooling;

[0056] Once the initial position of the tooling is obtained, the system activates the visual tracking algorithm to achieve visual tracking and dynamic locking of the target tooling.

[0057] Furthermore, in order to achieve visual tracking of the target tooling, it is necessary to determine the target's moving speed and coordinates. Taking the initial position of the tooling as the target, the target's moving speed is calculated to obtain the target's moving speed coordinates. The specific calculation formula is as follows:

[0058]

[0059] in, These are the accelerations read in the x and y directions, respectively. These are the x and y coordinate values ​​read, respectively; ξ x ξ y , These are the counterweight parameters in the x and y directions, respectively; Q n The noise matrix is ​​Gaussian. The x-coordinate of the top-left corner is obtained by visual tracking at time n. The y-coordinate of the top-left corner is obtained by visual tracking at time n. The x-coordinate of the lower right corner is obtained by visual tracking at time n. The y-coordinate of the lower right corner is obtained by visual tracking at time n. Let be the velocity in the x direction at time n. Let y be the velocity at time n, and Δt be the time interval between each sampling update.

[0060] S3: Based on visual tracking of the tooling positioning holes, the area where the tooling is located is clipped to obtain the main distribution area of ​​the threaded holes;

[0061] Meanwhile, in order to accelerate the processing speed of the algorithm, based on the visual tracking of the tooling positioning holes, the target area of ​​the image captured by the camera is cropped to obtain the main distribution area of ​​the threaded holes.

[0062] The clipping coordinates of the target area are determined by the vision fixture, the fixture positioning holes, and the target tracking speed, using the following formula:

[0063]

[0064] in, To crop the image, we need the x and y coordinates of the top-left corner of the image at time n+1. The x and y coordinates of the bottom right corner of the cropped image at time n+1;

[0065] These are the coordinates of the top-left corner x at time n, obtained from visual tracking and visual recognition of the positioning hole, respectively. These are the coordinates of the top-left corner y at time n, obtained from visual tracking and visual recognition of the positioning hole, respectively. The coordinates of the lower right corner x at time n are obtained by visual tracking and visual recognition of the positioning hole, respectively. The coordinates of the lower right corner y at time n are obtained by visual tracking and visual recognition of the positioning hole, respectively. Let be the velocity in the x direction at time n. Let y be the velocity at time n, and Δt be the time interval between each sampling update.

[0066] According to the cropping coordinate formula, the coordinate values ​​of the cropped image are... Take the maximum value of the visual tracking or positioning hole visual recognition detection value in the x-direction at time n+1, and use the target tracking speed at time n. Position compensation is performed based on the sampling update time interval Δt;

[0067] Cropping image coordinates Take the maximum value of the visual tracking or positioning hole visual recognition detection value in the y direction at time n+1, and use the target tracking speed at time n. Position compensation is performed based on the sampling update time interval Δt;

[0068] Cropping image coordinates Then, take the minimum value of the visual tracking or positioning hole visual recognition detection value in the x-direction at time n+1, and use the target tracking speed at time n. Position compensation is performed based on the sampling update time interval Δt;

[0069] Cropping image coordinates Then, take the minimum value of the visual tracking or positioning hole visual recognition detection value in the y direction at time n+1, and use the target tracking speed at time n. Position compensation is performed at the sampling update time interval Δt.

[0070] S4: Based on the main distribution area of ​​the threaded holes, identify and extract the threaded holes to obtain the mask information and edge feature points of the threaded holes;

[0071] After obtaining the trimmed threaded hole area, the threaded holes are identified and extracted.

[0072] Identification of threaded holes in workpieces: Since threaded holes appear darker than other areas under illumination, the cropped image is binarized and subjected to opening and closing operations during the identification process to obtain the independent regions of each threaded hole on the workpiece and obtain the mask information of each threaded hole.

[0073] Extraction of threaded holes in workpieces: After gradient processing of the area covered by the mask for each threaded hole, the feature points of the edge are obtained by corner point extraction method; noise is removed by cluster analysis of the feature points of the edge to obtain the feature points of the threaded hole edge.

[0074] As a specific implementation method, the SIFT corner extraction method is used; the RANSAC algorithm can be used for cluster analysis.

[0075] Furthermore, step S4 also includes repositioning the threaded hole boundary based on the positional deviation between the threaded hole edge feature points and the actual threaded hole. If the obtained threaded hole edge feature points and the actual position are greater than or equal to the offset error threshold, then step S3 is repeated to re-cut the area where the tooling is located.

[0076] S5: Determine the 6D pose of the workpiece based on the feature points at the edge of the threaded hole.

[0077] Considering the perspective transformation of the screw hole within the camera's field of view, the circle will become an ellipse, and the actual position of the center will also shift. Directly fitting the center of the threaded hole for positioning results in significant errors. Therefore, this invention solves the pose directly through the edge of the threaded hole, and then uses the reprojection error of the edge to iteratively solve the 6D pose of the workpiece, as detailed below:

[0078] Define the effective feature points detected at the edge of the threaded hole as follows: The subscript i represents the i-th point among the edge points of the screw hole, and the superscript k represents the k-th screw hole;

[0079] Based on the model space projection of the pinhole camera, the homogeneous pixel coordinates of the edge points detected in the image are represented as follows:

[0080]

[0081] in, Let S be the homogeneous coordinates of the spatial edge point in the world coordinate system. cam Let K be the scale parameter matrix. cam R cam T cam These are the camera's intrinsic parameter matrix, rotation transformation matrix, and translation matrix;

[0082] And preset the true value of the feature point The detection error of all points at the edge of a series of threaded holes can be written as:

[0083]

[0084] Define the 6D pose of the workpiece as X = [xyzabc] T , where x, y, z are the Cartesian coordinates of the workpiece in the robot coordinate system, and a, b, c are the rotation angles along the three axes x, y, and z;

[0085] According to the Levenberg-Marquarelt optimization algorithm, the pose of the target workpiece can be solved as follows:

[0086] X = -(J T J+λI) -1 J T Err

[0087] Where X is the target 6D pose, λ is the introduced coefficient ranging from 0 to 1, I is the identity matrix, J is the Jacobian matrix, and ● T To represent the transpose of a vector or matrix, J can be specifically expressed as...

[0088]

[0089] Accurate solution of the workpiece's 6D pose information is crucial for robot screw fastening.

[0090] In a specific implementation, the 6D pose of the workpiece is directly used as the robot's coordinate guidance information to control the robot to perform the screw fastening operation. By directly identifying the circumferential pixels when locating the circle, and ultimately calculating the position and pose using these points, instead of fitting these circumferential points into a circle, the positioning error introduced by the circular offset caused by perspective transformation within the camera's field of view is eliminated. This allows for better guidance of the robot to align with the threaded hole, thereby greatly increasing the success rate of screw fastening.

[0091] This method enables online dynamic screw driving without interrupting the production line, thus greatly improving the overall production efficiency. Rapid identification and positioning of threaded holes and tooling positioning holes allows for real-time positioning of the area to be screwed in a very short time. Furthermore, with the support of a visual tracking algorithm, dynamic online tracking of the target area can be achieved, and the tracking error can be controlled within a very small range (typically less than 1mm, angle less than 1 degree).

[0092] Furthermore, the present invention also provides a threaded hole identification, tracking, and positioning system, such as... Figure 2 As shown, the system includes:

[0093] The coarse positioning module is used to capture images of the workpiece and tooling, visually identify the positioning holes of the tooling holding the workpiece, and coarsely position the area where the tooling is located. When the conveyor belt transports the workpiece carrying the screws to be screwed into the camera's field of view, it triggers a photoelectric switch. The photoelectric switch sends a start signal to turn on the camera to capture images of the workpiece and tooling. By processing the captured images, the positioning holes of the tooling holding the workpiece are visually identified to achieve coarse positioning of the tooling area.

[0094] The visual tracking module is used to determine the initial position of the tooling based on the coarse positioning of the area where the tooling is located, and to enable visual tracking to dynamically lock the target tooling.

[0095] The trimming module is used to trim the area where the tooling is located based on visual tracking of the tooling positioning holes, and to obtain the main distribution area of ​​the threaded holes;

[0096] The identification and extraction module is used to identify and extract threaded holes based on their main distribution areas, obtaining the mask information and edge feature points of the threaded holes. By binarizing and performing opening / closing operations on the cropped image, the independent regions of each threaded hole on the workpiece are obtained, acquiring the mask information for each threaded hole. After gradient processing of the masked areas of each threaded hole, the edge feature points are obtained using a corner point extraction method. Finally, noise is removed by clustering analysis of the edge feature points to obtain the edge feature points of the threaded holes.

[0097] The identification and extraction module is also used for repositioning the threaded hole boundary. If the obtained threaded hole edge feature points are greater than or equal to the offset error threshold, the area where the tooling is located is re-trimmed.

[0098] The 6D pose acquisition module is used to determine the 6D pose of the workpiece based on the feature points on the edge of the threaded hole.

[0099] Furthermore, the present invention also provides a threaded hole identification, tracking and positioning device, including a processor and a memory, wherein the processor executes program data stored in the memory to implement a threaded hole identification, tracking and positioning method as described above.

[0100] Finally, the present invention provides a computer-readable storage medium, characterized in that it is used to store control program data, wherein the control program data, when executed by a processor, implements a threaded hole identification, tracking and positioning method as described above.

[0101] It should be noted that the exemplary embodiments mentioned in this invention describe methods or systems based on a series of steps or apparatus. However, this invention is not limited to the order of the steps described above; that is, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

Claims

1. A method for identifying, tracking, and locating threaded holes, characterized in that, Includes the following steps: S1: Take pictures containing the workpiece and the tooling, visually identify the positioning holes of the tooling that holds the workpiece, and perform coarse positioning of the area where the tooling is located. S2: Determine the initial position of the tooling based on the coarse positioning of the area where the tooling is located, enable visual tracking, and achieve dynamic locking of the target tooling; S3: Based on visual tracking of the target tooling, the area where the tooling is located is clipped to obtain the main distribution area of ​​the threaded holes; The target area is cropped, and the cropping coordinates are: in, To crop the image, we need the x and y coordinates of the top-left corner of the image at time n+1. The x and y coordinates of the bottom right corner of the cropped image at time n+1; These are the coordinates of the top-left corner x at time n, obtained from visual tracking and visual recognition of the positioning hole, respectively. These are the coordinates of the top-left corner y at time n, obtained from visual tracking and visual recognition of the positioning hole, respectively. These are the coordinates of the lower right corner x at time n for visual tracking and visual recognition of the positioning hole, respectively. The coordinates of the lower right corner y at time n are obtained by visual tracking and visual recognition of the positioning hole, respectively. Let be the velocity in the x direction at time n. Let be the velocity in the y-direction at time n, and Δt be the time interval between each sampling update; S4: Based on the main distribution area of ​​the threaded holes, identify and extract the threaded holes to obtain the mask information and edge feature points of the threaded holes; S5: Determine the 6D pose of the workpiece based on the feature points at the edge of the threaded hole.

2. The threaded hole identification, tracking, and positioning method according to claim 1, characterized in that, The dynamic locking in S2 includes: Taking the initial position of the tooling as the target, the target's movement speed is calculated to obtain the target's movement speed coordinates. The specific calculation formula is as follows: in, These are the accelerations read in the x and y directions, respectively. These are the x and y coordinate values ​​read, respectively; ξ x ξ y , These are the counterweight parameters in the x and y directions, respectively; Q n The noise matrix is ​​Gaussian. The x-coordinate of the top-left corner is obtained by visual tracking at time n. The y-coordinate of the top-left corner is obtained by visual tracking at time n. The x-coordinate of the lower right corner is obtained by visual tracking at time n. The y-coordinate of the lower right corner is obtained by visual tracking at time n. Let be the velocity in the x direction at time n. Let be the velocity in the y-direction at time n, and Δt be the time interval between each sampling update.

3. The threaded hole identification, tracking, and positioning method according to claim 1, characterized in that, In step S4, the threaded hole is identified and extracted. The specific method is as follows: The cropped image is binarized and subjected to opening and closing operations to obtain the independent regions of each threaded hole on the workpiece, and the mask information of each threaded hole is obtained. After gradient processing of the area covered by the mask of each threaded hole, the feature points of the edge are obtained by the corner point extraction method. The feature points of the edge are clustered to remove noise, and the edge feature points of the threaded hole are obtained.

4. The threaded hole identification, tracking, and positioning method according to claim 3, characterized in that, S4 also includes repositioning of the threaded hole boundary. If the obtained threaded hole edge feature point is greater than or equal to the offset error threshold, then step S3 is repeated to re-cut the area where the tooling is located.

5. The threaded hole identification, tracking, and positioning method according to claim 1, characterized in that, The method of determining the 6D pose of the workpiece based on the feature points of the threaded hole edge includes: Define the effective feature points detected at the edge of the threaded hole as follows: The subscript i represents the i-th point among the edge points of the threaded hole, and the superscript k represents the k-th threaded hole; Based on the model space projection of the pinhole camera, the homogeneous pixel coordinates of the edge points detected in the image are represented as follows: in, Let S be the homogeneous coordinates of the spatial edge point in the world coordinate system. cam Let K be the scale parameter matrix. cam R cam T cam These are the camera's intrinsic parameter matrix, rotation transformation matrix, and translation matrix; The preset truth value of the feature point is The detection error of all points at the edge of a series of threaded holes can be written as: Define the 6D pose of the workpiece as X = [xyzabc] T , where x, y, z are the Cartesian coordinates of the workpiece in the robot coordinate system, and a, b, c are the rotation angles along the three axes x, y, and z; The pose of the target workpiece can be solved as follows: X=-(J T J+λI) -1 J T Err Where X is the target 6D pose, λ is the introduced coefficient ranging from 0 to 1, I is the identity matrix, and J is the Jacobian matrix.

6. The threaded hole identification, tracking, and positioning method according to claim 1, characterized in that, It also includes S6, which guides the robot's coordinates based on the workpiece's 6D pose and controls the robot to perform screw fastening operations.

7. A threaded hole identification, tracking, and positioning system for implementing the threaded hole identification, tracking, and positioning method according to claim 1, characterized in that, include: The coarse positioning module is used to capture images containing the workpiece and the tooling. By visually recognizing the positioning holes of the tooling that holds the workpiece, the area where the tooling is located is coarsely positioned. The visual tracking module is used to determine the initial position of the tooling based on the coarse positioning of the area where the tooling is located, and to enable visual tracking to dynamically lock the target tooling. The trimming module is used to trim the area where the tooling is located based on visual tracking of the tooling positioning holes, and to obtain the main distribution area of ​​the threaded holes; The identification and extraction module is used to identify and extract threaded holes based on their main distribution areas, and obtain the mask information and edge feature points of the threaded holes. If the obtained edge feature points of the threaded holes are greater than or equal to the offset error threshold, the threaded hole boundary is repositioned and the area where the tooling is located is re-trimmed. The 6D pose acquisition module is used to determine the 6D pose of the workpiece based on the feature points on the edge of the threaded hole.

8. A threaded hole identification, tracking, and positioning device, characterized in that, The device includes a processor and a memory, wherein the processor executes program data stored in the memory to implement a threaded hole identification, tracking, and positioning method according to any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, Used to store control program data, wherein the control program data, when executed by a processor, implements a threaded hole identification, tracking, and positioning method according to any one of claims 1-6.

Images