Mechanical arm guiding method and system of track board fine adjustment robot

Abstract

The invention discloses a mechanical arm guiding method and system for a track board fine adjustment robot, and the method comprises the steps: controlling a mechanical arm to drive a camera to shoot a calibration board, and building a global vision pose mapping relation from a pixel coordinate system to a tightening shaft coordinate system based on collected coordinate data; during actual operation, the mechanical arm is controlled to move to a preset ideal position to shoot a target image, and the center position and the rotation angle of a fine adjustment claw tightening groove are extracted through a visual recognition algorithm; based on the established visual pose mapping relation, the relative position relation of the target under the pixel coordinate system is converted into the target offset under the tightening axis coordinate system, and the target coordinate for guiding the mechanical arm to operate is synthesized by combining the ideal position coordinate; according to the target coordinate and the rotation angle, the mechanical arm and the end effector are controlled to complete butt joint operation; through the visual pose mapping module and an indirect mapping resolving strategy, the influence of installation errors and tool offset is overcome, and the operation precision, efficiency and reliability of the track slab fine adjustment robot are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to a rail plate fine adjustment robot, in particular to a mechanical arm guiding method and system of a rail plate fine adjustment robot. BACKGROUND

[0002] In rail transit construction, rail plate fine adjustment is a key process to ensure line smoothness and train operation safety, and the primary step is to accurately and quickly insert the tightening shaft of the fine adjustment robot into the tightening slot of the fine adjustment jaw. At present, the visual guiding method commonly used in the industry relies on establishing a fixed pixel-to-millimeter ratio relationship to realize coordinate conversion. This method can work in ideal conditions, but its limitations are exposed in actual complex industrial sites. On the one hand, this method strictly depends on the ideal installation posture that the camera optical axis is completely perpendicular to the workpiece plane, but actual camera installation errors and workpiece pose deviations will destroy this premise, causing significant errors due to perspective distortion. On the other hand, this method fails to consider the tool offset problem inherent in the non-coincidence of the camera observation center and the physical center of the actuator (tightening shaft), even if the pixel coordinate conversion is accurate, the actuator (tightening shaft) will fail to insert due to spatial pose deviation. These problems directly lead to the interruption of automated work processes, not only severely restricting construction efficiency, but also potentially damaging components due to repeated attempts, posing safety risks for long-term rail operation and restricting the improvement of rail plate fine adjustment automation level. SUMMARY

[0003] The purpose of the present application is to provide a rail plate fine adjustment robot mechanical arm guiding method that is fast, high-precision, and strong in anti-interference ability, and another purpose of the present application is to provide a rail plate fine adjustment robot mechanical arm guiding system.

[0004] Technical solution: The rail plate fine adjustment robot mechanical arm guiding method disclosed by the present application comprises the following contents:

[0005] The camera is controlled by the mechanical arm to take pictures of the calibration plate at different poses, and the pose coordinates in the tightening shaft coordinate system and the corresponding center coordinates of the calibration plate in the pixel coordinate system are collected, and the visual pose mapping relationship from the pixel coordinate system to the tightening shaft coordinate system is solved based on the corresponding coordinates;

[0006] The mechanical arm is controlled to move to a preset ideal working pose and take a target workpiece image, and the center point coordinates of the target workpiece in the pixel coordinate system and its rotation angle are extracted from the image through a visual recognition algorithm;

[0007] Based on the visual pose mapping relationship, the relative relationship between the center point coordinates of the target workpiece and the pixel coordinates corresponding to the ideal working pose is mapped to the target offset in the tightening axis coordinate system, and combined with the coordinates of the ideal working pose, the target coordinates guiding the operation of the robotic arm end effector are synthesized.

[0008] Based on the target coordinates and the identified rotation angle, the movement of the robotic arm and end effector is controlled to complete the operation on the target workpiece.

[0009] Preferably, the calibration plate is a U-shaped calibration plate.

[0010] Preferably, the visual pose mapping relationship is obtained by solving the planar perspective transformation matrix, the process of which includes:

[0011] The method for extracting the center coordinates of the calibration board in the pixel coordinate system includes: sequentially performing grayscale conversion, Gaussian filtering, and adaptive threshold segmentation on the acquired calibration board image to obtain a binary image; performing edge detection and contour analysis on the binary image to filter out loop-shaped contours; performing sub-pixel level edge optimization on the filtered contours; and calculating the coordinates of the center of the smallest circumcircle of its outer ring contour as the center pixel coordinates. ;

[0012] The obtained tightening axis coordinates With center pixel coordinates The corresponding data pairs constitute the sample dataset, in which the number of sets of tightening axis coordinates and pixel coordinates are four or more.

[0013] Based on the sample dataset, the planar perspective transformation matrix is ​​calculated by solving the least-squares solution of the linear equation system. .

[0014] Preferably, the step of solving the least-squares solution of the linear equation system to obtain the planar perspective transformation matrix includes:

[0015] The planar perspective transformation matrix For a 3×3 matrix, the expression is:

[0016] ,

[0017] in, to For matrix The parameters to be determined (due to the scale invariance of homogeneous coordinates, they are usually set to...) (After normalization, there are actually 8 degrees of freedom). These parameters together define the rotation, translation, and perspective transformation relationships between the pixel coordinate system and the tightening axis coordinate system. Their specific values ​​are determined by solving a system of linear equations using multiple sets of corresponding coordinate data pairs through the calibration process.

[0018] According to the homogeneous coordinate transformation relationship Expanding into a non-homogeneous proportional form:

[0019] ,

[0020] ,

[0021] After cross-multiplication to eliminate the denominator, for each pair of corresponding points... Two linear equations are obtained:

[0022] ,

[0023] Based on n sets of corresponding points, construct a system of 2n linear equations and represent it in matrix form. Where A is a 2n×9 coefficient matrix, ;

[0024] ,

[0025] ,

[0026] The system of equations is ,because Solving by the least squares method makes smallest ;

[0027] Perform singular value decomposition on coefficient matrix A , where V is a 9×9 orthogonal matrix, and the last column of matrix V is taken as the least squares solution of the system of equations;

[0028] The vector Reconstruct it into a 3×3 matrix and normalize it, let... The planar perspective transformation matrix is ​​obtained. .

[0029] Preferably, the step of extracting the center point coordinates and rotation angle of the target workpiece using a visual recognition algorithm includes:

[0030] Control the robotic arm to move to the preset ideal photo position. And acquire images;

[0031] The target detection model is used to coarsely locate the target workpiece in the image and output its preliminary bounding box, center point coordinates and rotation angle.

[0032] Using the coarsely located bounding box as the center, expand outwards by a preset number of pixels to extract an image of the ROI (region of interest) containing the complete target workpiece;

[0033] For the ROI image, perform the following subpixel-level fine localization operations in sequence:

[0034] a. Grayscale conversion;

[0035] b. Use the Otsu algorithm for threshold segmentation to separate the tightening groove from the background;

[0036] c. Perform an edge smoothing algorithm to eliminate edge burrs from adhering to the background;

[0037] d. Based on a preset area threshold, filter out noisy regions and select candidate target regions;

[0038] e. Use the ORB corner detection algorithm to extract the corner features of the candidate target region;

[0039] f. Perform line fitting on the extracted corner points to determine the edge of the target workpiece;

[0040] Based on the fitted shape, calculate its geometric center coordinates. The coordinates of the center point and the rotation angle of the target workpiece.

[0041] Preferably, the process of mapping the relative relationship to the target offset and combining it with the coordinates of the ideal working pose to synthesize the target coordinates is achieved through the following mathematical formula:

[0042] ;

[0043] in, Let be the tightening axis coordinate of the ideal working position. These are the theoretical pixel coordinates corresponding to this pose. The coordinates of the center point of the identified target workpiece. Let be the planar perspective transformation matrix. Tighten the axis coordinates of the target obtained from the solution.

[0044] Preferably, the mathematical formula Its establishment is based on the following principles:

[0045] Defined in the ideal working position Below, the actual transformation from pixel coordinates to tightening axis coordinates is: , so that:

[0046] ,

[0047] and ;

[0048] Define the planar perspective transformation matrix obtained through calibration as follows: , so that:

[0049] ,

[0050] ;

[0051] in, According to the transformation relationship with pixel coordinates The corresponding tightening axis coordinates; According to the transformation relationship with pixel coordinates The corresponding tightening axis coordinates;

[0052] Based on the invariance of coordinate offset in the tightening axis coordinate system, we have:

[0053] ;

[0054] Will and Substituting into the above equation, we get:

[0055] ,

[0056] The formula is obtained by rearranging the parts.

[0057] Preferably, after obtaining the target tightening axis coordinates Then, control the robotic arm to move to the final target position. ,in This is a fixed offset between the pre-calibrated camera field of view center and the tightening shaft center of the tightening machine.

[0058] Preferably, when guiding the end effector of the robotic arm, the programmable logic controller (PLC) generates motion control instructions based on the target coordinates, drives the joints of the robotic arm to move through multi-axis collaborative control logic, and simultaneously adjusts the spatial posture of the tightening shaft according to the identified rotation angle of the target workpiece, so that the axis of the tightening shaft is aligned with the center line of the target workpiece.

[0059] The present invention provides a guide system for a track slab fine-tuning robot arm, comprising:

[0060] The calibration module is used to control the robotic arm and camera to perform calibration movements and process calibration board images to construct a visual pose mapping relationship from pixel coordinates to tightening axis coordinates.

[0061] The visual recognition module is used to process the acquired target workpiece image when the robotic arm is in a preset ideal working posture, and output the center point coordinates and rotation angle of the target workpiece in the pixel coordinate system.

[0062] The coordinate mapping calculation module connects the calibration module and the vision recognition module. It is used to receive the visual pose mapping relationship, the center point coordinates and the ideal working pose information, and to calculate and output the target coordinates to guide the operation of the robotic arm.

[0063] The motion control module connects the coordinate mapping and calculation module and the vision recognition module, and is used to drive the robotic arm and end effector to complete the operation based on the target coordinates and rotation angle.

[0064] Preferably, the coordinate mapping solution module includes an offset calculation unit and a coordinate synthesis unit; the offset calculation unit is used to calculate the physical offset between the center point coordinates of the target workpiece and the pixel coordinates of the ideal working pose based on the visual pose mapping relationship; the coordinate synthesis unit is used to add the physical offset to the tightening axis coordinates of the ideal working pose to synthesize the target coordinates.

[0065] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages: By introducing a planar perspective transformation matrix to establish a global visual pose mapping relationship, and combining it with an indirect coordinate mapping solution strategy based on relative offset, a robust guidance scheme that does not depend on the ideal installation posture is formed, realizing the rapid and accurate docking of the tightening shaft and the tightening groove. This not only improves the work efficiency and automation level, but also solves the problems of positioning inaccuracy and insertion failure caused by installation errors and tool offsets in existing devices, significantly improving the operational reliability and engineering applicability of the track plate fine-tuning robot. Attached Figure Description

[0066] Figure 1 This is a schematic diagram showing the relationship between the camera axis and the normal to the upper surface of the tightening groove in this invention;

[0067] Figure 2 This is a schematic diagram showing the relationship between the camera axis and the tightening shaft axis of the present invention;

[0068] Figure 3 This is a schematic diagram of the camera's image capture position according to the present invention;

[0069] Figure 4 This is a schematic diagram of the spiral calibration plate of the present invention;

[0070] Figure 5 This is a diagram showing the transformation relationship between the pixel coordinate system and the tightening axis coordinate system of the present invention;

[0071] Figure 6 This is a flowchart of the transformation matrix solving process of the present invention;

[0072] Figure 7 This is a flowchart illustrating the overall workflow of the present invention. Detailed Implementation

[0073] The technical solution of the present invention will be further described below with reference to the accompanying drawings.

[0074] Example 1:

[0075] like Figures 1-7 As shown in this embodiment, a robotic arm guidance method for a track slab fine-tuning robot includes the following:

[0076] Step 1: Control the robotic arm to drive camera 1 to take pictures of the calibration board in different poses, collect the pose coordinates under the tightening axis coordinate system and the corresponding center coordinates of the calibration board in the pixel coordinate system, and solve the visual pose mapping relationship from the pixel coordinate system to the tightening axis coordinate system based on the corresponding coordinates.

[0077] The visual pose mapping relationship involves solving the planar perspective transformation matrix, and the process includes:

[0078] The method for extracting the center coordinates of the calibration board in the pixel coordinate system includes: sequentially performing grayscale conversion, Gaussian filtering, and adaptive threshold segmentation on the acquired calibration board image to obtain a binary image; performing edge detection and contour analysis on the binary image to filter out loop-shaped contours; performing sub-pixel level edge optimization on the filtered contours; and calculating the coordinates of the minimum circumcircle center of its outer ring contour as the center pixel coordinates. ;

[0079] The obtained tightening axis coordinates With center pixel coordinates The corresponding data pairs constitute the sample dataset, in which the number of sets of tightening axis coordinates and pixel coordinates are four or more.

[0080] Based on the sample dataset, the planar perspective transformation matrix is ​​calculated by solving the least-squares solution of the linear equation system. .

[0081] Solving the least-squares solution of a system of linear equations to obtain the planar perspective transformation matrix includes:

[0082] Planar perspective transformation matrix For a 3×3 matrix, the expression is:

[0083] ,

[0084] in, to For matrix The parameters to be determined (due to the scale invariance of homogeneous coordinates, they are usually set to...) (After normalization, there are actually 8 degrees of freedom). These parameters together define the rotation, translation, and perspective transformation relationships between the pixel coordinate system and the tightening axis coordinate system. Their specific values ​​are determined by solving a system of linear equations using multiple sets of corresponding coordinate data pairs through the calibration process.

[0085] According to the homogeneous coordinate transformation relationship Expanding into a non-homogeneous proportional form:

[0086] ,

[0087] ,

[0088] After cross-multiplication to eliminate the denominator, for each pair of corresponding points... Two linear equations are obtained:

[0089] ;

[0090] Based on n sets of corresponding points, construct a system of 2n linear equations and represent it in matrix form. Where A is a 2n×9 coefficient matrix, ;

[0091] ,

[0092] ,

[0093] The system of equations is ,because Solving by the least squares method makes smallest ;

[0094] Perform singular value decomposition on coefficient matrix A , where V is a 9×9 orthogonal matrix, and the last column of matrix V is taken as the least squares solution of the system of equations;

[0095] vector Reconstruct it into a 3×3 matrix and normalize it, let... The planar perspective transformation matrix is ​​obtained. .

[0096] Step 2: Control the robotic arm to move to the preset ideal working posture and take an image of the target workpiece 2. Extract the coordinates of the center point of the target workpiece 2 in the pixel coordinate system and its rotation angle from the image using a visual recognition algorithm.

[0097] The target workpiece 2 is specifically the fine-tuning claw tightening groove.

[0098] The center point coordinates and rotation angle of the target workpiece 2 are extracted using a visual recognition algorithm, including:

[0099] Control the robotic arm to move to the preset ideal photo position. And acquire images;

[0100] The YOLOv11 target detection model is used to coarsely locate the target workpiece 2 in the image, and its preliminary bounding box, center point coordinates and rotation angle are output.

[0101] Using the coarsely located bounding box as the center, expand outwards by a preset number of pixels to extract the ROI (region of interest) image containing the complete target workpiece 2;

[0102] For the ROI image, perform the following subpixel-level fine localization operations in sequence:

[0103] a. Grayscale conversion;

[0104] b. Use the Otsu algorithm for threshold segmentation to separate the tightening groove from the background;

[0105] c. Perform an edge smoothing algorithm to eliminate edge burrs from adhering to the background;

[0106] d. Based on a preset area threshold, filter out noisy regions and select candidate target regions;

[0107] e. Use the ORB corner detection algorithm to extract corner features of candidate target regions;

[0108] f. Perform linear fitting on the extracted corner points to determine the edge of the target workpiece 2;

[0109] Based on the fitted shape, calculate its geometric center coordinates. Used as the center point coordinates and rotation angle of the target workpiece 2.

[0110] Step 3: Based on the visual pose mapping relationship, the relative relationship between the center point coordinates of the target workpiece 2 and the pixel coordinates corresponding to the ideal working pose is mapped to the target offset in the tightening axis coordinate system, and combined with the coordinates of the ideal working pose, the target coordinates guiding the operation of the robotic arm end effector are synthesized.

[0111] The relative relationship is mapped to the target offset and combined with the coordinates of the ideal working pose to synthesize the target coordinates, which is achieved through the following mathematical formula:

[0112] ;

[0113] in, The tightening axis coordinates are for the ideal working position. These are the theoretical pixel coordinates corresponding to this pose. The coordinates of the center point of the identified target workpiece 2, The perspective transformation matrix is ​​a planar matrix. Tighten the axis coordinates of the target obtained from the solution.

[0114] Mathematical formula Its establishment is based on the following principles:

[0115] Defined in the ideal working position Below, the actual transformation from pixel coordinates to tightening axis coordinates is: , so that:

[0116] ,

[0117] and ;

[0118] Define the transformation matrix obtained through calibration as follows: , so that:

[0119] ,

[0120] ;

[0121] in, According to the transformation relationship with pixel coordinates The corresponding tightening axis coordinates; According to the transformation relationship with pixel coordinates The corresponding tightening axis coordinates;

[0122] Based on the invariance of coordinate offset in the tightening axis coordinate system, we have:

[0123] ;

[0124] Will and Substituting into the above equation, we get:

[0125] ,

[0126] The formula can be obtained by rearranging the steps.

[0127] Step 4: Based on the target coordinates and the identified rotation angle, control the movement of the robotic arm and end effector to complete the operation on the target workpiece 2.

[0128] After obtaining the target tightening axis coordinates Then, control the robotic arm to move to the final target position. ,in The fixed positional deviation between the optical center of camera 1 and the mechanical center of the tightening axis in the tightening axis coordinate system, obtained by manual measurement or tool calibration.

[0129] When guiding the end effector of the robotic arm, the programmable logic controller (PLC) generates motion control instructions based on the target coordinates. Through multi-axis collaborative control logic, it drives the movement of each joint of the robotic arm and simultaneously adjusts the spatial posture of the tightening shaft according to the identified rotation angle of the target workpiece 2, so that the axis of the tightening shaft is aligned with the center line of the target workpiece 2.

[0130] Example 2:

[0131] This embodiment of a track slab fine-tuning robot's robotic arm guidance system includes:

[0132] The calibration module controls the robotic arm and camera 1 to perform calibration movements and processes the calibration plate image to construct a visual pose mapping relationship from the pixel coordinate system to the tightening axis coordinate system. The vision recognition module processes the acquired image of the target workpiece 2 when the robotic arm is in a preset ideal working pose and outputs the center point coordinates and rotation angle of the target workpiece 2 in the pixel coordinate system. The coordinate mapping calculation module connects the calibration module and the vision recognition module. It receives the visual pose mapping relationship, center point coordinates, and ideal working pose information, and calculates and outputs the target coordinates to guide the robotic arm operation. The motion control module connects the coordinate mapping calculation module and the vision recognition module. It drives the robotic arm and end effector to complete the operation based on the target coordinates and rotation angle.

[0133] The coordinate mapping solution module includes an offset calculation unit and a coordinate synthesis unit. The offset calculation unit is used to calculate the physical offset between the center point coordinates of the target workpiece 2 and the pixel coordinates of the ideal working pose based on the visual pose mapping relationship. The coordinate synthesis unit is used to add the physical offset to the tightening axis coordinates of the ideal working pose to synthesize the target coordinates.

[0134] Example 3:

[0135] As shown in the figure, the robotic arm guidance method for the track slab fine-tuning robot provided in this embodiment includes the following steps:

[0136] Step 1: Calibrate the transformation matrix between the camera's 1-pixel coordinate system and the tightening axis coordinate system. .

[0137] like Figure 6 As shown, the aim is to establish a precise visual pose mapping relationship between the pixel coordinate system (with the top left corner of the image as the origin, the horizontal axis as the u-axis (pixel column direction), and the vertical axis as the v-axis (pixel row direction) and the tightening axis coordinate system (with the robotic arm base as the origin, the horizontal axis as the X-axis, and the vertical axis as the Y-axis). The specific implementation process is as follows:

[0138] Scene setup and data collection preparation:

[0139] like Figure 4 As shown, the high-precision circular calibration plate is placed horizontally directly below the field of view of camera 1. During placement, the upper surface of the calibration plate must be calibrated using a level to ensure it is parallel to the working reference plane of the robotic arm. Adjust the vertical distance between the calibration plate and camera 1, and remove dust, oil, and other impurities from the surface of the calibration plate to avoid obstructing or contaminating the image clarity. This ensures that camera 1 can completely and clearly capture all feature areas of the calibration plate from any preset shooting position. In the calibration control module of the host computer software, multiple sets of precise shooting positions for camera 1 (such as...) can be preset and set through a visual interactive interface. Figure 3 As shown), and the tightening axis coordinate parameters for each group of shooting positions (format: It is stored in the software cache in real time, and each location is marked with a unique identifier for easy subsequent calls and status tracing.

[0140] Automated image acquisition:

[0141] The host computer software sends a photo-taking position call command to the PLC via the S7 protocol. After receiving the command, the PLC parses the tightening axis coordinate parameters of the corresponding photo-taking position in the cache, generates and sends motion control commands to the robotic arm controller. The robotic arm controller drives each joint to move along a preset trajectory, smoothly moving the industrial camera 1 installed on the end effector to the designated photo-taking position. Once the robotic arm has moved to the correct position, the PLC sends a "ready" signal back to the host computer. After receiving the signal, the host computer sends a photo-taking trigger command through the camera 1 SDK. Camera 1 takes a picture at the current position, containing a complete image of the calibration board features, and transmits the image data back to the host computer's image cache in real time. Simultaneously, it records the association information between the image and the corresponding photo-taking position, completing the photo-taking for a single position. This process is then repeated sequentially for all preset photo-taking positions until multiple sets (n≥4) of calibration images have been acquired.

[0142] Calibration board image processing and center coordinate extraction:

[0143] For each set of calibration board images returned from the shooting positions, traditional vision algorithms are used to extract high-precision center pixel coordinates. The process is as follows:

[0144] Image preprocessing: First, grayscale processing is performed to convert the color image into a single-channel grayscale image, reducing color information interference and computational load; then, Gaussian filtering is used to smooth the image to suppress high-frequency noise caused by light fluctuations and camera sensor noise during shooting.

[0145] Binarization and contour separation: The grayscale image is converted into a binary image through adaptive thresholding. The grayscale difference between the calibration plate and the background is used to clearly separate the calibration plate area from the background area, ensuring that the edges of the curve pattern are fully highlighted.

[0146] Contour extraction and optimization: The contour edges of the loop pattern are extracted using an edge detection algorithm (such as the Canny algorithm). Reasonable high and low thresholds are set to ensure accurate capture of inner and outer ring edges and filter out false edges. The target contours that meet the preset size and shape characteristics of the loop calibration plate are selected using a contour analysis algorithm. Sub-pixel level edge optimization is performed on the selected contours, and the edge positioning accuracy is improved from the pixel level to the sub-pixel level using an interpolation algorithm.

[0147] Center coordinate calculation: After obtaining the high-precision loop profile, a geometric center calculation algorithm is used. The center of the smallest circumcircle of its outer ring profile is used as the center reference of the calibration plate. The edge points of the outer ring profile are fitted using the least squares method to construct the circumcircle equation, and the center coordinates of the circle are then calculated. This refers to the coordinates of the center pixel in the current image, expressed in homogeneous coordinate form. .

[0148] Data binding and transformation matrix Solution:

[0149] Each group of valid center pixel coordinates The tightening axis coordinates corresponding to the image acquisition time Bind the data to ensure that the two data points are from the same acquisition time, forming a "tightening axis coordinate - pixel coordinate" corresponding data pair, and store it in the data file.

[0150] Based on the collected n sets (n≥4) of corresponding data pairs, solve for the planar perspective transformation matrix. The matrix is ​​a 3×3 matrix, and its expression is:

[0151] ,

[0152] According to the homogeneous coordinate transformation relationship Expanding this into a non-homogeneous proportional form and eliminating the denominators through cross-multiplication, two linear equations are obtained for each pair of corresponding points. Based on n pairs of corresponding points, a system of linear equations containing 2n equations is constructed and expressed in matrix form. Where A is a 2n×9 coefficient matrix, Since 2n > 9, the system of equations is overdetermined and can be solved using the least squares method. Singular value decomposition (SVD) is then performed on the coefficient matrix A. Where V is a 9×9 orthogonal matrix, the last column of matrix V is taken as the least squares solution of the system of equations. . This vector Reconstruct it into a 3×3 matrix and then normalize it (let...). That is, the planar perspective transformation matrix is ​​obtained. .

[0153] Step 2: Visually identify the target workpiece 2.

[0154] The target workpiece 2 is specifically the fine-tuning claw tightening groove.

[0155] In actual operation, the host computer software will preset an ideal shooting position. The robotic arm is moved to that position and an image containing the fine-tuning claw is taken.

[0156] The host computer software calls the visual recognition module to process the image. This module combines coarse localization using a target detection model with fine localization using traditional visual algorithms.

[0157] Coarse localization: The YOLOv11 model is used to detect the fine-tuning claw tightening groove in the image, and its preliminary bounding box, two-dimensional coordinates of the center point, and rotation angle relative to the horizontal direction are output.

[0158] Fine localization: Based on the coarse localization results of YOLOv11, the detection box is expanded outwards by a certain number of pixels to extract the ROI (Region of Interest) image. Then, the following operations are performed on this ROI image:

[0159] a. Grayscale conversion;

[0160] b. Threshold segmentation: The Otsu algorithm is used to achieve pixel-level separation between the tightening groove and the background;

[0161] c. Edge smoothing: Eliminates edge burrs from adhering to the background;

[0162] d. Region filtering: Filter background noise based on area thresholds, retaining target regions that meet size characteristics;

[0163] e. Corner detection: The ORB corner detection algorithm is used to extract the corners and edge feature points of the tightening groove;

[0164] f. Rectangle detection: Line fitting is performed on the extracted corner points to determine the four edges of the tightening groove, and finally, the coordinates of its center point are accurately calculated. And rotation angle. Through this process, positioning accuracy can be improved to the sub-pixel level.

[0165] Step 3: Coordinate transformation.

[0166] like Figure 5 As shown, in actual operation, it is necessary to determine the pixel coordinates of the identified tightening groove. Transform to the tightening axis coordinate system .

[0167] This invention does not directly solve for the transformation relationships in actual operations. Instead, it cleverly utilizes the calibrated matrix. The calculation is performed indirectly based on the principle of invariance of coordinate offset in the tightening axis coordinate system. The core formula is as follows:

[0168] ;

[0169] in, Ideal photo location The corresponding theoretical pixel coordinates. The derivation principle of this formula is as follows:

[0170] Defined in the ideal working position Below, the actual transformation from pixel coordinates to tightening axis coordinates is: , so that:

[0171] ,

[0172] and ;

[0173] Define the transformation matrix obtained through calibration as follows: , so that:

[0174] ,

[0175] ;

[0176] in, According to the transformation relationship with pixel coordinates The corresponding tightening axis coordinates; According to the transformation relationship with pixel coordinates The corresponding tightening axis coordinates;

[0177] Based on the invariance of coordinate offset in the tightening axis coordinate system, we have:

[0178] ;

[0179] Will and Substituting into the above equation, we get:

[0180] ,

[0181] The formula can be obtained by rearranging the steps.

[0182] The derivation principle of this formula ensures that coordinate transformation can be completed with high precision even if there are errors in the installation of camera 1 or the surface of the tightening groove is not level.

[0183] Step 4: Guide the tightening shaft to insert.

[0184] The PLC tightens the axis coordinates according to the converted target. The system generates and sends precise motion control commands to the robotic arm's motion actuators. These commands, through multi-axis collaborative control logic, drive each joint of the robotic arm to move along a preset trajectory. The robotic arm first moves to the target position. Considering that there is a fixed offset between the center of the field of view of camera 1 and the physical center of the tightening axis. The robotic arm eventually moved to the position. Meanwhile, the PLC fine-tunes the spatial posture of the tightening shaft based on the rotation angle of the tightening groove output by the vision recognition module, ultimately achieving high-precision alignment between the axis of the tightening shaft and the center line of the tightening groove of the fine-tuning claw. This ensures that the tightening shaft is inserted into the groove smoothly and accurately, thus laying the foundation for subsequent track plate fine-tuning operations.

[0185] Through practical engineering applications, the robotic arm guidance method provided in this embodiment has the following significant advantages:

[0186] By introducing a planar perspective transformation matrix to establish a global visual pose mapping relationship, the dependence of traditional methods on ideal vertical projection conditions is effectively overcome, and it can adapt to actual camera 1 installation errors and changes in the posture of the working surface. An indirect coordinate mapping solution strategy based on relative offsets is adopted, cleverly avoiding the direct solution of complex coordinate transformation relationships, ensuring both accuracy and improving computational efficiency. By integrating deep learning models with traditional vision algorithms, rapid and accurate identification of the tightening groove of the fine-tuning claw is achieved, with positioning accuracy reaching sub-pixel level. The entire guidance process is highly automated, requiring no manual intervention, significantly improving the efficiency and reliability of track slab fine-tuning operations. The method has strong versatility and adaptability, and can meet the operational needs of different models of track slab fine-tuning robots.

Claims

1. A method for guiding the robotic arm of a track slab fine-tuning robot, characterized in that, Includes the following: Control the robotic arm to drive the camera (1) to take pictures of the calibration plate in different poses, collect the pose coordinates in the tightening axis coordinate system and the corresponding center coordinates of the calibration plate in the pixel coordinate system, and solve the visual pose mapping relationship from the pixel coordinate system to the tightening axis coordinate system based on the corresponding coordinates. Control the robotic arm to move to the preset ideal working posture and take an image of the target workpiece (2). Extract the coordinates of the center point of the target workpiece (2) in the pixel coordinate system and its rotation angle from the image through a visual recognition algorithm. Based on the visual pose mapping relationship, the relative relationship between the center point coordinates of the target workpiece (2) and the pixel coordinates corresponding to the ideal working pose is mapped to the target offset in the tightening axis coordinate system, and combined with the coordinates of the ideal working pose, the target coordinates that guide the operation of the end effector of the robotic arm are synthesized. Based on the target coordinates and the identified rotation angle, the movement of the robotic arm and end effector is controlled to complete the operation on the target workpiece (2).

2. The robotic arm guidance method according to claim 1, characterized in that, The visual pose mapping relationship is obtained by solving the planar perspective transformation matrix, and the process includes: The method for extracting the center coordinates of the calibration board in the pixel coordinate system includes: sequentially performing grayscale conversion, Gaussian filtering, and adaptive threshold segmentation on the acquired calibration board image to obtain a binary image; performing edge detection and contour analysis on the binary image to filter out loop-shaped contours; performing sub-pixel level edge optimization on the filtered contours; and calculating the coordinates of the minimum circumcircle center of its outer ring contour as the center pixel coordinates. ; The obtained tightening axis coordinates With center pixel coordinates The corresponding data pairs constitute the sample dataset, in which the number of sets of tightening axis coordinates and pixel coordinates are four or more. Based on the sample dataset, the planar perspective transformation matrix is ​​calculated by solving the least-squares solution of the linear equation system. .

3. The robotic arm guidance method according to claim 2, characterized in that, The process of solving the least-squares solution of the linear equation system to obtain the planar perspective transformation matrix includes: The planar perspective transformation matrix For a 3×3 matrix, the expression is: ， in, to For matrix The parameters to be determined; According to the homogeneous coordinate transformation relationship Expanding into a non-homogeneous proportional form: ， ， After cross-multiplication to eliminate the denominator, for each pair of corresponding points... Two linear equations are obtained: ， Based on n sets of corresponding points, construct a system of 2n linear equations and represent it in matrix form. Where A is a 2n×9 coefficient matrix, ; ， ， The system of equations is ,because Solving by the least squares method makes smallest ; Perform singular value decomposition on coefficient matrix A , where V is a 9×9 orthogonal matrix, and the last column of matrix V is taken as the least squares solution of the system of equations; The vector Reconstruct it into a 3×3 matrix and normalize it, let... The planar perspective transformation matrix is ​​obtained. .

4. The robotic arm guidance method according to claim 1, characterized in that, The extraction of the center point coordinates and rotation angle of the target workpiece (2) through a visual recognition algorithm includes: Control the robotic arm to move to the preset ideal photo position. And acquire images; The target detection model is used to coarsely locate the target workpiece (2) in the image and output its preliminary bounding box, center point coordinates and rotation angle; Using the coarsely positioned bounding box as the center, expand outwards by a preset number of pixels to extract the ROI (region of interest) image containing the complete target workpiece (2); For the ROI image, perform the following subpixel-level fine localization operations in sequence: a. Grayscale processing; b. Use the Otsu algorithm for threshold segmentation to separate the tightening groove from the background; c. Perform an edge smoothing algorithm to eliminate edge burrs from adhering to the background; d. Based on a preset area threshold, filter out noisy regions and select candidate target regions; e. Use the ORB corner detection algorithm to extract the corner features of the candidate target region; f. Perform line fitting on the extracted corner points to determine the edge of the target workpiece (2); Based on the fitted shape, calculate its geometric center coordinates. The coordinates of the center point and the rotation angle of the target workpiece (2).

5. The robotic arm guidance method according to claim 2, characterized in that, The process of mapping the relative relationship to the target offset and combining it with the coordinates of the ideal working pose to synthesize the target coordinates is achieved through the following mathematical formula: ； in, Let be the tightening axis coordinate of the ideal working position. These are the theoretical pixel coordinates corresponding to this pose. The coordinates of the center point of the identified target workpiece (2) Let be the planar perspective transformation matrix. Tighten the axis coordinates of the target obtained by solving.

6. The robotic arm guidance method according to claim 5, characterized in that, The mathematical formula Its establishment is based on the following principles: Defined in the ideal working position Below, the actual transformation from pixel coordinates to tightening axis coordinates is: , so that: ， and ; Define the transformation matrix obtained through calibration as follows: , so that: ， ； in, According to the transformation relationship With pixel coordinates The corresponding tightening axis coordinates; According to the transformation relationship With pixel coordinates The corresponding tightening axis coordinates; Based on the invariance of coordinate offset in the tightening axis coordinate system, we have: ； Will and Substituting into the above equation, we get: ， The formula is obtained by rearranging the components.

7. The robotic arm guidance method according to claim 6, characterized in that, After obtaining the target tightening axis coordinates Then, control the robotic arm to move to the final target position. ,in The fixed offset between the pre-calibrated camera (1) field of view center and the center of the tightening shaft of the tightening machine.

8. The robotic arm guidance method according to claim 1, characterized in that, When guiding the end effector of the robotic arm, the programmable logic controller (PLC) generates motion control instructions based on the target coordinates. Through multi-axis collaborative control logic, it drives the movement of each joint of the robotic arm and simultaneously adjusts the spatial posture of the tightening shaft according to the rotation angle of the identified target workpiece (2), so that the axis of the tightening shaft is aligned with the center line of the target workpiece (2).

9. A robotic arm guidance system for a track slab fine-tuning robot, characterized in that, include: The calibration module is used to control the robotic arm and the camera (1) to perform calibration movements and process the calibration board image to construct the visual pose mapping relationship from the pixel coordinate system to the tightening axis coordinate system. The visual recognition module is used to process the image of the target workpiece (2) when the robotic arm is in the preset ideal working posture, and output the center point coordinates and rotation angle of the target workpiece (2) in the pixel coordinate system. The coordinate mapping calculation module connects the calibration module and the vision recognition module. It is used to receive the visual pose mapping relationship, the center point coordinates and the ideal working pose information, and to calculate and output the target coordinates to guide the operation of the robotic arm. The motion control module connects the coordinate mapping and calculation module and the vision recognition module, and is used to drive the robotic arm and end effector to complete the operation based on the target coordinates and rotation angle.

10. The robotic arm guidance system according to claim 9, characterized in that: The coordinate mapping solution module includes an offset calculation unit and a coordinate synthesis unit; the offset calculation unit is used to calculate the physical offset between the center point coordinates of the target workpiece (2) and the pixel coordinates of the ideal working pose based on the visual pose mapping relationship; the coordinate synthesis unit is used to add the physical offset to the tightening axis coordinates of the ideal working pose to synthesize the target coordinates.

Images