Weight replacement positioning method based on machine vision and PLC control

Abstract

The invention discloses a heavy object replacement positioning method based on machine vision and PLC control, and belongs to the technical field of heavy object replacement in high-altitude narrow space. According to the method, firstly, a calibrated 3D camera is used for collecting three-dimensional point clouds of stone pier bolt holes and weight hole positions, pose data are extracted through preprocessing and feature matching, and coordinate conversion is achieved through hand-eye calibration and transmitted to a PLC. And then the PLC analyzes rotation and translation parameters required by the weight according to a preset algorithm, controls the hydraulic cylinder and the linear module to adjust the pose of the weight, gradually completes rotation and X-axis and Y-axis movement, and achieves alignment of each hole site of the weight and a corresponding stone pier bolt hole. The device can realize automatic precision positioning of heavy object replacement in a high-altitude narrow environment, has the advantages of high precision, strong adaptability, safe and reliable operation and the like, and can effectively solve the problem of difficulty in manual alignment.

Description

Technical Field

[0001] This invention relates to the field of heavy object replacement technology in high-altitude confined spaces, specifically to a heavy object replacement positioning method based on machine vision and PLC control. Background Technology

[0002] Replacing heavy objects in a confined space at a high altitude presents significant challenges. Space constraints, a narrow operating area, limited personnel movement, and the inability to deploy large equipment are all significant issues. Furthermore, due to manufacturing errors in the heavy object, the four holes on the heavy object may not perfectly align with the four holes on the stone pier. The four holes on the heavy object may also have positional deviations, and the four cylindrical prongs may not be straight but rather angled. Manual verification cannot confirm that bolts can pass through all four holes on both the heavy object and the stone pier, leading to errors in installation accuracy and ultimately preventing precise installation of the heavy object.

[0003] Therefore, the method of combining machine vision and PLC control is used to replace heavy objects in high-altitude and confined spaces. On-site operators can remotely control the PLC, which greatly improves the accuracy of installation and also ensures the safety of workers. Summary of the Invention

[0004] The technical problem to be solved by this invention is: how to solve the problems of low installation accuracy and low installation efficiency during the replacement of heavy objects in high-altitude and confined spaces, accurately position the heavy objects, and thus make the heavy objects reach the optimal position. This invention provides a heavy object replacement positioning method based on machine vision and PLC control, which can install heavy objects more accurately and improve replacement efficiency.

[0005] The present invention solves the above-mentioned technical problems through the following technical solution, and the present invention includes the following steps:

[0006] S1: Use two calibrated 3D cameras to acquire images of four bolt holes on the stone pier, correct the images using the intrinsic and extrinsic parameters of the 3D cameras, and generate point clouds based on the depth map and color map;

[0007] S2: Calculate the normal vector of the point cloud, remove outliers, set the region of interest in 3D space, the point cloud within the region will be retained, and the point cloud outside the region will be removed to obtain the 3D point cloud;

[0008] S3: Extract the edges of the 3D point cloud, then merge multiple point clouds in the point cloud list into one point cloud to obtain the original point cloud;

[0009] S4: Roughly match the point cloud template with the original point cloud to output the rough candidate pose of the target object; based on the 3D coarse matching, perform 3D fine matching to accurately match the point cloud template with the original point cloud and output the best pose of the target object; after 3D fine matching, remove the poses of the superimposed objects.

[0010] S5: Hand-eye calibration is used to transform the camera coordinate system and the actuator coordinate system.

[0011] S6: Perform mutual transformation between pose quaternions and Euler angles, and transmit the data to the PLC;

[0012] S7: The 3D camera transmits the following data to the PLC: center coordinates of bolt hole No. 1 on the stone pier ( , ), the center coordinates of bolt hole No. 2 on the stone pier ( , ), the center coordinates of bolt hole No. 3 on the stone pier ( , ), the center coordinates of bolt hole No. 4 on the stone pier ( , ), radius of bolt holes on stone piers ;

[0013] S8: The PLC controls the actuator to move the heavy object into the camera's field of view;

[0014] S9: Use the calibrated 3D camera to acquire images of the holes on the weight, correct the images using the intrinsic and extrinsic parameters of the 3D camera, generate point clouds using depth and color maps, and then repeat steps S2 to S6.

[0015] S10: The 3D camera transmits the following data to the PLC: coordinates of the center of the upper surface of the weight at hole number 1 ( ), the center coordinates of the upper surface of hole No. 2 of the weight ( ), the center coordinates of the upper surface of the weight at hole position 3 ( ), the center coordinates of the upper surface of hole No. 4 of the weight ( ), coordinates of the center of the lower surface of hole No. 1 of the weight ( ), the center coordinates of the lower surface of the weight at hole number 2 ( ), the center coordinates of the lower surface of the weight at hole position 3 ( ), the center coordinates of the lower surface of the weight at hole number 4 ( ), the center coordinates of the weight ( ), radius of the hole on the weight ;

[0016] S11: Based on the data obtained in step S10, calculate the effective center of the first hole of the weight. Effective radius of hole No. 1 of the weight The effective center of the weight at hole position 2 Effective radius of hole No. 2 of the weight The effective center of the weight at hole position 3 Effective radius of hole No. 3 of the weight The effective center of the weight at hole position 4 Effective radius of hole No. 4 of the weight And calculate the installation error of hole position 1 of the weight. Installation error of hole No. 2 of the weight Installation error of hole No. 3 of the weight Installation error of hole No. 4 of the weight ;

[0017] S12: Based on the data obtained in steps S7 and S11, calculate the rotation angle range of the weight, and the PLC controls the actuator to complete the rotation of the weight.

[0018] S13: After completing the rotation of the weight, repeat steps S9 to S11 to recalculate the effective center of hole position 1 of the weight. The effective center of the weight at hole position 2 The effective center of the weight at hole position 3 The effective center of the weight at hole position 4 ;

[0019] S14: Based on the data obtained in steps S7 and S13, calculate the translation range of the weight in the X-axis direction, and the PLC controls the actuator to complete the translation of the weight in the X-axis direction.

[0020] S15: After completing the translation of the weight in the X-axis direction, repeat steps S9 to S11 to recalculate the effective center of the weight at hole position 1. The effective center of the weight at hole position 2 The effective center of the weight at hole position 3 The effective center of the weight at hole position 4 ;

[0021] S16: Based on the data obtained in steps S7 and S15, calculate the translation range of the weight in the Y-axis direction. The PLC controls the actuator to complete the translation of the weight in the Y-axis direction, thereby enabling the weight to reach the optimal position and completing the weight replacement and positioning work.

[0022] Furthermore, in step S1, after calibration and verification of the accuracy of intrinsic and extrinsic parameters using the Zhang Zhengyou calibration method, the 3D camera is fixed to the detection bracket, and color images of the left and right views of the four bolt holes of the stone pier are acquired simultaneously. Based on the intrinsic and extrinsic parameters, image distortion correction and stereo registration are completed in sequence. The disparity map is calculated and converted into a depth map through a semi-global block matching algorithm. After removing invalid depth values, the pixel coordinates of the color image and the corresponding depth values ​​are combined with the intrinsic parameters to perform inverse perspective projection transformation, generating a three-dimensional point cloud bound to RGB color information.

[0023] Furthermore, in step S2, outlier deletion is achieved using the weighted nearest neighbor method. This method introduces weights during calculation, increasing the weight of neighbors closer to the target point and decreasing the weight of neighbors farther from it. In the weighted nearest neighbor method, for a given query point x, its predicted value... The calculation is as follows:

[0024] ;

[0025] in, It is the value of the i-th neighbor. It is the weight of the i-th neighbor;

[0026] The weighting function uses inverse distance weights, and its expression is:

[0027] ;

[0028] in, To query the distance from a point to its i-th neighbor, For exponential parameters, To prevent division by zero of constants.

[0029] Furthermore, in step S3, the specific processing procedure is as follows:

[0030] S31: Calculate the normal vector of each point in the point cloud, analyze the rate of change of the normal vector in the neighborhood, and determine the point whose rate of change exceeds the threshold as an edge point;

[0031] S32: By aligning coordinates and fusion data, point clouds from different sources are unified into the same coordinate system and then integrated into a whole, thus achieving point cloud merging.

[0032] Furthermore, in step S4, the specific processing procedure is as follows:

[0033] S41: Coarse Matching

[0034] Point cloud features are extracted using ISS, SIFT3D, or FPFH algorithms. RANSAC algorithm is used for feature-based coarse registration. The number of RANSAC algorithm iterations and feature matching radius are set. Multiple candidate poses are generated by sampling different initial positions or parameters. Clustering methods are used to remove duplicates from similar poses.

[0035] S42: Fine Matching

[0036] The ICP algorithm is used for fine registration, and iterative optimization is performed based on the coarse registration results. The maximum number of iterations and distance threshold of the ICP algorithm are set, and the best pose is selected based on the registration error and ratio.

[0037] S43: Pose Selection

[0038] After fine matching, depth collision detection is used to check for collisions with other objects. The number of detection viewpoints and minimum spacing are set to filter the poses and remove the poses of overlapping objects.

[0039] Furthermore, in step S6, the formulas for transforming the pose quaternion and Euler angles are as follows:

[0040] Euler angles to quaternions:

[0041] ;

[0042] ;

[0043] ;

[0044] ;

[0045] in, Let be the rotation angles about the Z, Y, and X axes, respectively. Quaternions are defined as follows:

[0046] q= (||q||=1);

[0047] Quaternions to Euler Angles:

[0048] atan2(2( + ),1-2( + ));

[0049] =arcsin(2( - ));

[0050] =atan2(2( + ),1-2( + ));

[0051] in, , , , These represent the four components of a quaternion.

[0052] Furthermore, in step S11, the calculated parameters are as follows:

[0053] Effective center of hole No. 1 of the weight =( ), effective radius ;

[0054] Effective center of hole No. 2 of the weight =( ), effective radius ;

[0055] Effective center of hole No. 3 of the weight =( ), effective radius ;

[0056] Effective center of hole No. 4 of the weight =( ), effective radius ;

[0057] Installation error range for hole position 1 of the weight: = ;

[0058] Installation error range for hole position 2 of the weight: = ;

[0059] Installation error range for hole No. 3 of the weight: = ;

[0060] Installation error range for hole No. 4 of the weight: = ;

[0061] in, Let be the radius of the hole on the stone pier. Let be the radius of the hole on the weight.

[0062] Furthermore, in step S12, the rotation angle range of the weight is calculated, and the PLC controls the actuator to complete the rotation of the weight, with counterclockwise as the positive direction, and the weight rotating around its center point (…). Rotate by an angle θ as the center of rotation; the formula for calculating the effective center of each hole on the weight is as follows:

[0063] Effective center of hole No. 1 of the weight:

[0064] ;

[0065] Effective center of hole No. 2 of the weight:

[0066] ;

[0067] Effective center of hole No. 3 of the weight:

[0068] ;

[0069] Effective center of hole No. 4 on the weight:

[0070] ;

[0071] rotation angle of the weight Within the specified range, the following conditions must be met to ensure that all four pairs of holes can accommodate the bolt:

[0072] ;

[0073] ;

[0074] ;

[0075] ;

[0076] Calculated The range is ,make = .

[0077] Furthermore, in step S13, the weight completes its rotation. After the angle, Incorporate the effective center of the calculated weight at hole position 1 The effective center of the weight at hole position 2 The effective center of the weight at hole position 3 The effective center of the weight at hole position 4 as follows:

[0078] Effective center of hole No. 1 of the weight:

[0079] ;

[0080] Effective center of hole No. 2 of the weight:

[0081] ;

[0082] Effective center of hole No. 3 of the weight:

[0083] ;

[0084] Effective center of hole No. 4 on the weight:

[0085] .

[0086] Furthermore, in step S15, after completing the translation of the weight in the X-axis direction, the effective centers of each hole on the weight are calculated again as follows:

[0087] Effective center of hole No. 1 of the weight:

[0088] ;

[0089] Effective center of hole No. 2 of the weight:

[0090] ;

[0091] Effective center of hole No. 3 of the weight:

[0092] ;

[0093] Effective center of hole No. 4 on the weight:

[0094] .

[0095] Furthermore, in step S12, the rotation of the weight is achieved by pushing it with two hydraulic cylinders on the diagonal. First, it is determined that... The positive and negative signs, if If the result is positive, then hydraulic cylinders 1 and 4 are used to push the weight. If the value is negative, then hydraulic cylinders No. 2 and No. 3 are used to push the weight, and the pushing distance is... The calculation formula is as follows:

[0096] ;

[0097] in, Let be the side length of the upper surface of the weight. The angle of rotation of the weight. This refers to the distance between two hydraulic cylinders on the same side.

[0098] In step S14, the following calculation is performed based on the condition that all four pairs of holes must be able to allow the bolt to pass through: The range is ,make The PLC determines the movement of the weight along the X-axis. The sign of the sign determines the direction of the translation. If the result is positive, then hydraulic cylinders 1 and 3 are used to push the cylinder. The distance; if If the value is negative, use hydraulic cylinders #2 and #4 to push. The absolute distance is used to move the weight in the X direction. The distance;

[0099] In step S16, the following calculation is performed based on the condition that all four pairs of holes must be able to allow the bolt to pass through: The range is ,make = The PLC controls the movement of the linear module by controlling the rotation of the servo motor. The weight moves along the linear module in the Y direction. Relative positioning is used in the PLC, and the relative position is set to... This allows the heavy object to move in the Y direction. The distance, of which, This is the transmission ratio between the servo motor and the linear module.

[0100] Compared with the prior art, the present invention has the following advantages: This heavy object replacement and positioning method based on machine vision and PLC control achieves fully automatic and high-precision positioning and replacement of heavy objects in a confined space by using the high-precision three-dimensional point cloud reconstruction and pose recognition of the machine vision system and the precise closed-loop control of the actuator by the PLC. It is especially suitable for complex scenarios with processing errors, significantly improving the accuracy, efficiency and safety of the operation, and reducing the dependence on manual labor and operational risks. Attached Figure Description

[0101] Figure 1 This is a flowchart illustrating the heavy object replacement and positioning method based on machine vision and PLC control in an embodiment of the present invention.

[0102] Figure 2 This is a schematic diagram of the hole distribution on the stone pier in an embodiment of the present invention;

[0103] Figure 3 This is a schematic diagram of the hole distribution on the weight in an embodiment of the present invention;

[0104] Figure 4 This is a schematic diagram of the structure of the actuator in an embodiment of the present invention. Detailed Implementation

[0105] The embodiments of the present invention are described in detail below. These embodiments are implemented based on the technical solution of the present invention, and provide detailed implementation methods and specific operation processes. However, the scope of protection of the present invention is not limited to the following embodiments.

[0106] like Figure 1 As shown, this embodiment provides a technical solution: a method for heavy object replacement and positioning based on machine vision and PLC control, including the following steps:

[0107] S1: Use two calibrated 3D cameras to acquire images of four bolt holes on the stone pier, correct the images using the intrinsic and extrinsic parameters of the 3D cameras, and generate point clouds based on the depth map and color map;

[0108] S2: Calculate the normal vector of the point cloud, remove outliers, and set a region of interest (ROI) in 3D space. Points within the region will be retained, and points outside the region will be removed to obtain a 3D point cloud.

[0109] S3: Extract the edges of the 3D point cloud, then merge multiple point clouds in the point cloud list into one point cloud to obtain the original point cloud;

[0110] S4: Roughly match the point cloud template with the original point cloud to output the rough candidate pose of the target object; based on the 3D coarse matching, perform 3D fine matching to accurately match the point cloud template with the original point cloud and output the best pose of the target object; after 3D fine matching, remove the poses of the superimposed objects.

[0111] S5: Hand-eye calibration is used to transform the camera coordinate system and the actuator coordinate system.

[0112] S6: Perform mutual transformation between pose quaternions and Euler angles, and transmit the data to the PLC;

[0113] S7: The 3D camera transmits the following data to the PLC: center coordinates of bolt hole No. 1 on the stone pier ( , ), the center coordinates of bolt hole No. 2 on the stone pier ( , ), the center coordinates of bolt hole No. 3 on the stone pier ( , ), the center coordinates of bolt hole No. 4 on the stone pier ( , ), radius of bolt holes on stone piers ;

[0114] S8: The PLC controls the actuator to move the heavy object into the camera's field of view;

[0115] S9: Use the calibrated 3D camera to acquire images of the holes on the weight, correct the images using the intrinsic and extrinsic parameters of the 3D camera, generate point clouds using depth and color maps, and then repeat steps S2 to S6.

[0116] S10: The 3D camera transmits the following data to the PLC: coordinates of the center of the upper surface of the weight at hole number 1 ( ), the center coordinates of the upper surface of hole No. 2 of the weight ( ), the center coordinates of the upper surface of the weight at hole position 3 ( ), the center coordinates of the upper surface of hole No. 4 of the weight ( ), coordinates of the center of the lower surface of hole No. 1 of the weight ( ), the center coordinates of the lower surface of the weight at hole number 2 ( ), the center coordinates of the lower surface of the weight at hole position 3 ( ), the center coordinates of the lower surface of the weight at hole number 4 ( ), the center coordinates of the weight ( ), radius of the hole on the weight ;

[0117] S11: Based on the data obtained in S10, calculate the effective center of the circle at hole position 1 of the weight. Effective radius of hole No. 1 of the weight The effective center of the weight at hole position 2 Effective radius of hole No. 2 of the weight The effective center of the weight at hole position 3 Effective radius of hole No. 3 of the weight The effective center of the weight at hole position 4 Effective radius of hole No. 4 of the weight And calculate the installation error of hole position 1 of the weight. Installation error of hole No. 2 of the weight Installation error of hole No. 3 of the weight Installation error of hole No. 4 of the weight ;

[0118] S12: Based on the data obtained in steps S7 and S11, calculate the rotation angle range of the weight, and the PLC controls the actuator to complete the rotation of the weight.

[0119] S13: After completing the rotation angle of the weight, repeat steps S9 to S11 to recalculate the effective center of hole position 1 of the weight. The effective center of the weight at hole position 2 The effective center of the weight at hole position 3 The effective center of the weight at hole position 4 ;

[0120] S14: Based on the data obtained in steps S7 and S13, calculate the translation range of the weight in the X-axis direction, and the PLC controls the actuator to complete the translation of the weight in the X-axis direction.

[0121] S15: After completing the translation of the weight in the X-axis direction, repeat steps S9 to S11 to recalculate the effective center of the weight at hole position 1. The effective center of the weight at hole position 2 The effective center of the weight at hole position 3 The effective center of the weight at hole position 4 ;

[0122] S16: Based on the data obtained in steps S7 and S15, calculate the translation range of the weight in the Y-axis direction. The PLC controls the actuator to complete the translation of the weight in the Y-axis direction, thereby enabling the weight to reach the optimal position.

[0123] In this embodiment, in step S1, camera calibration obtains the camera's internal and external parameters, establishing a correspondence between points in the two-dimensional image and points in three-dimensional space. Specifically, the camera's internal parameters generally include the camera's focal length, distortion coefficients, pixel pitch, center point coordinates, and the width and height of the image. The camera's external parameters represent the camera's three-dimensional position in the world coordinate system, such as the camera's X-axis, Y-axis, and Z-axis coordinates, as well as the camera's orientation and rotation angles around the X, Y, and Z axes. Assuming the object point's coordinates in the world coordinate system are... =[X,Y,Z], then the coordinates of the corresponding point in the 2D image are =[u,v].

[0124] In more specific terms, arrive The coordinate transformation of conforms to the following relationship:

[0125] z =K

[0126] Where K= Indicates the camera's internal parameters. , This corresponds to the horizontal and vertical spacing between adjacent pixels. R represents the camera's external parameters. This represents the angle of rotation around the X, Y, and Z axes. t represents the camera's X, Y, and Z coordinates. Therefore: z = .

[0127] In this embodiment, camera calibration uses DLT for initial estimation.

[0128] z =K Known (u,v), To find K, R, and t for a coordinate system (X, Y, Z), we need to know a set of 2D-3D correspondences and the 2D and 3D coordinates of the corresponding points.

[0129] z =

[0130] =

[0131] =

[0132] =

[0133] Two equations can be set up for each pair of points, and a minimum of 6 pairs of points are needed to solve the equations. That is, at least 6 pairs of points need to be calibrated to complete the calibration. Of course, the more points calibrated, the better the robustness and the smaller the error.

[0134] In this embodiment, after calibration and verification of the accuracy of intrinsic parameters (focal length, principal point, distortion coefficient) and extrinsic parameters (rotation matrix, translation vector) using the Zhang Zhengyou calibration method, the 3D camera is fixed to the detection bracket, and color images of the four holes of the stone pier from the left and right perspectives are acquired simultaneously. Based on the intrinsic and extrinsic parameters, image distortion correction and stereo registration are completed in sequence. The disparity map is calculated by the semi-global block matching algorithm and converted into a depth map. After removing invalid depth values, the pixel coordinates of the color image and the corresponding depth values ​​are combined with the intrinsic parameters to perform inverse perspective projection transformation, generating a three-dimensional point cloud bound to RGB color information.

[0135] In this embodiment, in step S2, the normal vector of the point cloud is calculated, and outliers are removed. Specifically, to address this issue, the weighted nearest neighbor method introduces weights during the calculation process, increasing the weight of neighbors closer to the target point and decreasing the weight of neighbors farther from the target point. For a given query point x, its predicted value... The calculation is as follows:

[0136]

[0137] in, It is the value of the i-th neighbor. It is the weight of the i-th neighbor;

[0138] The weighting function uses inverse distance weights, and its expression is:

[0139]

[0140] in, To query the distance from a point to its i-th neighbor, The parameter is an exponential parameter (usually 1 or 2), but here we use 1. It is a very small constant (to prevent division by zero).

[0141] In this embodiment, in step S2, a region of interest (ROI) is set in 3D space. The point cloud within the region is preserved, while the point cloud outside the region is removed. The specific method involves concepts of spatial geometry and set theory. Here, a cylindrical ROI is used, with the mathematical condition being the distance from a point to the axis. And the projection along the axis lies within [0, h], as shown in the following mathematical expression:

[0142] R=

[0143] Where a is the axis vector, b is the base point, r is the radius, and h is the height.

[0144] In this embodiment, in step S3, the edges of the 3D point cloud are extracted, and then multiple point clouds in the point cloud list are merged into one point cloud to facilitate subsequent processing of the point cloud as a whole. The point cloud edge extraction method is edge detection based on normal vector changes. The mathematical principle is: calculate the normal vector of each point, analyze the rate of change of the normal vector in the neighborhood, and determine points whose rate of change exceeds a threshold as edge points. The method of merging multiple point clouds into one point cloud mainly consists of two major steps: coordinate transformation alignment and data fusion processing. Its core is to unify point clouds from different sources to the same coordinate system and then integrate them into a whole. The specific methods and principles are as follows:

[0145] The local coordinate system is transformed into the global coordinate system through rotation and translation. The alignment method uses iterative nearest point, which finds the closest point pairs between two sets of point clouds iteratively and calculates the optimal transformation matrix that minimizes the sum of squared distances between the point pairs.

[0146] In this embodiment, the specific operation method for roughly matching the point cloud template with the original point cloud in step S4 is as follows: First, feature extraction is performed using algorithms such as ISS, SIFT3D, or FPFH to extract point cloud features. Then, the RANSAC algorithm is used for feature-based coarse registration, with 100,000 iterations and a feature matching radius of 0.05m. Multiple candidate poses are generated by sampling different initial positions or parameters. Clustering methods are used to remove duplicates from similar poses. Finally, 3D fine matching is performed to accurately match the point cloud template with the original point cloud, outputting a more accurate pose for the target object. Specifically, ICP fine registration is used, and iterative optimization is performed based on the coarse registration results. The maximum ICP iteration is 200, and the distance threshold is 0.02m. The optimal pose is selected based on the registration error and ratio. Point cloud interference from overlapping objects can cause deviations in the calculation of the target object's contour and size. Removing this interference ensures accurate analysis based solely on the complete point cloud of the independent object. After 3D fine matching, the specific operation to remove the poses of the superimposed objects is as follows: First, depth collision detection is used to check for collisions with other objects. The number of detection viewpoints is 3, and the minimum spacing is set to 0.05m. Finally, the poses are filtered.

[0147] In this embodiment, the specific operation method for the mutual transformation between the camera coordinate system and the actuator coordinate system in step S5 is as follows: First, the transformation from the camera to the base is obtained through hand-eye calibration. Based on the forward kinematics of the robot, the transformation from the end effector to the base is as follows: The coordinate transformation formula is as follows:

[0148] From camera coordinate system to actuator coordinate system:

[0149]

[0150] From the actuator coordinate system to the camera coordinate system:

[0151]

[0152] In this embodiment, the formula for transforming the pose quaternion and Euler angles in step S6 is as follows:

[0153] Euler angles to quaternions:

[0154]

[0155]

[0156]

[0157]

[0158] in, Let q be the rotation angles about the Z, Y, and X axes, respectively. The quaternion is defined as: q = (||q||=1);

[0159] Quaternions to Euler Angles:

[0160] atan2(2( + ),1-2( + ))

[0161] =arcsin(2( - ))

[0162] =atan2(2( + ),1-2( + ))

[0163] in, , , , These represent the four components of the quaternion ( For the real part, , , (This is the imaginary part).

[0164] In this embodiment, in step S6, the camera outputs data to the PLC as follows: The PLC calls the imported MM_Start_Vis function (FC), enables it, and sets Startr_Vision to "on," thus starting the camera project. After the camera project is completed, the PLC calls the imported MM_Get_VisData function (FC), enables it, and sets Get_VisData to "on," storing the coordinate data in the Target_Pose structure in the PLC's variable table.

[0165] In this embodiment, in step S8, the specific operation of the actuator to transport the heavy object into the camera's field of view is as follows: the PLC controls two servo motors to rotate synchronously, and the rotation of the two servo motors drives the linear module to move sequentially. The distance is detected in real time by the laser rangefinder, and the target distance is set. When the laser sensor reaches the target distance, the PLC will stop the rotation of the servo motors, thereby completing the transportation of the heavy object into the camera's field of view.

[0166] In this embodiment, in step S11, the effective center of the first hole of the weight is calculated. Effective radius of hole No. 1 of the weight The effective center of the weight at hole position 2 Effective radius of hole No. 2 of the weight The effective center of the weight at hole position 3 Effective radius of hole No. 3 of the weight The effective center of the weight at hole position 4 Effective radius of hole No. 4 of the weight And calculate the installation error of hole position 1 of the weight. Installation error of hole No. 2 of the weight Installation error of hole No. 3 of the weight Installation error of hole No. 4 of the weight as follows:

[0167] Effective center of hole No. 1 of the weight =( ), effective radius

[0168] Effective center of hole No. 2 of the weight =( ), effective radius

[0169] Effective center of hole No. 3 of the weight =( ), effective radius

[0170] Effective center of hole No. 4 of the weight =( ), effective radius

[0171] Installation error range for hole position 1 of the weight: =

[0172] Installation error range for hole position 2 of the weight: =

[0173] Installation error range for hole No. 3 of the weight: =

[0174] Installation error range for hole No. 4 of the weight: =

[0175] in, Let be the radius of the hole on the stone pier. Let be the radius of the hole on the weight.

[0176] In this embodiment, in step S12, the rotation angle range of the weight is calculated, and the PLC controls the actuator to complete the rotation of the weight, with counterclockwise as the positive direction, and the weight around its center point ( Rotate by an angle θ as the center of rotation. After rotating by an angle θ, the effective center of the weight at hole position 1 is... The effective center of the weight at hole position 2 The effective center of the weight at hole position 3 The effective center of the weight at hole position 4 as follows:

[0177] Effective center of hole No. 1 of the weight:

[0178] ;

[0179] Effective center of hole No. 2 of the weight:

[0180] ;

[0181] Effective center of hole No. 3 of the weight:

[0182] ;

[0183] Effective center of hole No. 4 on the weight:

[0184] ;

[0185] rotation angle of the weight Within the specified range, the following conditions must be met for all four pairs of holes to allow the bolt to pass through:

[0186]

[0187]

[0188] Calculated The range is ,make = .

[0189] In this embodiment, in step S12, the PLC controls the actuator to complete the angle correction. The specific method is as follows: the rotation of the weight is achieved by pushing with two hydraulic cylinders on the diagonal. First, it is determined that... The positive and negative signs, if If the result is positive, then hydraulic cylinders 1 and 4 are used to push the weight. If the value is negative, then hydraulic cylinders No. 2 and No. 3 are used to push the weight, and the pushing distance is... The calculation formula is as follows:

[0190]

[0191] in, Let be the side length of the upper surface of the weight. The angle of rotation of the weight. This refers to the distance between two hydraulic cylinders on the same side.

[0192] In this embodiment, the weight completes its rotation in step S13. After the angle, Incorporate the effective center of the calculated weight at hole position 1 The effective center of the weight at hole position 2 The effective center of the weight at hole position 3 The effective center of the weight at hole position 4 as follows:

[0193] Effective center of hole No. 1 of the weight:

[0194] ;

[0195] Effective center of hole No. 2 of the weight:

[0196] ;

[0197] Effective center of hole No. 3 of the weight:

[0198] ;

[0199] Effective center of hole No. 4 on the weight:

[0200] .

[0201] In this embodiment, in step S14, the translation range of the weight in the X-axis direction is calculated, and the PLC controls the actuator to complete the translation of the weight in the X-axis direction so that the bolts can pass through all four pairs of holes. The following conditions must be met:

[0202]

[0203]

[0204] It can be found that: Scope ,make The translation of the weight along the X-axis is achieved by pushing it with two hydraulic cylinders on the same side. First, determine... The positive and negative signs, if If the result is positive, then hydraulic cylinders 1 and 3 are used to push the weight. If the value is negative, then hydraulic cylinders No. 2 and No. 4 are used to push the weight, and the pushing distance is... The absolute value of.

[0205] In this embodiment, in step S15, the weight is translated in the X-axis direction. Next, calculate the effective center of the circle at hole position 1 of the weight. The effective center of the weight at hole position 2 The effective center of the weight at hole position 3 The effective center of the weight at hole position 4 as follows:

[0206] Effective center of hole No. 1 of the weight:

[0207] ;

[0208] Effective center of hole No. 2 of the weight:

[0209] ;

[0210] Effective center of hole No. 3 of the weight:

[0211] ;

[0212] Effective center of hole No. 4 on the weight:

[0213] .

[0214] In this embodiment, in step S16, the translation range of the weight in the Y-axis direction is calculated, and the PLC controls the actuator to complete the translation of the weight in the Y-axis direction. For the bolts to pass through all four pairs of holes, the following conditions must be met:

[0215]

[0216]

[0217] It can be found that: Scope ,make = The PLC controls the rotation of the servo motor to control the movement of the linear module, thereby moving the heavy object in the Y direction. The PLC uses the relative positioning of the servo motor and sets the relative position... (K is the transmission ratio between the servo motor and the linear module), thus enabling the linear module to move the weight in the Y direction. .

[0218] like Figure 2 , 3 The diagrams shown are schematics of the hole distribution on the stone pier and the heavy object, respectively, and represent the view from which a 3D camera is installed on one side of the stone pier and the heavy object.

[0219] like Figure 4 The diagram shown is a structural schematic of the actuator in this embodiment. 1-1 is a linear module guide rail, 1-2 is a linear module slider, 2 is a servo motor, 3-1 is hydraulic cylinder 1, 3-2 is hydraulic cylinder 2, 3-3 is hydraulic cylinder 3, and 3-4 is hydraulic cylinder 4. The weight is located between hydraulic cylinders 1, 2, 3, and 4. The servo motor, in conjunction with the linear module, can drive the combination of the weight, hydraulic cylinders 1, 2, 3, and 4 to move.

[0220] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A method for changing and positioning heavy objects based on machine vision and PLC control, characterized in that, Includes the following steps: S1: Use two calibrated 3D cameras to acquire images of four bolt holes on the stone pier, correct the images using the intrinsic and extrinsic parameters of the 3D cameras, and generate point clouds based on the depth map and color map; S2: Calculate the normal vector of the point cloud, remove outliers, set the region of interest in 3D space, the point cloud within the region will be retained, and the point cloud outside the region will be removed to obtain the 3D point cloud; S3: Extract the edges of the 3D point cloud, then merge multiple point clouds in the point cloud list into one point cloud to obtain the original point cloud; S4: Roughly match the point cloud template with the original point cloud to output the rough candidate pose of the target object; based on the 3D coarse matching, perform 3D fine matching to accurately match the point cloud template with the original point cloud and output the best pose of the target object; after 3D fine matching, remove the poses of the superimposed objects. S5: Hand-eye calibration is used to transform the camera coordinate system and the actuator coordinate system. S6: Perform mutual transformation between pose quaternions and Euler angles, and transmit the data to the PLC; S7: The 3D camera transmits the following data to the PLC: center coordinates of bolt hole No. 1 on the stone pier ( , ), the center coordinates of bolt hole No. 2 on the stone pier ( , ), the center coordinates of bolt hole No. 3 on the stone pier ( , ), the center coordinates of bolt hole No. 4 on the stone pier ( , ), radius of bolt holes on stone piers ; S8: The PLC controls the actuator to move the heavy object into the camera's field of view; S9: Use the calibrated 3D camera to acquire images of the holes on the weight, correct the images using the intrinsic and extrinsic parameters of the 3D camera, generate point clouds using depth and color maps, and then repeat steps S2 to S6. S10: The 3D camera transmits the following data to the PLC: coordinates of the center of the upper surface of the weight at hole number 1 ( ), the center coordinates of the upper surface of hole No. 2 of the weight ( ), the center coordinates of the upper surface of the weight at hole position 3 ( ), the center coordinates of the upper surface of hole No. 4 of the weight ( ), coordinates of the center of the lower surface of hole No. 1 of the weight ( ), the center coordinates of the lower surface of the weight at hole number 2 ( ), the center coordinates of the lower surface of the weight at hole position 3 ( ), the center coordinates of the lower surface of the weight at hole number 4 ( ), the center coordinates of the weight ( ), radius of the hole on the weight ; S11: Based on the data obtained in step S10, calculate the effective center of the first hole of the weight. Effective radius of hole No. 1 of the weight The effective center of the weight at hole position 2 Effective radius of hole No. 2 of the weight The effective center of the weight at hole position 3 Effective radius of hole No. 3 of the weight The effective center of the weight at hole position 4 Effective radius of hole No. 4 of the weight And calculate the installation error of hole position 1 of the weight. Installation error of hole No. 2 of the weight Installation error of hole No. 3 of the weight Installation error of hole No. 4 of the weight ; S12: Based on the data obtained in steps S7 and S11, calculate the rotation angle range of the weight, and the PLC controls the actuator to complete the rotation of the weight. S13: After completing the rotation of the weight, repeat steps S9 to S11 to recalculate the effective center of hole position 1 of the weight. The effective center of the weight at hole position 2 The effective center of the weight at hole position 3 The effective center of the weight at hole position 4 ; S14: Based on the data obtained in steps S7 and S13, calculate the translation range of the weight in the X-axis direction, and the PLC controls the actuator to complete the translation of the weight in the X-axis direction. S15: After completing the translation of the weight in the X-axis direction, repeat steps S9 to S11 to recalculate the effective center of the weight at hole position 1. The effective center of the weight at hole position 2 The effective center of the weight at hole position 3 The effective center of the weight at hole position 4 ; S16: Based on the data obtained in steps S7 and S15, calculate the translation range of the weight in the Y-axis direction. The PLC controls the actuator to complete the translation of the weight in the Y-axis direction, thereby enabling the weight to reach the optimal position and completing the weight replacement and positioning work.

2. The method for heavy object replacement and positioning based on machine vision and PLC control according to claim 1, characterized in that, In step S1, after calibration and verification of the accuracy of intrinsic and extrinsic parameters using the Zhang Zhengyou calibration method, the 3D camera is fixed to the detection bracket, and color images of the left and right views of the four bolt holes of the stone pier are acquired simultaneously. Based on the intrinsic and extrinsic parameters, image distortion correction and stereo registration are completed in sequence. The disparity map is calculated and converted into a depth map through a semi-global block matching algorithm. After removing invalid depth values, the pixel coordinates of the color image and the corresponding depth values ​​are combined with the intrinsic parameters to perform inverse perspective projection transformation, generating a three-dimensional point cloud with bound RGB color information.

3. The method for heavy object replacement and positioning based on machine vision and PLC control according to claim 1, characterized in that, In step S2, outlier deletion is achieved using the weighted nearest neighbor method. This method introduces weights during calculation, increasing the weight of neighbors closer to the target point and decreasing the weight of neighbors farther away. In the weighted nearest neighbor method, for a given query point x, its predicted value... The calculation is as follows: ； in, It is the value of the i-th neighbor. It is the weight of the i-th neighbor; The weighting function uses inverse distance weights, and its expression is: ； in, To query the distance from a point to its i-th neighbor, For exponential parameters, To prevent division by zero of constants.

4. The method for changing and positioning heavy objects based on machine vision and PLC control according to claim 1, characterized in that, In step S3, the specific processing procedure is as follows: S31: Calculate the normal vector of each point in the point cloud, analyze the rate of change of the normal vector in the neighborhood, and determine the point whose rate of change exceeds the threshold as an edge point; S32: By aligning coordinates and fusion data, point clouds from different sources are unified into the same coordinate system and then integrated into a whole, thus achieving point cloud merging.

5. The method for heavy object replacement and positioning based on machine vision and PLC control according to claim 1, characterized in that, In step S6, the formulas for transforming pose quaternions and Euler angles are as follows: Euler angles to quaternions: ； ； ； ； in, Let be the rotation angles about the Z, Y, and X axes, respectively. Quaternions are defined as follows: ； Quaternions to Euler Angles: atan2(2( + )1-2( + )); = arcsin(2( But )); =atan2(2( + )1-2( + )); in, , , , These represent the four components of a quaternion.

6. The method for heavy object replacement and positioning based on machine vision and PLC control according to claim 1, characterized in that, In step S11, the calculated parameters are as follows: Effective center of hole No. 1 of the weight =( ), effective radius ; Effective center of hole No. 2 of the weight =( ), effective radius ; Effective center of hole No. 3 of the weight =( ), effective radius ; Effective center of hole No. 4 of the weight =( ), effective radius ; Installation error range for hole position 1 of the weight: = ; Installation error range for hole position 2 of the weight: = ; Installation error range for hole No. 3 of the weight: = ; Installation error range for hole No. 4 of the weight: = ; in, Let be the radius of the hole on the stone pier. Let be the radius of the hole on the weight.

7. The method for heavy object replacement and positioning based on machine vision and PLC control according to claim 6, characterized in that, In step S12, the rotation angle range of the weight is calculated, and the PLC controls the actuator to complete the rotation of the weight, with counterclockwise as the positive direction, and the weight around its center point ( Rotate by an angle θ as the center of rotation; the formula for calculating the effective center of each hole on the weight is as follows: Effective center of hole No. 1 of the weight: ； Effective center of hole No. 2 of the weight: ； Effective center of hole No. 3 of the weight: ； Effective center of hole No. 4 on the weight: ； rotation angle of the weight Within the specified range, the following conditions must be met to ensure that all four pairs of holes can accommodate the bolt: ； ； ； ； Calculated The range is ,make = .

8. The method for heavy object replacement and positioning based on machine vision and PLC control according to claim 7, characterized in that, In step S13, the weight completes the rotation. After the angle, Incorporate the effective center of the calculated weight at hole position 1 The effective center of the weight at hole position 2 The effective center of the weight at hole position 3 The effective center of the weight at hole position 4 as follows: Effective center of hole No. 1 of the weight: ； Effective center of hole No. 2 of the weight: ； Effective center of hole No. 3 of the weight: ； Effective center of hole No. 4 on the weight: 。 9. A method for changing and positioning heavy objects based on machine vision and PLC control according to claim 8, characterized in that, In step S15, after the translation of the weight in the X-axis direction is completed, the effective centers of each hole on the weight are calculated again as follows: Effective center of hole No. 1 of the weight: ； Effective center of hole No. 2 of the weight: ； Effective center of hole No. 3 of the weight: ； Effective center of hole No. 4 on the weight: 。 10. A method for heavy object replacement and positioning based on machine vision and PLC control according to claim 9, characterized in that, In step S12, the rotation of the weight is achieved by pushing it with two hydraulic cylinders on the diagonal. First, it is determined that... The positive and negative signs, if If the result is positive, then hydraulic cylinders 1 and 4 are used to push the weight. If the value is negative, then hydraulic cylinders No. 2 and No. 3 are used to push the weight, and the pushing distance is... The calculation formula is as follows: ； in, Let be the side length of the upper surface of the weight. The angle of rotation of the weight. This refers to the distance between two hydraulic cylinders on the same side. In step S14, the following calculation is performed based on the condition that all four pairs of holes must be able to allow the bolt to pass through: The range is ,make The PLC determines the movement of the weight along the X-axis. The sign of the sign determines the direction of the translation. If the result is positive, then hydraulic cylinders 1 and 3 are used to push the cylinder. The distance; if If the value is negative, use hydraulic cylinders #2 and #4 to push. The absolute distance is used to move the weight in the X direction. The distance; In step S16, the following calculation is performed based on the condition that all four pairs of holes must be able to allow the bolt to pass through: The range is ,make = The PLC controls the movement of the linear module by controlling the rotation of the servo motor. The weight moves along the linear module in the Y direction. Relative positioning is used in the PLC, and the relative position is set to... This allows the heavy object to move in the Y direction. The distance, of which, This is the transmission ratio between the servo motor and the linear module.

Images