Hydraulic punching robot system based on laser and vision fusion

Abstract

The invention relates to a hydraulic punching robot system based on laser and vision fusion, and belongs to the technical field of industrial automation, the system comprises a calibration measurement and calculation unit, a pose conversion unit, a vision calibration unit and an operation management and control unit; operation module self-inspection and multi-terminal calibration are carried out, and a target space pose is calculated through three-point coordinate measurement; the target space pose is converted to a mechanical arm base coordinate system, motion execution parameters are calculated, and a motion control instruction is generated; a target image is obtained, preprocessing and feature extraction are carried out, the position deviation between the motion execution end and the target reference point is solved, and the execution end is driven to carry out adjustment; after operation is completed, the execution end is driven to reset to the initial position, whole-process operation data are recorded, the state of the punching operation process is monitored in real time, and alarm is triggered and operation is stopped in time when abnormal operation is detected. Laser and vision fusion positioning is adopted, the punching precision and the operation automation level are improved, and it is guaranteed that operation is stable and reliable.

Description

Technical Field

[0001] This invention belongs to the field of industrial automation technology, specifically relating to a hydraulic drilling robot system based on laser and vision fusion. Background Technology

[0002] In industrial manufacturing and precision parts processing, drilling is a fundamental and critical process. Its positioning accuracy, efficiency, and safety directly impact product quality, production cycle time, and enterprise operating costs. Traditional drilling operations rely entirely on manual labor for core operations, including manual control of robotic arms, manual positioning of the drilling target, and manual monitoring of the process. Due to the inherent limitations of manual operation, the industry has long faced numerous technical challenges and production difficulties that urgently need to be addressed.

[0003] Manual positioning suffers from poor accuracy and instability, making it difficult to control drilling tolerances. Drilling positioning accuracy is highly dependent on the operator's experience and skill level, and is easily affected by subjective factors such as operator fatigue and operating condition. This results in significant fluctuations in positioning accuracy, making it difficult to meet the high precision requirements for drilling tolerances in precision machining scenarios. Consequently, machining defects such as hole position deviations and hole diameter discrepancies are prone to occur, reducing product yield.

[0004] The slow work cycle cannot meet the demands of mass production. Manually adjusting the robotic arm's posture and matching the positioning target takes a lot of time, the drilling cycle for a single workpiece is long, and the overall production efficiency is low. It is difficult to adapt to the large-scale, batch production rhythm of modern industry, which restricts the company's capacity improvement.

[0005] The operation poses significant safety hazards, and labor costs continue to rise. Operators need to operate the drilling equipment at close range, making them susceptible to risks such as mechanical compression and laser radiation, which poses a significant challenge to operational safety. At the same time, with changes in the labor market, labor costs are increasing year by year, and the long-term manual operation mode has increased the burden on enterprises' production and operation, squeezing profit margins.

[0006] The operation process lacks data-driven management, resulting in insufficient process optimization and quality traceability capabilities. Traditional drilling operations lack systematic methods for recording operational data, making it impossible to collect and store key data such as positioning accuracy, drilling time, and process parameters in real time. This hinders both accurate analysis and iterative optimization of the production process and the ability to trace and investigate the entire process when processing quality issues arise, leading to a low level of precision in production control.

[0007] Based on this, this invention combines the actual needs of industrial production with multiple technologies such as robotic arm kinematic modeling, laser positioning, machine vision, multi-sensor calibration, and upper computer integrated control to upgrade traditional drilling machines into robots, creating a hydraulic drilling robot system that integrates laser and vision. This system automates the entire drilling process, aiming to solve the problems of accuracy, efficiency, safety, and control in traditional manual drilling operations, and improve the intelligence and precision of enterprise production. Summary of the Invention

[0008] To address the aforementioned problems in the existing technology, this invention provides a hydraulic drilling robot system based on laser and vision fusion.

[0009] The objective of this invention can be achieved through the following technical solution: a hydraulic drilling robot system based on laser and vision fusion, comprising: a calibration and calculation unit, a pose conversion unit, a vision calibration unit, and an operation control unit; The calibration and calculation unit performs self-checks on the operation module, simultaneously calibrates multiple ends, obtains drilling process parameters, and calculates the target spatial pose using a three-point coordinate calculation method. The pose conversion unit converts the target spatial pose to the robot arm base coordinate system, calculates the corresponding motion execution parameters through kinematic modeling, generates motion control commands, drives the execution end to move to the target area, and outputs the target area's fitting and positioning status. The visual calibration unit acquires a target image, performs preprocessing and feature extraction, converts the extracted target feature points into actual spatial coordinates, calculates the positional deviation between the motion execution end and the target reference point, and drives the execution end to make adjustments based on the deviation calculation results. After the operation is completed, the operation control unit drives the execution end to reset to the initial position, records the entire process data of the operation, monitors the status of the drilling operation in real time, and triggers an alarm and suspends the operation in a timely manner when an operation abnormality is detected.

[0010] Specifically, the calibration process for the multiple ends is as follows: zero-point position calibration is performed on the motion execution end, while accuracy calibration and measurement parameter adjustment and matching are performed on the spatial positioning end, and detection parameter calibration and equipment calibration are performed on the vision inspection end. The calibration operations of the motion execution end, spatial positioning end, and vision inspection end are performed synchronously with the loading of the drilling process parameters corresponding to the workpiece to be processed.

[0011] Specifically, the process of calculating the target spatial pose using the three-point coordinate measurement method is as follows: align the target origin and two non-collinear feature points around the target to obtain the three-dimensional spatial coordinates. Using the target origin as a reference, obtain the normal vector of the plane where the target is located. Define the basis vectors of the target coordinate system accordingly. Perform normalization and orthogonalization processing on the basis vectors in sequence to obtain the unit orthogonal vectors of the target coordinate system. Arrange the unit orthogonal vectors in columns and combine them with the three-dimensional spatial coordinates of the target origin to form a homogeneous transformation matrix.

[0012] Specifically, the process of converting the target spatial pose to the robotic arm base coordinate system is as follows: calibrating the extrinsic parameters of the spatial positioning end and the visual detection end relative to the robotic arm base coordinate system, obtaining the transformation relationship between the detection end coordinate system and the robotic arm base coordinate system, performing coordinate system transformation operations on the homogeneous transformation matrix corresponding to the target spatial pose, converting the target pose information with the spatial positioning end coordinate system as a reference into pose data with the robotic arm base coordinate system as a reference, and generating the target pose in the robotic arm base coordinate system.

[0013] Specifically, the process of calculating the corresponding motion execution parameters through kinematic modeling is as follows: A kinematic model including forward and inverse directions is constructed based on the structural characteristics of the motion execution end. A transformation matrix between joints is established based on the improved link parameter method. The forward kinematic model is obtained through the product of the transformation matrices. Then, using the target pose in the robot arm's base coordinate system as known conditions, the motion parameters of redundant joints are preset based on the inverse kinematic model. The rotation angle parameters of the rotary joints and the extension / retraction parameters of the telescopic joints are calculated and generated. Simultaneously, the rationality of the preset motion range of the joints at the motion execution end is verified, and the motion execution parameters are generated.

[0014] Specifically, the process of generating motion control commands is as follows: the motion execution parameters are classified and organized according to the control dimensions of the joints at the motion execution end, the control register addresses and data formats corresponding to the joints are matched, and the classified motion execution parameters are converted into standardized control command data according to the preset communication protocol rules. At the same time, the timing logic and triggering identifier of motion execution are added to the control commands to generate the motion control commands.

[0015] Specifically, the process of outputting the bonding and positioning status of the target area is as follows: real-time acquisition of contact pressure data between the bonding component and the plane where the target is located, comparison with a preset bonding pressure threshold, and output of the bonding and positioning status of the target area based on the determination result.

[0016] Specifically, the preprocessing and feature extraction process is as follows: the target image is converted into a recognizable digital image, the digital image is filtered and denoised, and then the image's grayscale sum, grayscale center, and direction features are obtained through image moment operation. The target features are extracted by combining image moments with rotation, translation, and scaling invariance properties. At the same time, the extracted features are verified by template matching, and the target feature points are output.

[0017] Specifically, the process of converting the extracted target feature points into actual spatial coordinates is as follows: based on the target feature points and combined with the pre-calibrated camera intrinsic parameters, the pixel coordinates of the target feature points are converted into coordinates in the camera coordinate system. Then, based on the extrinsic parameter calibration results of the visual detection end relative to the robot arm base coordinate system, the coordinates of the target feature points in the camera coordinate system are converted into actual spatial coordinates in the robot arm base coordinate system through a coordinate system transformation matrix.

[0018] Specifically, the process of calculating the positional deviation between the motion execution end and the target reference point is as follows: obtain the actual spatial coordinates of the drilling operation end of the motion execution end in the robot arm base coordinate system, and at the same time, based on the actual spatial coordinates of the target reference point in the robot arm base coordinate system, perform multi-axis numerical comparison calculations to calculate the positional offset of the motion execution end relative to the target reference point in the motion axis, and integrate to generate the deviation calculation result.

[0019] Specifically, the process of driving the execution end to adjust based on the deviation calculation result is as follows: generating deviation compensation parameters based on the position offset in the deviation calculation result, performing a fusion calculation between the deviation compensation parameters and the original motion parameters of the execution end to generate the compensated motion control parameters of the execution end, converting the compensated motion control parameters into standardized adjustment commands according to a preset communication protocol, driving the electric adjustment mechanism of the execution end to adjust along the motion axis, and acquiring the position and pose data of the execution end in real time during the adjustment process and comparing it with the position of the target reference point.

[0020] Specifically, the process of triggering an alarm and suspending the operation in a timely manner when an abnormality is detected is as follows: continuously acquire the operation status data of the operation module, the pose data of the motion execution end, and the contact pressure data throughout the entire drilling operation process, compare them with the preset normal operation threshold range in real time, and based on the judgment result, trigger the system alarm mechanism and generate an operation suspension control command to stop the operation of the operation module.

[0021] The beneficial effects of this invention are as follows: To improve drilling positioning accuracy and meet the requirements of precision machining processes, a layered positioning strategy that integrates laser coarse positioning and vision fine positioning is adopted. This strategy specifically compensates for the shortcomings of insufficient control accuracy of hydraulically driven joints, eliminates the influence of experience and operating conditions on manual positioning, and improves positioning accuracy. This ensures that the positioning accuracy precisely meets the tolerance requirements of drilling precision parts, effectively reduces machining defects, and improves the overall product qualification rate.

[0022] Achieving fully automated operation and improving production efficiency: Through the coordinated cooperation of various functional units, the entire process of automated operation is completed, from system initialization, target positioning, robotic arm motion control to job reset and data recording. This replaces the traditional manual operation of robotic arms and target adjustment, shortens the drilling cycle of a single workpiece, adapts to the needs of large-scale and batch production, and effectively improves the overall production capacity of the enterprise.

[0023] Eliminating safety hazards of manual operation and reducing enterprise operating costs: The fully automated operation mode eliminates the need for operators to have close contact with the drilling equipment, completely avoiding operational safety risks such as mechanical extrusion and laser radiation, and significantly improving the operational safety of drilling operations; at the same time, it reduces the enterprise's dependence on professional operators, alleviates the production burden caused by the continuous increase in labor costs, and reduces the overall operating costs of the enterprise from both the aspects of safety protection and labor costs.

[0024] Achieving full-process data-driven management and control of operations facilitates process optimization and quality traceability. The operation control unit collects and records various key data in real time during the drilling process, while monitoring equipment operating status, bonding pressure, and other data throughout the process. This provides detailed data support for the analysis, iteration, and optimization of production processes. It also enables full-process traceability and investigation when processing quality problems occur, thereby improving the precision and digitalization of enterprise production management.

[0025] Adapted to the characteristics of hydraulic robotic arms, this design balances load capacity and positioning accuracy. Specifically addressing the high output torque and strong load-bearing capacity of hydraulic robotic arms, but with relatively low control precision, this design leverages laser coarse positioning to maximize load capacity, driving heavy-duty end effectors to move rapidly to the target area. Visual fine positioning and an electric adjustment mechanism then compensate for the accuracy shortcomings, achieving an organic combination of hydraulic drive advantages and high-precision positioning requirements. This design is suitable for heavy-duty, high-precision industrial drilling operations, enhancing the equipment's practicality and adaptability. Attached Figure Description

[0026] To facilitate understanding by those skilled in the art, the present invention will be further described below with reference to the accompanying drawings.

[0027] Figure 1 This is a system flowchart of the present invention; Figure 2 This is a schematic diagram of the structure in this invention; Figure 3 The diagram shows the structure of a robotic arm, with the following labels in order: 1-Joint 1 (rotary joint), 2-Joint 2 (rotary joint), 3-Joint 3 (telescopic joint), 4-Joint 4 (rotary joint), 5-Joint 5 (rotary joint), and 6-Joint 6 (rotary joint), illustrating the distribution of the joints in a six-degree-of-freedom hydraulic robotic arm. Figure 4 The diagram shows the coordinate distribution of each joint of the robotic arm. It includes the base coordinate system of the robotic arm and the local coordinate systems corresponding to each joint 1-6, demonstrating the spatial position and relative relationship of each joint coordinate system. Figure 5 A diagram of a drilling robot positioning system, showing the spatial assembly relationship of a laser positioning device, a vision camera, a hydraulic robotic arm, a target, and the workpiece to be processed. Figure 6 A diagram showing a spatial rectangular coordinate system and its position vector representation, labeled as: {A} - Spatial rectangular coordinate system. A P - An arbitrary point in space, representing the vector representation of a point in a Cartesian coordinate system; Figure 7 An attitude description diagram of the base coordinate system and the object coordinate system, where {A} - base coordinate system, {B} - object coordinate system. A X B / A Y B / A Z B -The unit column vector of the object's coordinate system relative to the base coordinate system. A P - Spatial location point; Figure 8 The diagram shows a laser positioning device, which includes a laser rangefinder, a two-degree-of-freedom gimbal, a target origin O, and target feature points A / B. It demonstrates the composition of the laser positioning device and the method for selecting target feature points. Figure 9 The target template detection diagram, with the following labels: target template - a preset target feature reference template, search image - the actual target image captured by the camera, and matching result - the overlapping matching area between the template and the search image, demonstrates the core principle and detection process of template matching; Figure 10 The schematic diagram of sensor extrinsic parameter calibration is labeled as: O t - Calibration plate coordinate system, O c -Camera coordinate system, O b - Robotic arm base coordinate system, O e - The coordinate system of the robotic arm's end effector suction cup demonstrates the spatial transformation relationship of each coordinate system at the sensor's external parameter calibration time.

[0028] General character explanations in the attached diagram: θ - joint rotation angle (°), d - link offset / extension (mm), α - link torsion angle (°), a - link length (mm), P c - Target space coordinates in the camera coordinate system, P b - Target space coordinates in the robot arm's base coordinate system t T c - The pose matrix of the calibration board coordinate system relative to the camera coordinate system, b T e - The pose matrix of the end effector suction cup of the robotic arm relative to the base coordinate system. Detailed Implementation

[0029] To further illustrate the technical means and effects of the present invention in achieving its intended purpose, the following detailed description of the specific implementation methods, structures, features, and effects of the present invention, in conjunction with the accompanying drawings and preferred embodiments, is provided.

[0030] Please see Figure 1-10 A hydraulic drilling robot system based on laser and vision fusion includes: a calibration and calculation unit, a pose conversion unit, a vision calibration unit, and an operation control unit; The calibration and calculation unit performs self-checks on the operation module, simultaneously calibrates multiple ends, obtains drilling process parameters, and calculates the target spatial pose using a three-point coordinate calculation method. The pose conversion unit converts the target spatial pose to the robot arm base coordinate system, calculates the corresponding motion execution parameters through kinematic modeling, generates motion control commands, drives the execution end to move to the target area, and outputs the target area's fitting and positioning status. The visual calibration unit acquires a target image, performs preprocessing and feature extraction, converts the extracted target feature points into actual spatial coordinates, calculates the positional deviation between the motion execution end and the target reference point, and drives the execution end to make adjustments based on the deviation calculation results. After the operation is completed, the operation control unit drives the execution end to reset to the initial position, records the entire process data of the operation, monitors the status of the drilling operation in real time, and triggers an alarm and suspends the operation in a timely manner when an operation abnormality is detected.

[0031] Specifically, the calibration process for the multiple ends is as follows: zero-point position calibration is performed on the motion execution end, while accuracy calibration and measurement parameter adjustment and matching are performed on the spatial positioning end, and detection parameter calibration and equipment calibration are performed on the vision inspection end. The calibration operations of the motion execution end, spatial positioning end, and vision inspection end are performed synchronously with the loading of the drilling process parameters corresponding to the workpiece to be processed.

[0032] Specifically, the process of calculating the target spatial pose using the three-point coordinate measurement method is as follows: align the target origin and two non-collinear feature points around the target to obtain the three-dimensional spatial coordinates. Using the target origin as a reference, obtain the normal vector of the plane where the target is located. Define the basis vectors of the target coordinate system accordingly. Perform normalization and orthogonalization processing on the basis vectors in sequence to obtain the unit orthogonal vectors of the target coordinate system. Arrange the unit orthogonal vectors in columns and combine them with the three-dimensional spatial coordinates of the target origin to form a homogeneous transformation matrix.

[0033] Specifically, the process of converting the target spatial pose to the robotic arm base coordinate system is as follows: calibrating the extrinsic parameters of the spatial positioning end and the visual detection end relative to the robotic arm base coordinate system, obtaining the transformation relationship between the detection end coordinate system and the robotic arm base coordinate system, performing coordinate system transformation operations on the homogeneous transformation matrix corresponding to the target spatial pose, converting the target pose information with the spatial positioning end coordinate system as a reference into pose data with the robotic arm base coordinate system as a reference, and generating the target pose in the robotic arm base coordinate system.

[0034] Specifically, the process of calculating the corresponding motion execution parameters through kinematic modeling is as follows: A kinematic model including forward and inverse directions is constructed based on the structural characteristics of the motion execution end. A transformation matrix between joints is established based on the improved link parameter method. The forward kinematic model is obtained through the product of the transformation matrices. Then, using the target pose in the robot arm's base coordinate system as known conditions, the motion parameters of redundant joints are preset based on the inverse kinematic model. The rotation angle parameters of the rotary joints and the extension / retraction parameters of the telescopic joints are calculated and generated. Simultaneously, the rationality of the preset motion range of the joints at the motion execution end is verified, and the motion execution parameters are generated.

[0035] Specifically, the process of generating motion control commands is as follows: the motion execution parameters are classified and organized according to the control dimensions of the joints at the motion execution end, the control register addresses and data formats corresponding to the joints are matched, and the classified motion execution parameters are converted into standardized control command data according to the preset communication protocol rules. At the same time, the timing logic and triggering identifier of motion execution are added to the control commands to generate the motion control commands.

[0036] Specifically, the process of outputting the bonding and positioning status of the target area is as follows: real-time acquisition of contact pressure data between the bonding component and the plane where the target is located, comparison with a preset bonding pressure threshold, and output of the bonding and positioning status of the target area based on the determination result.

[0037] In this embodiment, the specific execution flow of the pose conversion unit of the hydraulic drilling robot system based on laser and vision fusion is described in detail below: Kinematic modeling of a robotic arm is fundamental for trajectory planning and control, and mainly includes two aspects: forward kinematics and inverse kinematics. Forward kinematics modeling aims to calculate the transformation matrix of the end effector relative to the robot's base coordinate system based on the known joint variables of the six joints. This matrix provides the position and orientation information of the end effector relative to the robot's base. Inverse kinematics modeling, on the other hand, aims to deduce the motion parameters of each joint of the robotic arm based on the position and orientation of the end effector relative to the base coordinate system.

[0038] Forward kinematics of robotic arm: Considering that the robot involved in this project is a six-degree-of-freedom robotic arm, and has both rotational and telescopic joints, such as Figure 3 As shown, unlike the traditional parametric method, the improved DH method sets the coordinate system at the joint preceding the joint link. The advantage of this method is that the parameters are clearer than those of the traditional DH method, and it can also avoid joint ambiguity.

[0039] After establishing a base coordinate system on the robotic arm base, a translation operation is performed on the base coordinate system to move it to the center position of the next joint. If the next joint is a translational joint, the coordinate system is translated again; if the next joint is a rotational joint, the coordinate system is rotated, and so on, to obtain the final coordinate systems for each joint. Figure 4 As shown.

[0040] A point in space is described by the vector ai + bj + ck. Here, i, j, and k are unit vectors on the X, Y, and Z axes, respectively, and a, b, and c are the projected lengths (mm) of the point on the X, Y, and Z axes, respectively. The homogeneous transformation matrix for translation is:

[0041] The rotation operation corresponds to the three directions of x, y, and z. A rotation transformation with an angle θ is performed on x, y, and z, respectively, yielding the following transformation matrices:

[0042] The general formula for the transfer matrix between adjacent links of a robotic arm can be written as follows:

[0043] Where, d i Let α be the offset (mm) of the i-th link, representing the vertical distance between adjacent joint axes; i Let θ be the torsion angle (°) of the i-th link, representing the angle between adjacent joint axes; i Let be the rotation angle (°) of the i-th joint, representing the rotation angle about the joint axis; a iLet be the length (mm) of the i-th link, and represent the length of the common perpendicular of the axes of adjacent joints.

[0044] The robotic arm studied in this project contains five rotary joints and one telescopic joint. The parameters of each joint link are shown in the table below during kinematic modeling.

[0045]

[0046] Substituting the link parameters, we can obtain the link transformation matrix for each joint: ,

[0047] ,

[0048] ,

[0049] Explanation of symbols in the formulas: c1=cosθ1, s1=sinθ1; c2=cosθ2, s2=sinθ2; c4=cosθ4, s4=sinθ4; c5=cosθ5, s5=sinθ5; c6=cosθ6, s6=sinθ6; and all c in subsequent formulas... n s n All follow this rule for abbreviating cosine and sine.

[0050] Wherein, matrix T i-1 ,i represents the pose of the i-th joint relative to the (i-1)-th joint, s i sinθ i c i Represents cosθ i .

[0051] The result of the forward kinematics model is the product of the transformation matrices of each joint coordinate system, i.e.:

[0052] In the formula, , , ; , , ; , , ;

[0053] The T on the left06 This represents the transformation matrix from the base coordinate system to the terminal nozzle; T on the right... 01、 T 02 ...T 56 Let represent the transformation matrices from the base coordinate system to the 1st joint, from the 1st joint to the 2nd joint, ..., from the 5th joint to the 6th joint, respectively. x ,p y ,p z ] T Indicates the position of the end suction cup, [n x ,n y ,n z ] T 、[o x ,o y ,o z ] T 、[a x ,a y ,a z ] T This indicates the direction of the suction cup and the inverse kinematics of the robotic arm.

[0054] The inverse kinematics problem of a robotic arm refers to a problem where the end-effector pose of the robotic arm is known (i.e., the homogeneous transformation matrix T is known). 06 Solve for the angles θ of each joint. i This can be understood as knowing the end effector's pose and solving for the six joint variables using kinematic equations. Since joints 2, 4, and 5 are parallel rotary joints, they are coupled together in the zoy plane and have redundant degrees of freedom. During inverse kinematics (IK), a given end effector pose typically corresponds to a combination of an infinite number of joints 2, 4, and 5. Therefore, during IK, the exact value of one of the joints 2, 4, and 5 needs to be determined. In this project, the value of joint 4 is given in advance, and the values ​​of the other five joints are solved based on the end effector pose.

[0055] For the inverse kinematics model, the first step is to assume that the end effector of the robotic arm needs to reach a certain orientation and position. At this point, the transformation matrix of the end effector coordinate system relative to the base coordinate system is known, denoted as R:

[0056] Then we have:

[0057] Since the transformation matrix of joint 1 relative to the base coordinate system is:

[0058] The transformation matrix of other joints relative to their predecessor joint can be obtained by rotation and translation operations as described in forward kinematics, similar to formula (9).

[0059] Multiply the left side of equation (8) by T 01 The inverse of the equation yields:

[0060] Multiply the right side of equation (8) by T on the left. 01 The inverse of the equation yields:

[0061] In the formula, s6 represents sinθ6, and c6 represents cosθ6; s 24 Let sin(θ2+θ4), c 24 Cos(θ2+θ4) represents the sine and cosine values ​​of the sum of the rotation angles of joints 2 and 4, and other parameters are similar.

[0062] Solve for θ1: Since the elements in the second row and fourth column of equations (10) and (11) are equal, we obtain the trigonometric equation for θ1:

[0063] The two values ​​of θ1 can be solved as follows:

[0064] Solve for θ6: Since the elements in the second row and first column and the second row and second column of equations (10) and (11) are equal, we get:

[0065] Two solutions for θ6 can be obtained:

[0066] Solve for θ2 + θ4 + θ5: Since the elements in the first row and third column and the third row and third column of equations (10) and (11) are equal, we get:

[0067] Two sets of solutions for θ2+θ4+θ5 can be obtained:

[0068] Solve for d3: Multiply T on the right side of equation (10) 56 The inverse of T is obtained. 15 (Known quantities):

[0069] Since the elements in the first row and fourth column and the third row and fourth column of equations (10) and (11) are equal, we get:

[0070] Where, x 15 z 15 T respectively 15 The elements in the first row and fourth column and the third row and fourth column represent spatial coordinate components (mm).

[0071] By completing the square and summing the equations in equation (19), we obtain:

[0072] Considering that d3 is a positive value, d3 can be solved as follows:

[0073] Solve for θ2: Substituting into equation (19), we get:

[0074] Where k1 = 948.2·c4, k2 = 948.2·s4+d3, and 948.2 is the length a5 (mm) of the connecting rod 5, which is a fixed value.

[0075] Solve for θ5: Since θ4 is given in advance, θ can be calculated according to equation (17). 245 From θ2 calculated by equation (22), we can deduce θ5: θ5 = θ 245 -θ4-θ2.

[0076] In this embodiment, the calibration and calculation unit of the six-degree-of-freedom hydraulic drilling robot system is specifically implemented, and the specific process is as follows: After starting the drilling robot system, the calibration and calculation unit first triggers a comprehensive self-check of all operating modules, including the hydraulic robotic arm, laser positioning device, vision camera, and RS485 communication link. The self-check covers core items such as hardware operating status, data transmission connectivity, and sensor acquisition accuracy. After confirming that each hardware module is fault-free and that there is no packet loss in data transmission, the calibration operations of the motion execution end, spatial positioning end, and vision inspection end are carried out simultaneously, as well as the loading of the corresponding drilling process parameters for the workpiece to be processed.

[0077] Zero-point position calibration was performed on the six-degree-of-freedom hydraulic robotic arm at the motion execution end. A base coordinate system was established with the robotic arm base as the reference, and the zero-point regression and accuracy verification of the six joints were completed. The zero point of joint α rotation angle was 0° and the zero point of joint γ telescopic arm was 0mm, thus establishing a precise basic reference for the movement of the robotic arm.

[0078] Accuracy calibration was performed on the laser positioning device at the spatial positioning end, adjusting the measurement accuracy of the laser rangefinder to ±a.mm and the angle control accuracy of the two-degree-of-freedom gimbal to ±b.°. Simultaneously, the adjustment and matching of measurement parameters such as the horizontal rotation angle and pitch angle of the gimbal were completed to ensure the accuracy of spatial positioning.

[0079] The detection parameters of the industrial camera at the vision inspection end are calibrated and the overall equipment is calibrated. The core intrinsic parameters such as focal length and principal point coordinates are obtained by Zhang's camera calibration method. The image acquisition resolution and frame rate are calibrated to ensure the accuracy of target image acquisition and feature recognition.

[0080] Simultaneously, based on the processing requirements such as the workpiece's hole diameter and hole distribution, drilling process parameters such as the robotic arm's movement speed c.° / s, drilling depth d.mm, and target positioning threshold ±e.mm are applied, enabling the integrated completion of multi-end calibration and process configuration, thereby improving the overall efficiency of work preparation.

[0081] Drilling robots primarily use laser positioning devices and vision cameras for target positioning and drilling guidance, such as... Figure 5 As shown in the diagram, the laser positioning device calculates the target's pose using three-point coordinates. After the suction cup is placed against the target's plane, a vision camera performs secondary detection and positioning of the target, adjusting the drilling machine's position to align with the target and complete the drilling operation. This positioning strategy primarily adapts to the layered optimization of drive system characteristics and operational accuracy requirements—addressing the characteristic of hydraulically driven joints being "highly load-bearing but with low control precision," it employs a two-step approach: "coarse positioning solves the 'positioning' problem, and fine positioning solves the 'precision' problem." This leverages the load advantage of the hydraulic system while compensating for its precision shortcomings through the electric drive module and vision technology, ultimately achieving the positioning goal of "centimeter-level alignment → millimeter-level precision."

[0082] Pose description and homogeneous transformation: Pose description is fundamental for motion analysis and positioning control of a robotic arm, primarily consisting of two parts: position description and attitude description. Position description refers to determining the specific location of a point using three-dimensional coordinate values, while attitude description represents determining the orientation of a rigid body using Euler angles, rotation matrices, etc. To transform position and attitude information between different coordinate systems, a series of complex transformations, such as coordinate translation and rotation, are required.

[0083] Position description is a fundamental aspect of robotic arm motion control, typically representing the specific location of a point using coordinate values ​​in a three-dimensional coordinate system. In three-dimensional space, the position of a point can be represented by three coordinate values ​​(x, y, z), which represent the projected lengths of that point along the X, Y, and Z axes, respectively. For a robotic arm, the position description of its end effector specifically refers to the position coordinates of the end effector's center point relative to a fixed reference coordinate system (such as the base coordinate system).

[0084] like Figure 6 A spatial rectangular coordinate system {A} is established as shown. A p represents an arbitrary point in space. (Position vector) A p can be represented by equation (23).

[0085]

[0086] Posture description: Attitude description is a crucial step in determining the specific orientation of a robotic arm's end effector in space. It not only determines whether the robotic arm can approach the target object at the correct angle but also directly affects the stability and success rate of the grasping operation. To achieve attitude description, a series of mathematical tools are needed to characterize the rotational state of the rigid body, with rotation matrices being the most commonly used method. By establishing a 3×3 orthogonal matrix, the rotational state of the rigid body relative to the reference coordinate system can be intuitively reflected.

[0087] like Figure 7 As shown, a fixed coordinate system {B} is established in the spatial rectangular coordinate system {A} to describe the posture of object B. The orientation of object B relative to coordinate system {A} can be expressed by formula (24).

[0088]

[0089] in, A R B Let be the rotation matrix of the object coordinate system {B} relative to the base coordinate system {A}, which has no physical dimensions. A X B , A Y B , A Z B These represent the unit column vectors of the three coordinate axes of the object coordinate system {B} relative to the base coordinate system {A}.

[0090] Homogeneous transformation matrix: To analyze both coordinate translation and rotation simultaneously, a homogeneous transformation matrix is ​​needed. A homogeneous transformation matrix is ​​a 4×4 matrix used to describe the translation and rotation relationships between coordinate systems.

[0091] Suppose we have two coordinate systems {A} and {B}, where the translation vector of coordinate system {B} relative to coordinate system {A} is t, and the rotation matrix is... A R B Then the homogeneous transformation matrix A T B As shown in formula (25).

[0092]

[0093] Laser initial positioning: The laser positioning device mainly consists of a two-axis gimbal and a laser rangefinder, enabling precise acquisition of the target's three-dimensional coordinates, such as... Figure 8 As shown. If the distance measured by the laser is r, the horizontal rotation angle of the gimbal is α, and the elevation angle is β, the spatial coordinates of the laser target point can be obtained as follows:

[0094] x, y, z are the three-dimensional spatial coordinates (mm) of the laser target point in the gimbal coordinate system.

[0095] The joints of the drilling robot are hydraulically driven, with the core advantages of high output torque and strong load-bearing capacity, enabling stable movement of heavy components such as the end effector suction cup. However, hydraulic systems have inherent limitations: the compressibility of hydraulic oil and valve response delays result in low repeatability of joint movements (typically ±5~10mm), which cannot directly meet the millimeter-level precision requirements of drilling operations. Therefore, laser positioning is used to calculate the target's spatial pose, guiding the hydraulically driven robotic arm to move the end effector suction cup quickly to the target area and adhere to the wall surface, achieving coarse positioning at the ±5cm level. This provides a stable reference position for subsequent fine positioning (preventing fine positioning failure due to excessive initial deviation).

[0096] In practice, a target is typically a single point attached to a wall. When determining the target's coordinates, the wall's normal direction also needs to be determined. In this project, the pose of the robotic arm's end effector suction cup is determined using the target and two surrounding non-collinear points, including the origin O and points A and B located in the positive Y-axis and Z-axis directions, respectively. Figure 8 As shown.

[0097] The basis vectors of the target coordinate system can be defined directly:

[0098] To ensure that the basis vectors form an orthonormal basis, the basis vectors need to be normalized and orthogonalized to obtain unit orthonormal vectors. Arranging the orthogonal basis vectors column-wise and combining them with the target coordinates, a homogeneous transformation matrix T can be formed. l That is, a complete pose description of the target:

[0099] T l The homogeneous transformation matrix of the target coordinate system relative to the laser positioning device coordinate system is dimensionless; O x O y O z The coordinates (mm) of the target origin O are given.

[0100] Specifically, the preprocessing and feature extraction process is as follows: the target image is converted into a recognizable digital image, the digital image is filtered and denoised, and then the image's grayscale sum, grayscale center, and direction features are obtained through image moment operation. The target features are extracted by combining image moments with rotation, translation, and scaling invariance properties. At the same time, the extracted features are verified by template matching, and the target feature points are output.

[0101] Specifically, the process of converting the extracted target feature points into actual spatial coordinates is as follows: based on the target feature points and combined with the pre-calibrated camera intrinsic parameters, the pixel coordinates of the target feature points are converted into coordinates in the camera coordinate system. Then, based on the extrinsic parameter calibration results of the visual detection end relative to the robot arm base coordinate system, the coordinates of the target feature points in the camera coordinate system are converted into actual spatial coordinates in the robot arm base coordinate system through a coordinate system transformation matrix.

[0102] Specifically, the process of calculating the positional deviation between the motion execution end and the target reference point is as follows: obtain the actual spatial coordinates of the drilling operation end of the motion execution end in the robot arm base coordinate system, and at the same time, based on the actual spatial coordinates of the target reference point in the robot arm base coordinate system, perform multi-axis numerical comparison calculations to calculate the positional offset of the motion execution end relative to the target reference point in the motion axis, and integrate to generate the deviation calculation result.

[0103] Specifically, the process of driving the execution end to adjust based on the deviation calculation result is as follows: generating deviation compensation parameters based on the position offset in the deviation calculation result, performing a fusion calculation between the deviation compensation parameters and the original motion parameters of the execution end to generate the compensated motion control parameters of the execution end, converting the compensated motion control parameters into standardized adjustment commands according to a preset communication protocol, driving the electric adjustment mechanism of the execution end to adjust along the motion axis, and acquiring the position and pose data of the execution end in real time during the adjustment process and comparing it with the position of the target reference point.

[0104] This embodiment is a continuation of the aforementioned pose conversion unit embodiment, and implements the core execution flow of the visual calibration unit. The specific process is as follows: Visual secondary localization: Visual inspection primarily involves converting image signals of a target captured by a camera into digital signals that can be recognized by a computer, thereby accurately determining the target's location. During visual inspection, the camera is typically affected by the working environment, lighting conditions, and the camera's own manufacturing process, all of which can influence the obtained image signal. Therefore, it is necessary to preprocess the acquired image signals beforehand and then use feature extraction to achieve target identification and localization.

[0105] Gaussian filtering: Gaussian noise is typically generated due to insufficient lighting or uneven brightness distribution during camera shooting. Prolonged camera operation leading to excessive heat buildup in internal components can also produce Gaussian noise. The noise distribution follows a Gaussian normal distribution function, and its power spectral density follows a uniform distribution. Gaussian filtering can effectively eliminate this noise; it is a type of low-pass filter. The weights of pixel values ​​within a sliding window are determined by a Gaussian distribution function. Through Gaussian convolution, the weighted average of all pixels is output. The closer to the center point, the larger the weight ratio, and the closer the gray value is to the center point of the window, thus reducing image distortion.

[0106] Two-dimensional Gaussian is the foundation of Gaussian filtering, and its expression is as follows:

[0107] x and y are the horizontal and vertical coordinates (in pixels) of the pixel in the image coordinate system; σ is the standard deviation of the Gaussian function, which determines the weight distribution of the filtering window; g(x,y) is the Gaussian weight value of the pixel.

[0108] Image feature detection: In the two-dimensional gray-level density function of an image, image moments can be used to represent the features of the image. The two-dimensional gray-level density function of an image grayscale... f(x , y) Image features of a grayscale image can be represented using image moments.

[0109] The zeroth moment represents the sum of the gray levels in an image; the zeroth moment m 00 Represented as:

[0110] m 00 f(x,y) is the zeroth moment of the image, representing the total gray level of the image, and has no physical dimension; f(x,y) is the gray value of the (x,y) pixel in the image, ranging from 0 to 255; x and y are the horizontal and vertical coordinates (in pixels) of the pixel.

[0111] First moment: The first moment m of the image 10 and m 01 This is used to determine the grayscale center of the image:

[0112] m 10 and m 01 This is the first moment of the image, used to calculate the coordinates of the grayscale center.

[0113] The centroid coordinates of the image are x. c y c :

[0114] Second moment

[0115] m 20 m 11 m 02 It represents the second moment of the image, used to calculate the image orientation features.

[0116] Therefore, the orientation of the image is:

[0117] in

[0118] θ is the principal orientation angle (°) of the image, representing the main orientation of features in the image. The nth moment of the image has 2... n -1. For moments of order three or higher, it is more convenient to use the projection of the image onto the axis or axis than to use the description of the image itself.

[0119] Hu Ju: In practice, the image captured by the camera of the object being inspected will be tilted to some extent. Therefore, it is necessary to find the rotation-invariant property of the target image for extraction and analysis. The Hu invariant moment feature meets the requirements.

[0120] Geometric moments:

[0121] m pq f(x, y) represents the (p+q)th order geometric moment of the image, where p and q are the orders of the moments, which are non-negative integers; x and y are the pixel coordinates (pixels); and f(x, y) is the gray value of the pixel.

[0122] Center distance:

[0123] μ pq y is the center distance of the image at order (p+q), and the moment with the image gray center as the origin; x and y are the coordinates (pixels) of the image gray center.

[0124] The center distance reflects the distribution of gray values ​​in an object image relative to the gray mean. By normalizing the process and representing the center distance with geometric moments, we can obtain the Hu moment.

[0125]

[0126] As can be seen from the above formula, seven different invariant moments can be obtained. When processing images, the second-order moments M1 and M2 can be used to maintain the properties of rotation, translation, and scaling invariance.

[0127] Template matching and positioning: Template matching compares the similarity between a created sample template and the target object. It involves iterating through images of the target object using the template image to find similar characteristics. The principle is as follows: Figure 9 As shown: The mathematical expression for template matching is:

[0128] In the formula, M×N is the size of the template image, I and T represent the Hu moments of the search image and the target template, respectively, and T is the Hu invariant moment of the target template; D is the matching deviation value; (i,j) are the pixel coordinates of the search image; and (m,n) are the pixel coordinates of the template image. The template T traverses the detection area from the upper left corner of the detection area I. When I(i,j)=T(m,n), the value of D is zero. At this time, there is a region in the detection area that is the same as the matching template, and the template matching is successful, that is, the coordinates P of the target in the image are detected. t =[u t ,v t ] T .

[0129] Based on the camera imaging principle, the spatial coordinates P of the target in the camera coordinate system can be obtained. c =[x c ,y c ,z c ] T have:

[0130] u t v t f represents the pixel coordinates of the target feature point in the image; u f v x represents the focal length (in pixels) of the camera in the u and v directions; u0 and v0 are the pixel coordinates of the camera's principal point; x c y c z c The coordinates (mm) of the target feature point in the camera coordinate system are given.

[0131] Sensor extrinsic parameter calibration: This explanation uses visual camera calibration as an example, defining the calibration board coordinate system as O. t The camera coordinate system is O c The base coordinate system of the robotic arm is O b The coordinate system O of the calibration plate can be measured using a vision camera. t With camera coordinate system O c pose description t T c , is a known quantity; the pose of the end effector suction cup of the robotic arm is b T eAccording to the forward kinematics solution of the robotic arm, it is a known quantity; calibration plate O t In the robot arm's base coordinate system O b The pose description below is denoted as b T t , is an unknown quantity; camera coordinate system O c In the coordinate system O of the suction cup at the end of the robotic arm e The pose description below is denoted as e T c , is the quantity to be determined; such as Figure 10 As shown, the transformation matrices have the following relationships:

[0132] t T c , b T e , b T t , e T c These are all homogeneous transformation matrices, representing the pose transformation relationship between the corresponding coordinate systems.

[0133] The approach to sensor extrinsic parameter calibration involves controlling the robotic arm to move to different positions, observing the pose changes of the calibration plate in space, and then deriving the extrinsic parameters. b T t Relationship with multiple observations. Multiple sets of values ​​can be obtained as the robotic arm moves to different poses in space. t T i c and b T i e This allows for the establishment of multiple sets of matrix transformation relationships:

[0134] i and j represent the number of movements of the robotic arm, i≠j; t T i c , t T j c Let be the pose matrix of the calibration board coordinate system relative to the camera coordinate system during the i-th and j-th movements; b T i e , b T j e Let be the pose matrix of the robotic arm's end effector relative to the base coordinate system during the i-th and j-th movements, defined as follows:

[0135] b T ij Let be the pose matrix of the robotic arm during the j-th motion relative to the i-th motion; c T i j Let be the pose matrix of the camera during the j-th motion relative to the i-th motion.

[0136] Equation (42) can be simplified to:

[0137] A= b T i j B= c T i j X= b T t All are homogeneous transformation matrices. Where A and B are known quantities, and X is the quantity to be determined, both of which are homogeneous transformation matrices, each consisting of a 3×3 rotation matrix and a 3×1 translation matrix:

[0138] R is a 3×3 rotation matrix describing the attitude relationship between the coordinate systems; t is a 3×1 translation vector describing the positional relationship (mm) between the coordinate systems; O is a 3×1 zero vector, and 0 is a scalar 0.

[0139] Equation (45) can be expanded as follows:

[0140] R is the rotation matrix; t is the translation vector.

[0141] Rotation matrices have the following properties: R is an identity orthogonal matrix; The R matrix defines the motion of any point in space, which can be viewed as a rotation of an angle θ about a certain axis k, denoted as Rot(k, θ). Eigenvalue decomposition of R yields: R = V·diag(1,e) -jθ ,e jθ )·V -1 V is the characteristic matrix, with eigenvalues ​​1, e, and e respectively. -jθ ,e jθ The eigenvector corresponding to eigenvalue 1 is k.

[0142] Based on the above properties of the rotation matrix, R can be obtained from equation (46). bij =R bt R cij ·R bt 1 It can be known that R bij With R cijThese are similar matrices, meaning they have the same eigenvalues. Since the eigenvalues ​​of a rotation matrix are uniquely determined by the rotation angle, R... bij With R cij The corresponding rotation angles are equal, denoted as θ. Equation (46) can be written as:

[0143] Where k b k c R respectively bij With R cij The axis of rotation. Equation (43) can be written as:

[0144] From the above properties of rotation matrices, we have:

[0145] I is a 3×3 identity matrix; R bij for b T ij The rotation matrix.

[0146] It can be known that R bij -I has one eigenvalue that is 0. When θ is not equal to 0, R bij The rank of -I is 2.

[0147] If the robotic arm moves to three different positions during the calibration process, the following four relationships can be obtained:

[0148] From equation (50), we can see that R be satisfy:

[0149] Where, k b12 k c12 R respectively b12 R c12 The unit vector on the axis of rotation, k b23 k c23 Similar. Due to R bt At the same time, k c12 k c23 Switched to k b12 k b23 And will inevitably k c12 ×k c23 Switched to k b12 ×k b23 Right now:

[0150] In k c12 With k c23When they are not parallel, the matrix in the above formula is a full-rank matrix, that is:

[0151] Substituting the solution from the above equation into equation (50), we obtain the following about t b t The linear equation can be solved to obtain the calibration plate O. t In the robot arm's base coordinate system O b The pose description below is denoted as b T t , t bt for b T t The translation vector (mm) in the figure represents the positional relationship between the origin of the calibration plate coordinate system and the origin of the robot arm base coordinate system.

[0152] Substituting the obtained result into any set of moving points in equation (42), we obtain the camera O. c In the coordinate system O of the suction cup at the end of the robotic arm e The pose description below is denoted as e T c The target coordinates P identified by the camera c =[x c ,y c ,z c ] T Coordinates P transformed into the robot arm's base coordinate system b =[x b ,y b ,z b ] T There is [x] b ,y b ,z b ,1] T = b T e e T c [x c ,y c ,z c ,1] T The target pose identified by laser is similar. After being converted to the coordinate system of the robotic arm, the joint angles can be calculated through the inverse kinematics of the robotic arm, thereby controlling the robotic arm to move to the target position.

[0153] Specifically, the process of triggering an alarm and suspending the operation in a timely manner when an abnormality is detected is as follows: continuously acquire the operation status data of the operation module, the pose data of the motion execution end, and the contact pressure data throughout the entire drilling operation process, compare them with the preset normal operation threshold range in real time, and based on the judgment result, trigger the system alarm mechanism and generate an operation suspension control command to stop the operation of the operation module.

[0154] In this embodiment, the implementation is carried out for the operation control unit, and the specific process is as follows: Real-time monitoring of the entire drilling process: After receiving the "precision positioning successful" command from the host computer, the operation control unit initiates the drilling operation process. Simultaneously, it continuously collects three types of core data at a refresh rate of FHz through the host computer communication module and compares them in real time with the preset normal operation threshold range to achieve full-process, blind-spot-free status monitoring. Operational status data of the work module: collects hardware operation status codes, communication link on / off status, motor / drilling rig speed, hydraulic system pressure and other data from PLC, laser positioning device, vision camera, end drill, hydraulic drive system, electric adjustment mechanism, etc. Motion execution end pose data: Collect the actual rotation angle of each joint of the robotic arm, the length of the telescopic arm, and the real-time X / Y / Z axis spatial coordinates of the end drill in the robotic arm base coordinate system to reflect the pose stability of the execution end during the drilling process; Adhesion pressure data: The real-time contact pressure value between the suction cup and the target plane is collected by the suction cup pressure sensor to ensure that the suction cup adheres stably during the drilling process without loosening or excessive compression.

[0155] All collected data is transmitted to the host computer's operation control module in real time. Each data item is compared against the preset normal operation threshold range to determine whether the data is within the compliance range. If all data meet the threshold requirements, the operation status is determined to be normal, and the drilling operation continues. If any data exceeds the threshold range, the abnormal handling process is immediately triggered.

[0156] Operational anomaly detection and alarm suspension: When the control unit detects that any core data exceeds the normal operating threshold, it immediately initiates the abnormal alarm + operation suspension linkage process to quickly terminate the operation and avoid workpiece scrapping and equipment damage. The specific process is as follows: Anomaly type determination: Based on the type of data exceeding the threshold, the anomaly category is automatically determined, including hardware operation anomalies (such as communication interruption, drilling speed of 0), execution end pose deviation anomalies (such as end drill pose deviation > ±amm during drilling), and bonding pressure anomalies (such as pressure < low or > high), and the specific anomaly data and category are marked on the host computer interface. System alarm mechanism triggered: Simultaneous activation of audible and visual alarms; a red anomaly notification window pops up on the host computer interface, displaying the anomaly type, occurrence time, and details of data exceeding the threshold; the on-site audible and visual alarms emit a continuous alarm sound and flash red light, reminding staff to handle the situation promptly. Operation pause control command generation and issuance: The control unit immediately generates a standardized operation pause control command, encapsulates it according to the Modbus RTU communication protocol, and sends it to the PLC via RS485 serial port. After receiving the command, the PLC immediately issues a stop command to all operation modules to stop the end drill rotation, lock all joints of the robotic arm, and shut down the hydraulic / electric drive system to achieve emergency shutdown of the operation modules. Abnormal data retention: Automatically retain all collected data from the time of the anomaly to the pause, including pose, pressure, hardware status, etc., and store them in the host computer database to provide data support for subsequent anomaly cause investigation and process optimization.

[0157] After the staff investigates and resolves the abnormality, they confirm "abnormality eliminated" on the upper-level machine interface. The control unit clears the alarm status, generates a work recovery command, and drives each module to reset to the work state before the abnormality, so that the drilling operation can continue.

[0158] Drilling operation completed and execution end reset: When the terminal drilling rig completes the drilling operation according to the preset process parameters (reaching the preset drilling depth and holding time), the terminal drilling rig sensor sends a "drilling completed" signal to the control unit, and the control unit initiates the execution end reset process: Generate standardized reset control instructions and send the reset parameters of each joint / mechanism to the PLC according to the timing logic of "end drill retraction → electric adjustment mechanism return to position → hydraulic robotic arm joints reset", and match the corresponding register addresses. The PLC-driven end-of-line drilling rig first retracts along the Z-axis to a safe distance, then controls the electric adjustment mechanism to return to the initial position, and finally drives each rotary and telescopic joint of the hydraulic robotic arm to move sequentially to the factory-preset zero-point initial position, completing the reset of the entire actuator. During the reset process, the control unit collects the position data of the execution end in real time to ensure that the deviation between the reset position and the initial zero point is less than or equal to the preset threshold. After the reset is completed, the host computer interface displays "Execution end reset successful" and waits for the drilling operation instruction of the next workpiece.

[0159] Full-process data recording and traceability of operations: The operation control unit automatically, comprehensively, and traceably records operation data throughout the entire process of drilling operations, from start to finish. The recorded data is classified and stored according to the workpiece number, forming a complete data archive for drilling operations of a single workpiece.

[0160] All recorded data is stored in the host computer's local database and supports cloud-based synchronous backup. Staff can perform multi-dimensional queries and exports using keywords such as workpiece number, operation time, and equipment number, enabling data traceability of the entire drilling operation process and providing data support for production process optimization, equipment maintenance, and quality control.

[0161] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A hydraulic drilling robot system based on laser and vision fusion, characterized in that, include: Calibration and measurement unit, pose conversion unit, vision calibration unit, and operation control unit; The calibration and calculation unit performs self-checks on the operation module, simultaneously performs multi-end calibration, obtains drilling process parameters, and calculates the target spatial pose through a three-point coordinate calculation method. The pose conversion unit converts the target spatial pose to the robot arm base coordinate system, calculates the corresponding motion execution parameters through kinematic modeling, generates motion control commands, drives the execution end to move to the target area, and outputs the target area's fitting and positioning status. The visual calibration unit acquires a target image, performs preprocessing and feature extraction, converts the extracted target feature points into actual spatial coordinates, calculates the positional deviation between the motion execution end and the target reference point, and drives the execution end to make adjustments based on the deviation calculation results. After the operation is completed, the operation control unit drives the execution end to reset to the initial position, records the entire process data of the operation, monitors the status of the drilling operation in real time, and triggers an alarm and suspends the operation in a timely manner when an operation abnormality is detected.

2. The system according to claim 1, characterized in that, The specific process of multi-end calibration is as follows: zero-point position calibration is performed on the motion execution end, while accuracy calibration and measurement parameter adjustment and matching are performed on the spatial positioning end, and detection parameter calibration and equipment calibration are performed on the vision inspection end. The calibration operations of the motion execution end, spatial positioning end, and vision inspection end are performed synchronously with the loading of the drilling process parameters corresponding to the workpiece to be processed.

3. The system according to claim 1, characterized in that, The specific process of calculating the target spatial pose using the three-point coordinate measurement method is as follows: align the target origin and two non-collinear feature points around the target to obtain the three-dimensional spatial coordinates. Using the target origin as a reference, obtain the normal vector of the plane where the target is located. Define the basis vectors of the target coordinate system accordingly. Normalize and orthogonalize the basis vectors in sequence to obtain the unit orthogonal vectors of the target coordinate system. Arrange the unit orthogonal vectors in columns and combine them with the three-dimensional spatial coordinates of the target origin to form a homogeneous transformation matrix.

4. The system according to claim 1, characterized in that, The specific process of converting the target spatial pose to the robot arm base coordinate system is as follows: calibrating the extrinsic parameters of the spatial positioning end and the visual detection end relative to the robot arm base coordinate system, obtaining the transformation relationship between the detection end coordinate system and the robot arm base coordinate system, performing coordinate system transformation operation on the homogeneous transformation matrix corresponding to the target spatial pose, converting the target pose information with the spatial positioning end coordinate system as a reference into pose data with the robot arm base coordinate system as a reference, and generating the target pose in the robot arm base coordinate system.

5. The system according to claim 1, characterized in that, The specific process of calculating the corresponding motion execution parameters through kinematic modeling is as follows: A kinematic model including forward and inverse directions is constructed based on the structural characteristics of the motion execution end. A transformation matrix between joints is established based on the improved link parameter method. The forward kinematic model is obtained through the product of the transformation matrices. Then, using the target pose in the robot arm's base coordinate system as known conditions, the motion parameters of redundant joints are preset based on the inverse kinematic model. The rotation angle parameters of the rotary joints and the extension / retraction parameters of the telescopic joints are calculated and generated. Simultaneously, the rationality is verified by combining the preset motion range of the joints at the motion execution end, thus generating the motion execution parameters.

6. The system according to claim 1, characterized in that, The specific process of generating motion control commands is as follows: the motion execution parameters are classified and organized according to the control dimension of the joints at the motion execution end, the control register address and data format corresponding to the joints are matched, and the classified motion execution parameters are converted into standardized control command data according to the preset communication protocol rules. At the same time, the timing logic and triggering identifier of motion execution are added to the control commands to generate the motion control commands.

7. The system according to claim 1, characterized in that, The specific process of outputting the bonding and positioning status of the target area is as follows: real-time acquisition of contact pressure data between the bonding component and the plane where the target is located, comparison with a preset bonding pressure threshold, and output of the bonding and positioning status of the target area based on the determination result.

8. The system according to claim 1, characterized in that, The specific process of preprocessing and feature extraction is as follows: the target image is converted into a recognizable digital image, the digital image is filtered and denoised, and then the total gray level, gray level center and direction features of the image are obtained through image moment operation. The target features are extracted by combining image moments with rotation, translation and scaling invariance properties. At the same time, the extracted features are verified by template matching, and the target feature points are output.

9. The system according to claim 1, characterized in that, The specific process of converting the extracted target feature points into actual spatial coordinates is as follows: based on the target feature points and combined with the pre-calibrated camera intrinsic parameters, the pixel coordinates of the target feature points are converted into coordinates in the camera coordinate system. Then, based on the extrinsic parameter calibration results of the visual detection end relative to the robot arm base coordinate system, the coordinates of the target feature points in the camera coordinate system are converted into actual spatial coordinates in the robot arm base coordinate system through the coordinate system transformation matrix.

10. The system according to claim 1, characterized in that, The specific process for calculating the positional deviation between the motion execution end and the target reference point is as follows: obtain the actual spatial coordinates of the drilling end of the motion execution end in the robot arm base coordinate system, and at the same time, based on the actual spatial coordinates of the target reference point in the robot arm base coordinate system, perform multi-axis numerical comparison calculations to calculate the positional offset of the motion execution end relative to the target reference point in the motion axis, and integrate to generate the deviation calculation result.

11. The system according to claim 1, characterized in that, The specific process of driving the actuator to adjust based on the deviation calculation result is as follows: a deviation compensation parameter is generated based on the position offset in the deviation calculation result; the deviation compensation parameter is fused with the original motion parameter of the actuator to generate the compensated motion control parameter of the actuator; the compensated motion control parameter is converted into a standardized adjustment command according to a preset communication protocol; the electric adjustment mechanism of the actuator is driven to adjust along the motion axis; during the adjustment process, the position and pose data of the actuator are acquired in real time and compared with the position of the target reference point.

12. The system according to claim 1, characterized in that, The specific process of triggering an alarm and suspending the operation in a timely manner when an abnormality is detected is as follows: continuously acquire the operation status data of the operation module, the pose data of the motion execution end and the contact pressure data throughout the entire drilling operation process, compare them with the preset normal operation threshold range in real time, and based on the judgment result, trigger the system alarm mechanism and generate an operation suspension control command to stop the operation of the operation module.

Images