Water jet cutter robot autonomous operation three-dimensional detection and positioning calibration method and system

Abstract

The invention relates to the technical field of positioning calibration, and provides a water jet scalpel robot autonomous operation three-dimensional detection and positioning calibration method, which comprises the following steps: S1, manually controlling the tail end of a mechanical arm to carry out coarse scanning, and searching a calibration target; s2, fine scanning path planning of the line laser scanner is carried out, and the scanning directions X and Y of the line laser scanner are determined; s3, multiple times of scanning are carried out in the X direction and the Y direction, and multiple pieces of point cloud data are obtained through scanning; s4, recording and processing the point cloud data, and reserving the point cloud data of the calibration target; s5, performing error feature estimation and calibration on the reserved point cloud data of the calibration target; s6, manually controlling the calibration of the tail end of the water jet cutter, and correcting the space offset of the tail end of the cutter head of the water jet cutter; s7, filtering the point cloud data obtained during normal operation to obtain global error compensation; and S8, performing dynamic positioning correction on the operation target. The method is used for scanning positioning error compensation and correction of a hand-eye system of the water jet scalpel robot and dynamic error correction in the operation process.

Description

Technical Field

[0001] The present invention relates to the technical field of positioning and calibration, and in particular to a method and system for autonomous three-dimensional detection and positioning and calibration of a water jet robot. Background Art

[0002] At present, mobile bomb disposal robots are widely valued for their remote, safe and flexible features. Bomb disposal robots can replace bomb disposal personnel to enter dangerous areas to conduct reconnaissance, transfer, dismantle and destroy explosive devices or weapons. Due to the particularity of the objects and environments of bomb disposal operations, water jet cutting (also known as water jet) is a cold cutting method. It does not produce thermal deformation or thermal effects. Whether it is high-hardness materials such as glass, ceramics, metals, etc., or soft materials such as leather, rubber, cloth, etc., it has good cutting effects. Therefore, it has a good application prospect for operations on target objects such as shelters, containers, and explosives.

[0003] Installing a water jet on a robot, especially a multi-axis robotic arm robot, is one of the main ways to achieve flexible operation. The target detection and operation positioning modes during the operation process are mainly: (1) The target object is placed on a fixed tooling, and the water jet directly operates according to the preset trajectory, refer to patent CN112809834A "A high-pressure water jet cutting device for papermaking"; (2) An external fixed visual camera or line laser scanning detection is used to locate the target, and the water jet operates according to the pre-calibrated tooling positioning and target detection position; refer to patent TW202236984A "Intelligent automatic cutting and sorting system"; (3) A visual camera and (or) line laser scanning are installed at the end of the robotic arm to detect and locate the target, that is, a hand-eye integrated water jet robot. There is no search that has used this solution for high-precision multi-axis robotic arm water jet cutting, but there are other aspects, such as patent CN118386236A "Unmanned welding and grinding method of a teaching-free robot based on the combination of line laser scanning and stereoscopic vision".

[0004] As a non-contact measurement technology, line laser sensors can quickly obtain the contour information of the surface of the measured object. They have the characteristics of high efficiency, strong anti-interference ability, and high precision. They are currently widely used in various engineering fields. However, the measurement range of line laser sensors is small, and they can only measure the two-dimensional information at the laser line at a time. Relying on automated equipment with fixed tracks, they can only obtain local surface data of the measured object. By mounting line laser sensors on a flexible and high-precision articulated robot platform, the overall measurement of the measured object can be achieved, which improves the flexibility of the measurement system and has broad prospects in the fields of three-dimensional detection, target positioning, reverse engineering, etc.

[0005] In order to obtain the three-dimensional data of the object under test in the robot base coordinate system, it is necessary to calibrate the relationship between the robot terminal coordinate system and the line laser sensor coordinate system, that is, hand-eye calibration. At present, the hand-eye calibration methods for line laser scanning robots mainly include the following methods: fixed-point variable pose method based on calibration sphere. This method has requirements for the intersection position of the line laser and the calibration sphere. Since the radius of the intersection circle at different positions of the standard sphere is different, the obtained data is limited, which affects the accuracy of the circle fitting and makes the calibration operation difficult; laser plane calibration method. This method increases the complexity of calibration and is unnecessary for line laser sensors that have been pre-calibrated with internal references before leaving the factory; the method of hand-eye calibration of line laser contour sensors and robots with the help of high-precision external equipment, such as three-coordinate measuring instruments and laser trackers, has the disadvantages of high cost and inconvenient operation. In view of the above problems, a hand-eye calibration method with simple operation, strong applicability, high accuracy and high solution success rate is needed.

[0006] For this type of hand-eye integrated line laser calibration, relevant patents have proposed some solutions, such as patent CN118386236A "Autonomous welding and grinding method of a teaching-free robot based on the combination of line laser scanning and stereo vision", which discloses a method of combining stereo vision with line structured light scanning vision to solve the dual requirements of range and accuracy that cannot be taken into account in the existing technology. Based on the point cloud collected by the stereo camera, the posture of the welding or grinding part on the workpiece is solved, and the posture of the robot end is planned according to the point cloud position information and the eye-out-of-hand calibration to complete large-scale positioning: the teaching-free robot calculates the welding or grinding posture under large-scale positioning, and plans the path to perform targeted high-precision scanning on the identified position; extracts the sub-pixel position of the center of the light strip from the data obtained by high-precision scanning, extracts the characteristic points of the welding or grinding position, and obtains a high-precision path; the teaching-free robot welds or grinds the weld along the path under the obtained high-precision path.

[0007] Patent CN115078384A "A rapid detection device for pits and cracks on the surface of stone slabs" discloses a method of combining a three-dimensional line laser scanner with an image to perform a rapid detection device for pits and cracks on the surface of stone slabs. The stone slabs are transported by a conveying mechanism, and the image acquisition mechanism acquires images of the stone slabs; the three-dimensional line laser scanner is used to acquire three-dimensional point cloud data of pits and cracks for preprocessing and three-dimensional reconstruction to obtain the three-dimensional morphology of the pits and cracks; the robotic arm drives the three-dimensional line laser scanner to move; the main control device is connected to the conveying mechanism, the image acquisition mechanism, the three-dimensional line laser scanner and the robotic arm, and is used to process the acquired images to obtain the colors around the pits and cracks and their positions and shapes on the stone slabs, and control the robotic arm to drive the three-dimensional line laser scanner to move according to the position to obtain the three-dimensional morphology of the pits and cracks for minimum bounding box calculation, and volume estimation of the pits and cracks.

[0008] Patent CN108444383A "Integrated measurement method for machining process based on visual laser combination" uses left and right cameras to achieve online guidance of the position of the end measurement device of the robot arm through visual servo, so that the local features of the parts to be measured are within the measurement range of the laser scanner, and the local features are measured with high precision by using the laser scanner, and the end measurement device is tracked and positioned by the laser tracker; the position of the end measurement device is adjusted to measure the parts to be measured in different areas multiple times, and the local laser scanning data of multiple measurements are unified into the global coordinate system established based on the laser tracker. This method effectively solves the problem of integrated measurement of high-precision machining process in a large spatial range, and has the characteristics of high measurement accuracy, wide range and high robustness, and realizes large-space, cross-scale, and high-precision integrated measurement of machining process.

[0009] Patent CN115289999A "A self-calibration method for a three-dimensional laser scanning system" is aimed at the problem that when the three-dimensional laser scanning device is just built, there are installation errors in the rotating structure and the scanning probe part, the scanning accuracy is low, and there is a lack of unified calibration standards and specifications. The conventional standard template method requires the use of additional calibrated measuring instruments, such as total stations, and the process is relatively cumbersome; using a block with a known side length as a calibration object, in the point cloud obtained by scanning the block, the relationship between the measured distance between the block vertices and the parameters to be calibrated is established, and a set of suitable calibration parameters is found through an optimization algorithm so that the measured distance between the block vertices is closest to the actual side length of the block. Since no additional measuring instruments are used, the measurement errors of other measuring instruments themselves and the cumulative errors caused by target processing and installation are avoided, which improves calibration efficiency and scanning accuracy.

[0010] Patent CN108151667A "A global calibration method for a multi-line laser profile scanning sensor" uses a calibration method based on a three-dimensional target to achieve a one-time rapid global calibration of spatially distributed line scanning sensors without a common field of view. There is no need to place the target multiple times. Compared with traditional one-dimensional and two-dimensional targets, the calibration efficiency is increased by 60% to 70%. In addition, the calibration algorithm is independent of the position and dimensional tolerance of the target, but only related to the flatness of the target's characteristic surface, which greatly reduces the processing difficulty of the three-dimensional target and reduces the cost of the three-dimensional target while ensuring calibration efficiency and high precision.

[0011] Patent CN113681559A "A Linear Laser Scanning Robot Hand-Eye Calibration Method Based on Standard Cylinder" carries a line laser sensor at the end of the robot and places a standard cylinder in the robot workspace; by changing the end position of the scanning robot, the cylinder side scanning data of the line laser sensor under different robot positions is obtained; since the laser plane intersects with the cylinder side to obtain an elliptical cross section, the coordinates of the center point of the elliptical cross section are obtained using the RANSAC algorithm; the distance from the center point of the elliptical contour to the central axis of the cylinder is used to establish a constrained optimization equation; the fusion algorithm of the particle swarm algorithm and the generalized multiplier method is used to solve the constrained optimization problem, and the rotation and translation matrix of the hand-eye calibration is obtained at the same time. This invention only requires a standard cylinder, the calibration process is simple and easy to operate, the calibration result is not affected by the position of the calibration object and the initial value of the calibration parameter, the calibration accuracy is high, and the versatility is strong.

[0012] However, in order to solve the problem that the current robot hand-eye system has difficulty in balancing the dual requirements of range and accuracy, the above-mentioned retrieved public related patents and literature materials adopt a combination of multiple image vision and line structured light scanning vision, and use external high-precision positioning instruments or invented square, cylindrical, three-dimensional and other calibration targets to perform conventional hand-eye calibration, which has achieved certain results, but there are still some shortcomings.

[0013] First, the repeatability of the positioning accuracy of the end of the multi-axis robot arm, as well as the influence of the path speed and position of the end of the robot arm, are not fully considered. Line laser high-precision scanning is usually triggered by a high-stability encoder on a single linear track, and the synthetic posture of the end of the multi-axis robot arm is difficult to provide the accuracy of the linear guide rail. Therefore, the error and jitter of the point cloud obtained by line laser scanning are far greater than its own accuracy. It is difficult to give full play to the high-precision advantages of line laser scanning in the hand-eye integrated operation system; Second, due to systematic factors, the hand-eye calibration error of hand-eye integrated robots that do not use external high-precision calibration facilities is much greater than the theoretical error. With the help of specially designed external targets, calibration can only be performed for a specific scenario and application, and cannot be applied to fully autonomous multi-axis manipulator water jet robots. Therefore, it is necessary to design new calibration targets and calibration methods; Third, the current robot hand-eye calibration that combines image vision and line laser scanning does not take into account the dynamic characteristics of the end operating mechanism. General robot programming design also only gives a fixed static end load mass, so it is impossible to compensate for the positioning error interference caused by the reaction force of the water jet operation.

[0014] The above-mentioned shortcomings and deficiencies make it impossible for the existing technology to effectively solve the high-precision and stable operation of the hand-eye integrated autonomous water jet robot. Summary of the invention

[0015] In view of the above problems, the purpose of the present invention is to provide a method and system for the autonomous three-dimensional detection and positioning calibration of a water jet robot, which is used for the water jet robot to perform scanning positioning error compensation and correction of the hand-eye system and dynamic error correction during operation. The present invention uses a square plane target to calibrate the error of the scanning path direction of the line laser scanner at the end of the multi-axis manipulator, and performs global verification and compensation of the end water jet through manual positioning, as well as compensation for the reaction force of the water jet during operation, so as to compensate for the problems of hand-eye calibration error, dynamic scanning and operation error, and improve the accuracy and precision of the operation of the fully autonomous water jet robot.

[0016] The above-mentioned object of the present invention is achieved through the following technical solutions: A method for autonomous three-dimensional detection and positioning calibration of a water jet robot comprises the following steps: S1: manually controlling the end of the robotic arm to perform a rough scan to find a calibration target, so that the position and distance of the calibration target are within the effective working range of the line laser scanner; S2: Perform precise scanning path planning of the line laser scanner to determine scanning directions X and Y of the line laser scanner; S3: the line laser scanner performs multiple scans along the X and Y directions respectively, and obtains multiple point cloud data of the calibration target; S4: Record and process the point cloud data, remove redundant point cloud data, and retain the point cloud data of the calibration target; S5: performing error feature estimation and calibration on the retained point cloud data of the calibration target; S6: Manually control the water jet end calibration to correct the spatial offset of the water jet cutter head end; S7: filtering the point cloud data acquired during normal operation to obtain global error compensation; S8: Perform dynamic positioning correction on the operation target.

[0017] Further, in step S1, the end of the robotic arm is manually controlled to perform the rough scan to find the calibration target, so that the position and distance of the calibration target are within the effective working range of the line laser scanner, specifically: Place the calibration target in the working area with the plane normal of the calibration target facing the end of the robot. Manually control the movement of the robot end so that the visual camera faces the calibration target and the calibration target is in the middle area of ​​the field of view of the visual camera. Confirm that the position and distance of the calibration target are within the effective working range of the line laser scanner.

[0018] Furthermore, in step S2, the precise scanning path planning of the line laser scanner is performed to determine the scanning directions X and Y of the line laser scanner, specifically: Taking the robot end as the coordinate reference, it is determined that the line laser scanner can completely cover the working area of ​​the calibration target. The robot end drives the line laser scanner to move in a plane parallel to the calibration target, and the orthogonal scanning directions X and Y are determined according to the principle of the calibration target from left to right and from front to back.

[0019] Further, in step S3, the line laser scanner performs multiple scans along the X and Y directions respectively, and scans to obtain a plurality of point cloud data of the calibration target, specifically: Control the end of the robot to move forward and backward multiple times in the X and Y directions respectively, drive the line laser scanner to scan the calibration target, and control the scanning speed Vx or Vy of the line laser scanner within the stable range of the moving line speed of the multi-axis robot; At the same time, the spatial coordinates of the robot end relative to the robot body are recorded. If the continuous spatial distance change of the spatial coordinates reaches the stepping accuracy of the line laser scanner in the scanning direction, the spatial coordinates are used as a scanning signal. Otherwise, the moving speed value of the robot end is used as the basis for the stepping calculation of the line laser scanner. The line laser scanner scans and obtains a plurality of point cloud data of the calibration target.

[0020] Further, in step S4, the point cloud data is recorded and processed, redundant point cloud data is eliminated, and the point cloud data of the calibration target is retained, specifically: Calculate the coordinate transformation matrix of the line laser scanner relative to the robot body according to the physical model of the robot end and the assembly model of the line laser scanner; The point cloud scanned by the line laser scanner is converted to the robot coordinate system, redundant point cloud data is eliminated, and only the point cloud data of the calibration target is retained.

[0021] Further, in step S5, the error characteristic estimation and calibration are performed on the retained point cloud data of the calibration target, specifically: Calculate the mean values ​​Ey and Ex and the variances Dy and Dx of the point cloud pixel distances of the point cloud data in the linear laser direction Y and the scanning direction X for each scan; Fitting the point cloud plane of the calibration target for each scan of the point cloud data, and calculating the mean Ez and variance Dz of the point cloud pixel distance in the depth direction z; The mean is used to perform overall scaling calibration on the point cloud data obtained by normal scanning, and the variance is used to compensate for the spatial offset of the local fitting plane of the point cloud data obtained by normal scanning and each point cloud pixel relative to the local fitting plane.

[0022] Further, in step S6, the water jet end is manually calibrated to correct the spatial offset of the water jet cutter head end, specifically: Calculate the first spatial coordinates corresponding to the four target positioning holes in the point cloud data scanned multiple times, and take the average; Manually control the end of the water jet cutter head to be positioned at the four target positioning holes respectively, and record the second spatial coordinates of the end of the water jet cutter head in the current robot system; Compare the coordinate values ​​of the first spatial coordinate and the second spatial coordinate, and take the components in the x, y, and z directions to perform the spatial offset correction of the end of the water jet cutter head.

[0023] Furthermore, in step S7, the point cloud data acquired during normal operation is filtered to obtain global error compensation, specifically: Placing the calibration target in different positions in the working space, and repeating steps S1-S5 to obtain multiple sets of measurement data and calibration data of the point cloud data; At the same time, statistical data of multiple scans are obtained, the statistical data including: the moving speed of the robot end and the moving direction of the spatial scanning trajectory, the mean and variance obtained in step S5, and the spatial offset obtained in step S6; The transformation matrix after the robot hand-eye calibration is distorted, stretched, and translated so that the obtained point cloud data completely matches the actual physical size and flatness of the calibration target, thereby compensating for the positioning error of the detection in the calibration space area; When the water jet robot is operating normally, the line laser scanner scans the work target on any curved surface, and the compensation method is: the point cloud data of the scanned work target is divided into a number of point cloud pieces of uniform size in the projection direction of the scanning direction, and point-by-point calibration is performed according to the calibration offset obtained in steps S5, S6, and S7, and then the overall point cloud data is filtered to obtain global error compensation.

[0024] Furthermore, in step S8, dynamic positioning correction is performed on the operation target, specifically: After the robot completes the scanning calibration, the calibration target is removed, and the end of the robot points the end of the water jet cutter head toward the working target direction, and the current posture data of the multi-axis robot is recorded; Turn on the water jet high-pressure water pump and record the current water jet nozzle diameter, water pressure, sand ratio, and the current position data of the multi-axis robot; Record and compare the spatial coordinates of the robot end before and after the water pump is turned on, and record the displacement deviation under the influence of the reaction force; By changing different parameters including the water jet nozzle diameter, water pressure, sand ratio, and the orientation of the water jet cutter head, multiple groups of displacement deviations are recorded, and the fitting mean is statistically calculated as the compensation value for the positioning of the end of the water jet cutter head when the robot is operating.

[0025] A water jet robot autonomous operation three-dimensional detection and positioning calibration system for executing the water jet robot autonomous operation three-dimensional detection and positioning calibration method as described above, comprising: A manual rough scanning module, used to manually control the end of the robotic arm to perform a rough scan and find a calibration target so that the position and distance of the calibration target are within the effective working range of the line laser scanner; A precise scanning path planning module, used for performing precise scanning path planning of the line laser scanner and determining scanning directions X and Y of the line laser scanner; A point cloud data acquisition module, used for the line laser scanner to perform multiple scans along the X and Y directions respectively, to obtain multiple point cloud data of the calibration target; A point cloud data processing module, used to record and process the point cloud data, remove redundant point cloud data, and retain the point cloud data of the calibration target; An error characteristic estimation module, used for performing error characteristic estimation and calibration on the retained point cloud data of the calibration target; The water jet end calibration module is used to manually control the water jet end calibration and correct the spatial offset of the water jet cutter head end; A global error compensation module, used for filtering the point cloud data acquired during normal operation to obtain global error compensation; The operation target calibration module is used to perform dynamic positioning correction on the operation target.

[0026] Compared with the prior art, the present invention has at least one of the following beneficial effects: (1) Line laser scanning usually uses linear guides or linear movement of the target for three-dimensional scanning. A multi-axis robot is used as a carrier for the line laser scanner. Whether it is linear trajectory speed movement or step position triggering, it has a great impact on the line laser scanning error. The present invention uses a square high-flatness target to perform point cloud error correction on the line laser scanner at the end of the hand-eye integrated multi-axis robot. No external positioning calibration instrument or system is required. It has the characteristics of fast and high precision in the working space, can give full play to the high-precision advantages of the line laser scanner, and can adapt to the spatial arbitrary trajectory operation of different types of robots with 2 axes, 3 axes and above.

[0027] (2) A multi-axis robot based on hand-eye integration usually has a definite spatial position with the target in contact operations, which can ensure its positioning accuracy; in non-contact operations with the target, its end load is usually fixed, with a definite weight and inertia. The multi-axis water jet robot with hand-eye integration provided by the present invention has a non-contact operation, but during the operation, the robot has a reaction force in the opposite direction of the water jet. The present invention adopts a method of axial multi-dimensional force sensing of the end water jet, and performs correlation compensation with water jet working parameters, such as water pressure, nozzle diameter, sand particle diameter and other parameters, which can improve the dynamic positioning accuracy in water jet operations.

[0028] (3) In view of the actual characteristics of water jet operations, in addition to the visual and point cloud correction process, the present invention sets positioning points on a square high-flatness target and uses manual positioning to perform error correction on the end water jet of the multi-axis robot. This can compensate for the position error caused by the assembly or working vibration of the end water jet to a certain extent. It does not rely on external correction instruments or means, and combined with the line laser scanning correction of the present invention, it has the advantages of fast and high precision. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 This is a schematic diagram of the scanning calibration of the water jet robot of the present invention; Figure 2 This is an overall flow chart of the water jet robot autonomous operation three-dimensional detection and positioning calibration method of the present invention; Figure 3 This is a schematic diagram of the point cloud error of the present invention; Figure 4 This is a schematic diagram of the calibration of the end of the manually controlled water jet according to the present invention; Figure 5 This is the overall structure diagram of the water jet robot's autonomous three-dimensional detection and positioning calibration system of the present invention. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0031] Those skilled in the art will appreciate that, unless otherwise stated, the singular forms "a", "an", "said" and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present invention refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0032] The present invention provides a method and system for autonomous three-dimensional detection and positioning calibration of a water jet robot, and a hand-eye integrated water jet robot in which a water jet cutter head and a visual laser scanner are all installed at the end of a mechanical arm. The three-dimensional detection and positioning calibration system is as follows: Figure 1 As shown, it includes: a robot end 1-1, a water jet cutter head 1-2, a line laser scanner 1-3, a visual camera 1-4, a six-dimensional force sensor 1-5, a calibration target 1-6, and a target positioning hole 1-7.

[0033] The robot end 1-1 is used to fix the water jet cutter head 1-2, the line laser scanner 1-3, the visual camera 1-4, and the six-dimensional force sensor 1-5, wherein the direction of the water jet cutter head 1-2 is in a different direction from the line laser scanner 1-3, such as 180 degrees completely opposite, so as to prevent water from splashing onto the scanner during operation.

[0034] The visual camera 1-4 performs hand-eye calibration on the robot end 1-1, and locates the work target in a large range, and determines the scanning range, scanning distance and scanning path for the line laser scanner 1-3.

[0035] The calibration target 1-6 is a square plane plate with a flatness much higher than the resolution of the line laser scanner 1-3, and there are four target positioning holes 1-7 near the four corners on the plane.

[0036] The installation axis of the six-dimensional force sensor 1-5 is in a parallel position with the water jet cutter head 1-2. The best installation method is a coaxial position, and it is coaxial with the output rotation axis of the robot terminal 1-1 to measure the reaction force during the water jet operation.

[0037] The following is described by specific examples: First embodiment like Figure 2 As shown, this embodiment provides a method for autonomous three-dimensional detection and positioning calibration of a water jet robot, comprising the following steps: S1: Manually control the end of the robotic arm to perform a rough scan to find a calibration target, so that the position and distance of the calibration target are within the effective working range of the line laser scanner.

[0038] In this implementation, step S1 is specifically as follows: placing the calibration target in the working area with the plane normal of the calibration target facing the end of the robot, manually controlling the movement of the end of the robot so that the visual camera faces the calibration target, and the calibration target is in the middle area of ​​the field of view of the visual camera, and confirming that the position and distance of the calibration target are within the effective working range of the line laser scanner.

[0039] S2: Perform precise scanning path planning of the line laser scanner to determine the scanning directions X and Y of the line laser scanner.

[0040] In this embodiment, step S2 is specifically as follows: Figure 1 As shown, taking the robot end as the coordinate reference, it is determined that the line laser scanner can completely cover the working area of ​​the calibration target, and the robot end drives the line laser scanner to move in a plane parallel to the calibration target, and determines the orthogonal scanning directions X and Y according to the principle of the calibration target from left to right and from front to back.

[0041] S3: The line laser scanner performs multiple scans along the X and Y directions respectively to obtain multiple point cloud data of the calibration target.

[0042] In this embodiment, step S3 specifically includes: controlling the end of the robot to move forward and backward multiple times in the X and Y directions respectively, driving the line laser scanner to scan the calibration target, and controlling the scanning speed Vx or Vy of the line laser scanner to be within the stable range of the moving line speed of the multi-axis robot; At the same time, the spatial coordinates of the robot end relative to the robot body are recorded. If the continuous spatial distance change of the spatial coordinates reaches the stepping accuracy of the scanning direction of the line laser scanner, the spatial coordinates are used as a scanning signal; otherwise, the moving speed value of the robot end is used as the basis for the stepping calculation of the line laser scanner; the line laser scanner scans to obtain multiple point cloud data of the calibration target.

[0043] Among them, the spatial coordinates of the robot end relative to the robot body are recorded. If the continuous spatial distance change of the spatial coordinates reaches the stepping accuracy of the line laser scanner in the scanning direction, the spatial coordinates are used as the scanning signal. Otherwise, the moving speed value of the robot end is used as the basis for the stepping calculation of the line laser scanner. Specifically: When the robot is working, the position of its end (the part where the line laser scanner is installed) is very critical information. We need to establish a coordinate system to describe the position of the robot's end, and this coordinate system is based on the robot body. For example, assuming that the center position of the robot body is the origin of the coordinate system, the three-dimensional spatial coordinates (x, y, z) of the robot's end in this coordinate system can be obtained in real time through the sensors or kinematic models inside the robot. This is like being in a room (representing the robot's workspace) and using a fixed corner of the room (the robot body) as the starting point to determine the position of a moving object (the robot's end).

[0044] The concept of stepping accuracy: There is an important parameter called stepping accuracy in the scanning process of line laser scanners. It indicates the position accuracy required for the scanner to perform the next scanning action in the scanning direction each time. You can think of it as the accuracy requirement for the scanner to scan each time it moves to the next "small grid" (the size of this "small grid" is the stepping accuracy) when scanning an object. Judgment condition: When the recorded spatial coordinates of the end of the robot are continuously changing, and the distance change in the scanning direction reaches the stepping accuracy required by the scanner, an important condition is met. For example, if the scanning direction of the scanner is along the x-axis, and the stepping accuracy requirement is 1 mm, then this condition is triggered when the coordinates of the end of the robot in the x-axis direction continuously change by 1 mm.

[0045] Once the above distance change reaches the stepping accuracy, the spatial coordinates of the robot end recorded at this time are used as a scanning signal. This signal is very important for the scanner. It can trigger the scanner to scan next time or serve as a key mark in subsequent data processing. For example, the scanner can adjust its scanning frequency according to this signal or record the scanned data at this time and associate it with this spatial coordinate, so as to accurately splice and process the scanned point cloud data later. This is like giving the scanner an instruction, telling it "You have now reached the right position and can scan next time."

[0046] If the continuous spatial distance change of the robot's end spatial coordinates does not reach the stepping accuracy of the line laser scanner's scanning direction, it means that the position change of the robot's end has not met the scanner's strict stepping requirements. In this case, it is necessary to change another way to provide a basis for the scanner's stepping calculation. At this time, the moving speed value of the robot's end is used for calculation. By knowing the moving speed of the robot's end and combining some characteristic parameters of the scanner itself (such as scanning frequency, scanning line spacing, etc.), it is possible to indirectly infer when the scanner should perform the next scanning action.

[0047] For example, if the end of the robot moves at a constant speed and the scanning frequency of the scanner is known to be 10 times per second, then this speed can be used to calculate how far the end of the robot should move between scans, so as to reasonably arrange the next scanning action of the scanner and ensure that the scanning process can obtain the required information as accurately as possible.

[0048] S4: Record and process the point cloud data, remove redundant point cloud data, and retain the point cloud data of the calibration target.

[0049] In this embodiment, step S4 is specifically as follows: based on the physical model of the robot end and the assembly model of the line laser scanner, the coordinate transformation matrix of the line laser scanner relative to the robot body is calculated; the point cloud scanned by the line laser scanner is converted to the robot coordinate system, and redundant point cloud data is eliminated, and only the point cloud data of the calibration target is retained.

[0050] For example, step S4 may adopt the following specific solution: 1. Calculate the coordinate transformation matrix of the line laser scanner relative to the robot body 1. Clarify key information of the model Robot terminal physical model: Carefully study the structure of the robot end to determine the range of angles within which each joint can move, such as the range within which a joint can rotate. At the same time, accurately grasp the length of the connecting rods between the joints and the position coordinates of the joint rotation center in the coordinate system set by the end itself. These detailed information can be obtained from the robot's product manual, technical data, or through actual measurement operations.

[0051] Line laser scanner assembly model: Find out how the line laser scanner is installed on the end of the robot. Clarify its positional offset relative to the robot's end coordinate system, that is, the specific offset values ​​of the scanner installation position in different directions (such as x, y, and z directions) under the end coordinate system. In addition, it is also necessary to determine the scanner's posture relative to the end coordinate system, that is, the angle value of its rotation around different coordinate axes (such as x, y, and z axes). These installation-related parameters can be obtained through precise measurement and calibration processes when installing the scanner.

[0052] 2. Establish relevant coordinate system Robot body coordinate system: Usually the center of the robot base or a fixed point that is convenient for describing the motion state of the entire robot is selected as the origin of this coordinate system. Then the direction of the coordinate axis is determined according to the right-hand rule, for example, the x-axis extends horizontally toward the front of the robot, the y-axis extends horizontally and perpendicular to the x-axis to the left, and the z-axis extends vertically upward. In this way, a fixed coordinate system is built to describe the position and posture of the entire robot in space.

[0053] Robot end coordinate system: This coordinate system is established based on the actual shape and structure of the robot end and its movement characteristics. Generally, a representative point of the end is selected as the origin, such as the center of the end joint or the center of the end effector mounting surface. The direction of the coordinate axis is also determined according to the right-hand rule. It should be noted that there is a corresponding relationship between the position and posture of this end coordinate system and the robot body coordinate system established earlier, which changes with the movement of the robot joint.

[0054] Line laser scanner coordinate system: This coordinate system is established with a specific point on the line laser scanner, such as the laser emission point or the geometric center of the scanner, as the origin. The direction of the coordinate axis can be determined according to the design characteristics of the scanner itself and the principle of convenient measurement. For example, let the x-axis be along the direction of the laser scanning, the y-axis be perpendicular to the scanning plane and upward, and the z-axis and the x- and y-axis form a corresponding relationship according to the right-hand rule. And it is necessary to know the installation relationship between this coordinate system and the robot end coordinate system, that is, the relationship described by the position offset and rotation angle determined previously.

[0055] 3. Calculate the coordinate transformation matrix Coordinate transformation from the robot body to the robot end: Using the relevant knowledge and principles of robot kinematics, according to the angle of the joints in the physical model of the robot end, the length of the connecting rod and other parameters, a series of specific steps are used to calculate the coordinate transformation from the robot body coordinate system to the robot end coordinate system. Although the specific formula is not written here, the general idea is to gradually determine the influence of each joint on the end position and posture through the motion analysis of each joint of the robot, and finally summarize the coordinate transformation relationship from the body to the end.

[0056] Coordinate transformation from the robot end to the line laser scanner: According to the installation position (that is, the offset values) and posture (the rotation angle values) and other parameters determined in the line laser scanner assembly model, the coordinate transformation from the robot end coordinate system to the line laser scanner coordinate system is calculated using the relevant theory of homogeneous coordinate transformation. Simply put, the information such as position offset and rotation angle is converted into a form that can be used to calculate the coordinate transformation, and then calculated according to certain rules to obtain the coordinate transformation matrix from the end to the scanner.

[0057] Calculate the final coordinate transformation matrix: Multiply the previously calculated coordinate transformation matrix from the robot body to the robot end and the coordinate transformation matrix from the robot end to the line laser scanner according to the rules of matrix multiplication, so that the coordinate transformation matrix of the line laser scanner relative to the robot body can be obtained.

[0058] 2. Convert the point cloud to the robot coordinate system and remove redundant point cloud data 1. Understand point cloud data The point cloud data obtained by the line laser scanner is recorded with reference to the scanner's own coordinate system. The data of each point contains the three-dimensional coordinate information (such as x_s, y_s, z_s) in the scanner's coordinate system, and may also have some other attribute information, such as reflection intensity. These point cloud data are collected in a certain order (such as the order of scan lines or time order) during the scanning process and stored in the computer's memory or external storage device.

[0059] 2. Convert point cloud to robot coordinate system The coordinate transformation matrix of the line laser scanner relative to the robot body that has been calculated previously is used.

[0060] For each point in the point cloud data, first convert it from the ordinary 3D coordinate form to the homogeneous coordinate form. For example, if a point was originally (x_s, y_s, z_s), it will become (x_s, y_s, z_s, 1).

[0061] Then use this point in homogeneous coordinate form and the coordinate transformation matrix calculated previously to perform a special multiplication operation (although the specific formula is not written here, it is done according to the general rules of homogeneous coordinate multiplication) to obtain a new result. Then convert this new result from homogeneous coordinate form back to ordinary three-dimensional coordinate form, so that the coordinates of this point in the robot coordinate system are obtained. According to this method, all point cloud data points are converted in this way, and the entire point cloud data is converted from the scanner coordinate system to the robot coordinate system.

[0062] 3. Eliminate redundant point cloud data Identify the characteristics of the redundant point cloud: When scanning the calibration target, except for the point cloud reflected from the target surface and received by the scanner, which is the valid point cloud we want (that is, the point cloud data of the calibration target), other point clouds generated due to various reasons are redundant. For example, point clouds reflected from the walls and ground around the scanning area, or point clouds generated due to incomplete scanning due to occlusion of the scanner's own components, are all redundant point clouds.

[0063] Culling based on spatial region range: If you know the approximate size of the calibration target and its position range in the robot coordinate system, you can set a bounding box to remove redundant point cloud data. First, determine the coordinate range of the six faces of the bounding box in the robot coordinate system, such as the minimum x coordinate is x_min, the maximum x coordinate is x_max, the minimum y coordinate is y_min, the maximum y coordinate is y_max, the minimum z coordinate is z_min, and the maximum z coordinate is z_max. Then check all the point cloud data points that have been converted to the robot coordinate system one by one. For each point, if its x coordinate is less than x_min or greater than x_max, or its y coordinate is less than y_min or greater than y_max, or its y coordinate is less than y_min or greater than y_max, or its z coordinate is less than z_min or greater than z_max, then treat this point as redundant point cloud data and remove it.

[0064] Eliminate based on point cloud data attributes: If the calibration target has a specific reflection intensity characteristic (different from the surrounding environment), you can remove redundant point cloud data by setting a threshold for the reflection intensity. First determine the approximate range of the calibration target's reflection intensity, for example, the lower limit of the reflection intensity threshold is I_min and the upper limit is I_max. Then check all the point cloud data points that have been converted to the robot coordinate system one by one. For each point, if its reflection intensity is less than I_min or greater than I_max, then remove this point as redundant point cloud data. In this way, you can only keep the point cloud data of the calibration target.

[0065] S5: Estimating and calibrating error characteristics of the retained point cloud data of the calibration target.

[0066] In this embodiment, step S5 is specifically as follows: Figure 3 , the ideal state of the scanned point cloud of the calibration target should be Figure 3 As shown on the left, the line laser point distance Dy and the scanning distance Dx should be constant, and all point clouds should be in the same plane in the depth direction. Figure 3 The right shows the actual point cloud data status. Each point cloud pixel has errors of Δdy, Δdx, and Δdz. The error is related to the movement speed and position accuracy of the robot end along the scanning direction. There are large differences between different robot products, and the accuracy is often much lower than that of a line laser scanner. Calculate the mean Ey, Ex and variance Dy, Dx of the point cloud pixel distance in the linear laser direction Y and scanning direction X of the point cloud data for each scan; fit the point cloud plane of the calibration target for each scan of the point cloud data, and calculate the mean Ez and variance Dz of the point cloud pixel distance in the depth direction z; wherein the mean is used to perform overall scaling calibration on the point cloud data obtained by normal operation scanning, and the variance is used to compensate for the spatial offset of the local fitting plane of the point cloud data obtained by normal operation scanning and each point cloud pixel relative to the local fitting plane.

[0067] For example, step S5 may adopt the following specific solution: 1. Calculate the mean of point cloud data in different directions 1. Clarify the composition and processing objects of point cloud data First of all, it should be clear that the point cloud data obtained by each scan of the line laser scanner is a collection of many discrete points. We regard each point as an independent individual, which contains some information. Here we focus on its coordinate values ​​in different directions. For example, for one of the points, we can describe it in a simple way. Suppose we call it the scanning direction X direction, the line laser direction Y direction, and the depth direction Z direction. Then this point has coordinate values ​​in these three directions. We can represent this point like this: the point number is i, its coordinate value in the X direction is Xi, its coordinate value in the Y direction is Yi, and its coordinate value in the Z direction is Zi. We need to perform the following processing on all these points obtained from each scan.

[0068] 2. Calculate the mean values ​​Ey and Ex in the line laser direction Y and scanning direction X Calculate Ex (mean value in scanning direction X): We need to extract the coordinate value Xi of each point in the X direction from all the points obtained in each scan. Imagine that we take out these coordinate values ​​one by one and put them aside.

[0069] Then, we prepare a place to add up these Xi values. We can call this place the sum storage place, and set it to 0 at the beginning. Next, we look at these points one by one, and each time we see a point, we add its Xi value to this sum storage place.

[0070] After we have looked at all the points, we know how many points we got in this scan, and we record the number of points as N. Then, the mean value Ex of the point cloud pixel distance in the scanning direction X is equal to the value in the sum storage we just added up divided by N. In simple terms, it is to add up the coordinate values ​​of all points in the X direction and divide it by the total number of points, so that the value of Ex is obtained.

[0071] Calculate Ey (the mean value in the Y direction of the line laser): The same approach is used, but this time for each point's coordinate value Yi in the Y direction. We still prepare a new sum storage location, which is initially set to 0. Then we look at all the points one by one, and each time we see a point, we add its Yi value to this new sum storage location.

[0072] When we have looked at all the points and know the total number of points N, then the mean Ey of the point cloud pixel distance in the line laser direction Y is equal to the value in the new sum storage divided by N.

[0073] 3. Calculate the mean value Ez in the depth direction Z But before calculating Ez, we have to do one thing for each scanned point cloud data, that is, to fit the point cloud plane corresponding to these point cloud data. The fitting process will be described in detail later. Now we know that we must first complete this fitting to obtain the point cloud plane corresponding to the point cloud data scanned this time.

[0074] After completing the fitting, we can start calculating Ez. We extract the coordinate value Zi of each point in the Z direction from all the points in this scan.

[0075] Similarly, prepare a sum storage place and set it to 0. Then look at all the points one by one, and each time you see a point, add its Zi value to this sum storage place.

[0076] When we have looked at all the points and know the total number of points N, then the mean Ez of the point cloud pixel distance in the depth direction Z is equal to the value in the sum storage divided by N.

[0077] 2. Calculate the variance of point cloud data in different directions 1. Calculate the variance Dy and Dx in the line laser direction Y and scanning direction X Calculate Dx (the variance in the scanning direction X): First, according to the previous method of calculating Ex, we have obtained the mean value Ex of the point cloud pixel distance in the scanning direction X of this scan.

[0078] Next, we need to re-examine all the points obtained from each scan. For each point, we need to calculate the difference between the coordinate value Xi of this point in the X direction and the mean Ex just obtained. This difference can be recorded as the difference between Xi and Ex. Then, we perform an operation on this difference, that is, square it, to get a new value, which can be recorded as the square of the difference between Xi and Ex.

[0079] We prepare a new place to add up the squared values ​​of the differences between Xi and Ex, and set it to 0 at the beginning. Then we look at all the points one by one, and every time we see a point, we add the squared value of the difference between Xi and Ex to this new place.

[0080] After we have looked at all the points, we know how many points we got in this scan, recorded as N. Then, the variance Dx of the point cloud pixel distance in the scanning direction X is equal to the value in the new place we just added up divided by N. In simple terms, it is to add up the square of the difference between the coordinate value of each point in the X direction and the mean, and then divide it by the total number of points, so that the value of Dx is obtained.

[0081] Calculate Dy (the variance of the laser line in the Y direction): Similarly, firstly, according to the previous method of calculating Ey, we have obtained the mean value Ey of the point cloud pixel distance in the laser direction Y of this scan.

[0082] Then we look at all the points obtained from each scan again. For each point, we need to calculate the difference between the Y coordinate value Yi of this point and the mean value Ey just obtained, and record it as the difference between Yi and Ey. Then we square this difference to get the square of the difference between Yi and Ey.

[0083] Prepare a new place to add up the squared values ​​of the Yi and Ey differences, and set it to 0 at the beginning. Then look at all the points one by one, and every time you see a point, add the squared value of its Yi and Ey difference to this new place.

[0084] After we have looked at all the points, we will know how many points we got in this scan, recorded as N. Then, the variance Dy of the point cloud pixel distance in the line laser direction Y is equal to the value in the new place we just added divided by N.

[0085] 2. Calculate the variance Dz in the depth direction Z First, according to the previous method of calculating Ez, we have obtained the mean value Ez of the point cloud pixel distance in the depth direction Z of this scan.

[0086] Next, we look at all the points obtained from each scan. For each point, we need to calculate the difference between the coordinate value Zi of this point in the Z direction and the mean value Ez just obtained, and record it as the difference between Zi and Ez. Then square this difference to get the square of the difference between Zi and Ez.

[0087] Prepare a new place to add up the squared values ​​of the difference between Zi and Ez, and set it to 0 at the beginning. Then look at all the points one by one, and every time you see a point, add the squared value of the difference between Zi and Ez to this new place.

[0088] After we have looked at all the points, we will know how many points we got in this scan, recorded as N. Then, the variance Dz of the point cloud pixel distance in the depth direction Z is equal to the value in the new place we just added divided by N.

[0089] 3. Fitting point cloud plane We need to fit the point cloud plane for each scanned point cloud data. Here we can use a relatively simple way to understand this fitting process. We need to find a plane so that the sum of the distances from all point cloud data points to this plane is the smallest. Although the specific algorithm may be more complicated, we can use some existing software tools or libraries to complete this fitting operation. For example, there are some software specifically used for point cloud processing. They have corresponding functions that allow you to directly input point cloud data, and then it will help us fit a plane. Alternatively, you can use the point cloud processing library in some programming languages, follow the interfaces and instructions they provide, input point cloud data, and get a fitted plane.

[0090] 4. Use mean and variance to calibrate and compensate point cloud data 1. Overall scaling calibration (using mean) For the point cloud data obtained by normal operation scanning, we take the calibration using Ex as an example: We need to look at all the point cloud data points obtained by normal operation scanning. For each point, we assume that its coordinate value in the X direction is Xj, its coordinate value in the Y direction is Yj, and its coordinate value in the Z direction is Zj.

[0091] We need to calibrate the coordinate value of this point in the X direction. Specifically, divide the coordinate value Xj of this point in the X direction by Ex just calculated to get a new value, which we can record as the calibrated value of Xj. In other words, the calibrated value of Xj is equal to Xj divided by Ex.

[0092] In the same way, use Ey to calibrate the coordinate value of each point in the Y direction, and the calibrated value of Yj is equal to Yj divided by Ey; use Ez to calibrate the coordinate value of each point in the Z direction, and the calibrated value of Zj is equal to Zj divided by Ez.

[0093] 2. Gap offset compensation (using variance) Take the depth direction Z as an example to illustrate the process of using variance to compensate for spatial offset: First, based on the point cloud plane fitted above, we need to calculate the distance from each point cloud data point obtained by normal operation scanning to this plane. Here we can use some software tools or libraries to help us calculate this distance. For example, some point cloud processing software or libraries have special functions that can be used to calculate the distance from a point to a plane. We can follow their instructions, input the point cloud data point and the fitted plane, and then we can get the distance from this point to the plane. We record this distance as the distance from Zj to the plane.

[0094] Then, we need to compensate the spatial offset of each point based on the calculated variance Dz. For example, we can set a compensation coefficient, and we can take any value for this compensation coefficient, such as 1 (in actual applications, it may need to be adjusted according to specific circumstances). Then we need to compensate the coordinate value Zj of this point in the Z direction. Specifically, add the original coordinate value Zj of this point in the Z direction to this compensation coefficient multiplied by the distance from Zj to the plane divided by Dz to get a new value. We can record this new value as the calibrated value of Zj. In other words, the calibrated value of Zj is equal to Zj + 1×the distance from Zj to the plane divided by Dz.

[0095] In the same way, the variance Dx and Dy in the scanning direction X and the line laser direction Y are used to compensate for the spatial offset of each point in the corresponding direction, so as to realize the spatial offset compensation of the local fitting plane of the point cloud data obtained by normal operation scanning and each point cloud pixel relative to the local fitting plane.

[0096] S6: Manually control the water jet end calibration to correct the spatial offset of the water jet cutter head end.

[0097] In this embodiment, if Figure 4 As shown, step S6 specifically comprises: calculating the first spatial coordinates corresponding to the four target positioning holes in the point cloud data scanned multiple times, and taking the average; manually controlling the end of the water jet cutter head to be positioned at the four target positioning holes respectively, and recording the second spatial coordinates of the end of the water jet cutter head in the current robot system; comparing the coordinate values ​​of the first spatial coordinate and the second spatial coordinate, and taking the components in the x, y, and z directions to perform the spatial offset correction of the end of the water jet cutter head.

[0098] S7: Filter the point cloud data acquired during normal operation to obtain global error compensation.

[0099] In this embodiment, step S7 is specifically as follows: placing the calibration target in different positions in the working space, repeating steps S1-S5 to obtain multiple groups of measurement data and calibration data of the point cloud data; obtaining statistical data of multiple scans at the same time, the statistical data include: the moving speed of the robot end and the moving direction of the spatial scanning trajectory, the mean and variance obtained in step S5, and the spatial offset obtained in step S6; the transformation matrix after the robot hand-eye calibration is distorted, stretched, and translated so that the obtained point cloud data completely matches the actual physical size and flatness of the calibration target, thereby compensating for the positioning error of the detection in the calibration space area; when the water jet robot is operating normally, the line laser scanner scans the working target of any curved surface, and the compensation method is as follows: dividing the point cloud data of the scanned working target into a number of uniformly sized point cloud pieces in the projection direction of the scanning direction, performing point-by-point calibration according to the calibration offsets obtained in steps S5, S6, and S7, and then filtering the overall point cloud data to obtain global error compensation.

[0100] For example, step S7 may adopt the following specific solution: 1. Obtain multiple sets of measurement data, calibration data, and statistical data 1. Place the calibration target and repeat the scan Place the calibration target in different positions in the workspace. You can achieve multiple positions of the calibration target by designing a specific fixture or using the robot's own motion function. For example, in a Cartesian coordinate system, you can change the position of the calibration target in the x, y, and z directions and the rotation angle around the x, y, and z axes.

[0101] Repeat steps S1 - S5 for each calibration target in different positions. Steps S1 - S5 should be operations related to robot scanning and data acquisition. After each operation, corresponding point cloud data is obtained, thereby obtaining multiple sets of measurement data and calibration data. These data will provide the basis for subsequent analysis and processing.

[0102] 2. Collect statistics The moving speed of the robot end and the moving direction of the spatial scanning trajectory: During the robot scanning process, the moving speed information of the end is obtained through the sensors inside the robot (such as encoders, etc.). The moving speed can be stored in a specific data format in the robot's control system, such as in millimeters per second or meters per second. For the moving direction of the spatial scanning trajectory, it can be determined by recording the coordinate changes of the robot end at each moment. For example, if the x-coordinate value of the robot end increases over a period of time, and the y and z-coordinate values ​​are relatively stable, then the scanning trajectory has a moving component in the x-direction. These direction information can be expressed in the form of vectors, such as (x-direction component, y-direction component, z-direction component).

[0103] Mean and variance obtained in step S5: According to the method of step S5 (such as the operation of calculating the mean and variance of point cloud data in different directions mentioned above), the mean and variance of each scanned point cloud data in the relevant direction are calculated again. These values ​​will be used in the subsequent calibration and compensation process.

[0104] The spatial offset obtained in step S6: The spatial offset information of each scan is obtained by using the method for determining the spatial offset in step S6. This may involve comparing the coordinate differences under different measurement methods or different reference systems.

[0105] 2. Correct the conversion matrix after robot hand-eye calibration 1. Distortion Correction Analyze the distortion of the object shape in the point cloud data. Distortion can be determined by comparing the point cloud data of the calibration target in different poses with the known actual shape of the calibration target. For example, if the calibration target is originally rectangular, but has a trapezoidal shape in the point cloud data, it may indicate a distortion error.

[0106] Adjust the transformation matrix after hand-eye calibration of the robot according to the degree and direction of distortion. This can be achieved by introducing appropriate rotations or nonlinear transformations in the matrix. The specific adjustment method may require mathematical derivation and experimentation based on the robot kinematic model and the principle of hand-eye calibration. For example, if it is found that the point cloud data has an angular deviation in a certain plane, a rotation term around an axis perpendicular to the plane can be added to the transformation matrix to correct the distortion.

[0107] 2. Stretch correction Check the difference between the size of the point cloud data in different directions and the actual physical size of the calibration target to determine if there is a stretch error. You can calculate the length of a specific line segment or feature in the point cloud data and compare it with the actual length of the corresponding line segment or feature on the calibration target. For example, if the actual distance between two marked points on the calibration target is 100 mm, but the distance measured in the point cloud data is 120 mm, it indicates that there is stretch in that direction.

[0108] Modify the transformation matrix based on the scale and direction of the stretch. If there is stretching in a certain direction, it can be corrected by adjusting the scaling factor of the corresponding direction in the transformation matrix. For example, if there is stretching in the x-direction, the elements in the transformation matrix related to the x-direction coordinate transformation can be multiplied by an appropriate scaling factor to make the size of the point cloud data in that direction match the actual size.

[0109] 3. Translation correction The translation error is determined by comparing the overall position of the point cloud data with the actual position of the calibration target. The coordinates of some known feature points on the calibration target can be compared in the point cloud data with their actual coordinates. For example, if the center feature point on the calibration target is actually at (0, 0, 0) but is at (5, 3, 2) in the point cloud data, then there is a translation error of (5, 3, 2).

[0110] According to the translation error vector, the corresponding translation term is added to the transformation matrix for correction. This can directly modify the elements related to the coordinate translation in the transformation matrix so that the overall position of the point cloud data coincides with the actual position of the calibration target, thereby compensating for the positioning error of the detection in the calibration space area.

[0111] 3. Compensate for the point cloud data of the operating target scanned during normal operation 1. Point cloud data segmentation When the water jet robot is operating normally, the line laser scanner scans the working target on any curved surface to obtain point cloud data of the working target.

[0112] The point cloud data of the scanned work target is divided in the projection direction of the scanning direction. First, the scanning direction is determined, which can be determined according to the installation angle and scanning principle of the line laser scanner. For example, if the laser emission direction of the scanner is the main scanning direction, this direction can be used as a reference. Then, the point cloud data is divided into several uniformly sized point cloud slices according to a predetermined method. The size of each point cloud slice can be determined according to the range of the point cloud data and the desired number of point cloud slices. For example, if the length of the point cloud data in the scanning direction is 1000 units and it is desired to be divided into 10 point cloud slices, the length of each point cloud slice in the scanning direction is 100 units.

[0113] 2. Point-by-point calibration Calibrate each point cloud piece point by point according to the calibration offsets obtained in steps S5, S6, and S7. For each point in the point cloud piece, adjust its coordinates according to the calibration offset. The calibration offset may include information such as position offset and scaling factor. For example, if the calibration offset has a translation of 5 units in the x direction and a scaling factor of 0.9, for a point in the point cloud with an x ​​coordinate value of x0, the calibrated x coordinate value is (x0 * 0.9 + 5). Perform similar calibration operations for each point in each direction.

[0114] 3. Filtering Perform filtering on the overall point cloud data after point-by-point calibration. You can choose a suitable filtering algorithm, such as statistical filtering, radius filtering, etc.

[0115] Statistical filtering: Count the number and distribution of other points in a certain neighborhood around each point. If the number of points around a point is too small or the distance distribution between a point and its neighborhood points is abnormal (for example, the distance is too large or too small), the point is considered a noise point and is removed. For example, if the neighborhood radius of a point is set to 5 units, if the number of points in this neighborhood is less than 10, the point is considered a noise point.

[0116] Radius filtering: Set a sphere with a fixed radius with each point as the center. If the number of points in the sphere exceeds or falls below a certain threshold, the point is processed accordingly (such as retained or removed). For example, for a sphere with a radius of 3 units, if the number of points in the sphere is greater than 20, the point is retained, otherwise it is removed. Noise points are removed through filtering to obtain point cloud data after global error compensation.

[0117] S8: Perform dynamic positioning correction on the operation target.

[0118] In this embodiment, step S8 is specifically as follows: after the robot completes the scanning calibration, the calibration target is removed, and the robot end directs the water jet cutter head end toward the working target direction, and the current posture data of the multi-axis robot is recorded; the water jet high-pressure water pump is turned on, and the current water jet nozzle diameter, water pressure, sand ratio, and the current posture data of the multi-axis robot are recorded; the spatial coordinates of the robot end before and after the water pump operation are turned on are recorded and compared, and the displacement deviation under the influence of the reaction force is recorded; different parameters including the water jet nozzle diameter, water pressure, sand ratio, and the direction of the water jet cutter head are changed, multiple groups of displacement deviations are recorded, and the fitting mean is statistically calculated as the compensation value for the positioning of the water jet cutter head end when the robot is operating.

[0119] In addition, it should be noted that the multi-axis robot of the present invention is not limited to a two-section robot arm, a three-axis robot arm, or a five-axis to seven-axis robot arm, or other multi-joint or multi-track or flexible robots. Optionally, the visual camera is not limited to a two-dimensional black and white or color camera, and can also be an RGBD depth camera, a structured light camera, or a binocular camera. Optionally, the calibration target can be made of metal processing, or plastic, glass, ceramics, and other materials that are not easily deformed, depending on the accuracy requirements of the use scenario, and its size can be selected according to the coverage area of ​​the line laser scanner.

[0120] Second embodiment like Figure 5 As shown, this embodiment provides a water jet robot autonomous operation three-dimensional detection and positioning calibration system for executing the water jet robot autonomous operation three-dimensional detection and positioning calibration method as in the first embodiment, comprising: A manual rough scanning module 1 is used to manually control the end of the robot arm to perform a rough scan to find a calibration target so that the position and distance of the calibration target are within the effective working range of the line laser scanner; A precise scanning path planning module 2 is used to perform precise scanning path planning of the line laser scanner and determine scanning directions X and Y of the line laser scanner; A point cloud data acquisition module 3 is used for the line laser scanner to perform multiple scans along the X and Y directions respectively, and scan to obtain multiple point cloud data of the calibration target; A point cloud data processing module 4 is used to record and process the point cloud data, remove redundant point cloud data, and retain the point cloud data of the calibration target; An error characteristic estimation module 5, used for performing error characteristic estimation and calibration on the retained point cloud data of the calibration target; The water jet end calibration module 6 is used to manually control the water jet end calibration and correct the spatial offset of the water jet cutter head end; A global error compensation module 7, used for filtering the point cloud data acquired during normal operation, so as to obtain global error compensation; The operation target calibration module 8 is used to perform dynamic positioning correction on the operation target.

[0121] A computer-readable storage medium stores computer code. When the computer code is executed, the above method is executed. A person skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program. The program can be stored in a computer-readable storage medium. The storage medium can include: a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc.

[0122] The above is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.

[0123] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0124] It should be noted that the above embodiments can be freely combined as needed. The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered as the protection scope of the present invention.

Claims

1. A method for autonomous three-dimensional detection and positioning calibration of a water jet robot, characterized in that: The following steps are involved: S1: manually controlling the end of the robotic arm to perform a rough scan to find a calibration target, so that the position and distance of the calibration target are within the effective working range of the line laser scanner; S2: Perform precise scanning path planning of the line laser scanner to determine scanning directions X and Y of the line laser scanner; S3: the line laser scanner performs multiple scans along the X and Y directions respectively, and obtains multiple point cloud data of the calibration target; S4: Record and process the point cloud data, remove redundant point cloud data, and retain the point cloud data of the calibration target; S5: performing error feature estimation and calibration on the retained point cloud data of the calibration target; S6: Manually control the water jet end calibration to correct the spatial offset of the water jet cutter head end; S7: filtering the point cloud data acquired during normal operation to obtain global error compensation; S8: Perform dynamic positioning correction on the operation target.

2. The method for autonomous three-dimensional detection and positioning calibration of a water jet robot according to claim 1, characterized in that: In step S1, the end of the robotic arm is manually controlled to perform the rough scan to find the calibration target, so that the position and distance of the calibration target are within the effective working range of the line laser scanner, specifically: Place the calibration target in the working area with the plane normal of the calibration target facing the end of the robot. Manually control the movement of the robot end so that the visual camera faces the calibration target and the calibration target is in the middle area of ​​the field of view of the visual camera. Confirm that the position and distance of the calibration target are within the effective working range of the line laser scanner.

3. The method for autonomous three-dimensional detection and positioning calibration of a water jet robot according to claim 2, characterized in that: In step S2, the precise scanning path planning of the line laser scanner is performed to determine the scanning directions X and Y of the line laser scanner, specifically: Taking the robot end as the coordinate reference, it is determined that the line laser scanner can completely cover the working area of ​​the calibration target. The robot end drives the line laser scanner to move in a plane parallel to the calibration target, and the orthogonal scanning directions X and Y are determined according to the principle of the calibration target from left to right and from front to back.

4. The method for autonomous three-dimensional detection and positioning calibration of a water jet robot according to claim 2, characterized in that: In step S3, the line laser scanner performs multiple scans along the X and Y directions respectively to obtain a plurality of point cloud data of the calibration target, specifically: Control the end of the robot to move forward and backward multiple times in the X and Y directions respectively, drive the line laser scanner to scan the calibration target, and control the scanning speed Vx or Vy of the line laser scanner within the stable range of the moving line speed of the multi-axis robot; At the same time, the spatial coordinates of the robot end relative to the robot body are recorded. If the continuous spatial distance change of the spatial coordinates reaches the stepping accuracy of the line laser scanner in the scanning direction, the spatial coordinates are used as a scanning signal. Otherwise, the moving speed value of the robot end is used as the basis for the stepping calculation of the line laser scanner. The line laser scanner scans and obtains a plurality of point cloud data of the calibration target.

5. The method for autonomous three-dimensional detection and positioning calibration of a water jet robot according to claim 2, characterized in that: In step S4, the point cloud data is recorded and processed, redundant point cloud data is removed, and the point cloud data of the calibration target is retained, specifically: Calculate the coordinate transformation matrix of the line laser scanner relative to the robot body according to the physical model of the robot end and the assembly model of the line laser scanner; The point cloud scanned by the line laser scanner is converted to the robot coordinate system, redundant point cloud data is eliminated, and only the point cloud data of the calibration target is retained.

6. The method for autonomous three-dimensional detection and positioning calibration of a water jet robot according to claim 2, characterized in that: In step S5, error characteristic estimation and calibration are performed on the retained point cloud data of the calibration target, specifically: Calculate the mean values ​​Ey and Ex and the variances Dy and Dx of the point cloud pixel distances of the point cloud data in the linear laser direction Y and the scanning direction X for each scan; Fitting the point cloud plane of the calibration target for each scan of the point cloud data, and calculating the mean Ez and variance Dz of the point cloud pixel distance in the depth direction z; The mean is used to perform overall scaling calibration on the point cloud data obtained by normal scanning, and the variance is used to compensate for the spatial offset of the local fitting plane of the point cloud data obtained by normal scanning and each point cloud pixel relative to the local fitting plane.

7. The method for autonomous three-dimensional detection and positioning calibration of a water jet robot according to claim 6, characterized in that: In step S6, the water jet end is manually calibrated to correct the spatial offset of the water jet head end, specifically: Calculate the first spatial coordinates corresponding to the four target positioning holes in the point cloud data scanned multiple times, and take the average; Manually control the end of the water jet cutter head to be positioned at the four target positioning holes respectively, and record the second spatial coordinates of the end of the water jet cutter head in the current robot system; Compare the coordinate values ​​of the first spatial coordinate and the second spatial coordinate, and take the components in the x, y, and z directions to perform the spatial offset correction of the end of the water jet cutter head.

8. The method for autonomous three-dimensional detection and positioning calibration of a water jet robot according to claim 7, characterized in that: In step S7, the point cloud data acquired during normal operation is filtered to obtain global error compensation, specifically: Placing the calibration target in different positions in the working space, and repeating steps S1-S5 to obtain multiple sets of measurement data and calibration data of the point cloud data; At the same time, statistical data of multiple scans are obtained, the statistical data including: the moving speed of the robot end and the moving direction of the spatial scanning trajectory, the mean and variance obtained in step S5, and the spatial offset obtained in step S6; The transformation matrix after the robot hand-eye calibration is distorted, stretched, and translated so that the obtained point cloud data completely matches the actual physical size and flatness of the calibration target, thereby compensating for the positioning error of the detection in the calibration space area; When the water jet robot is operating normally, the line laser scanner scans the work target on any curved surface, and the compensation method is: the point cloud data of the scanned work target is divided into a number of point cloud pieces of uniform size in the projection direction of the scanning direction, and point-by-point calibration is performed according to the calibration offset obtained in steps S5, S6, and S7, and then the overall point cloud data is filtered to obtain global error compensation.

9. The method for autonomous three-dimensional detection and positioning calibration of a water jet robot according to claim 2, characterized in that: In step S8, dynamic positioning correction is performed on the operation target, specifically: After the robot completes the scanning calibration, the calibration target is removed, and the end of the robot points the end of the water jet cutter head toward the working target direction, and the current posture data of the multi-axis robot is recorded; Turn on the water jet high-pressure water pump and record the current water jet nozzle diameter, water pressure, sand ratio, and the current position data of the multi-axis robot; Record and compare the spatial coordinates of the robot end before and after the water pump is turned on, and record the displacement deviation under the influence of the reaction force; By changing different parameters including the water jet nozzle diameter, water pressure, sand ratio, and the orientation of the water jet cutter head, multiple groups of displacement deviations are recorded, and the fitting mean is statistically calculated as the compensation value for the positioning of the end of the water jet cutter head when the robot is operating.

10. A water jet robot autonomous operation three-dimensional detection and positioning calibration system for executing the water jet robot autonomous operation three-dimensional detection and positioning calibration method according to any one of claims 1 to 9, characterized in that: include: A manual rough scanning module, used to manually control the end of the robotic arm to perform a rough scan and find a calibration target so that the position and distance of the calibration target are within the effective working range of the line laser scanner; A precise scanning path planning module, used for performing precise scanning path planning of the line laser scanner and determining scanning directions X and Y of the line laser scanner; A point cloud data acquisition module, used for the line laser scanner to perform multiple scans along the X and Y directions respectively, to obtain multiple point cloud data of the calibration target; A point cloud data processing module, used to record and process the point cloud data, remove redundant point cloud data, and retain the point cloud data of the calibration target; An error characteristic estimation module, used for performing error characteristic estimation and calibration on the retained point cloud data of the calibration target; The water jet end calibration module is used to manually control the water jet end calibration and correct the spatial offset of the water jet cutter head end; A global error compensation module, used for filtering the point cloud data acquired during normal operation to obtain global error compensation; The operation target calibration module is used to perform dynamic positioning correction on the operation target.

Images