A man-machine collaborative multi-dimensional multi-mode industrial robot precision measurement method and device

Abstract

The application discloses a kind of man-machine cooperation formula multi-dimensional multi-mode industrial robot precision measurement method and device, it is related to precision measurement technical field, comprising: through ant colony-genetic hybrid optimization admittance control algorithm based on six-dimensional force data output target admittance control parameter, initial starting waypoint pose and initial endpoint waypoint pose are determined in combination with dichotomy iterative optimization strategy;Adaptive registration algorithm based on the angle constraint of double straight line is used to correct waypoint pose, and target starting waypoint pose and target endpoint waypoint pose are determined;According to target admittance control parameter, target starting waypoint pose and target endpoint waypoint pose, the measured workpiece is measured, and segmented weighted ICP registration algorithm is used to determine global transformation matrix, when it is judged that waypoint pose needs to be optimized, transformation is not executed waypoint pose to update target starting waypoint pose or / and target endpoint waypoint pose, until the workpiece measurement result is determined.The reliability of industrial robot precision measurement is improved based on the above scheme.

Description

Technical Field

[0001] The present invention relates to the technical field of precision measurement, and in particular to a precision measurement method and device for a human-machine collaborative multi-dimensional and multi-modal industrial robot. Background Technique

[0002] Precision measurement of industrial robots is the core inspection link of intelligent manufacturing and plays a decisive role in the automation level of the production line and the product quality. With the evolution of the manufacturing industry towards high-end, refined, and flexible development, the dynamic changes of dimensions such as the height and width of the test pieces with batches and models have become the norm in mainstream scenarios. Therefore, higher requirements are put forward for the rapid switching and flexible adaptation capabilities of the measurement scheme. However, in actual industrial applications, the existing precision measurement technologies can only stably adapt to the detection of workpieces with fixed dimensions and are difficult to adapt to the flexible production requirements of the dynamic changes in the height and width of workpieces. There are limitations such as low adaptation efficiency, high switching costs, and insufficient flexibility, and it is urgent to improve the reliability of precision measurement of industrial robots. Summary of the Invention

[0003] The present invention provides a precision measurement method and device for a human-machine collaborative multi-dimensional and multi-modal industrial robot, which is used to improve the reliability of precision measurement of industrial robots.

[0004] A precision measurement method for a human-machine collaborative multi-dimensional and multi-modal industrial robot provided by the first aspect of the present invention includes:

[0005] Driving the end of the robotic arm of the industrial robot to test the test workpiece, outputting target admittance control parameters based on six-dimensional force data through an ant colony-genetic hybrid optimization admittance control algorithm, and determining the initial starting path point pose and the initial and final path point poses of the test workpiece in combination with the dichotomy iterative optimization strategy;

[0006] Based on the calibrated pose of the workpiece detection frame of the test workpiece, according to the 2D workpiece image of the measured workpiece of the same model, an adaptive registration algorithm based on the double-line angle constraint is used to correct the initial starting path point pose and the initial and final path point poses, and determine the target starting path point pose and the target end path point pose;

[0007] Measuring the measured workpiece according to the target admittance control parameters, the target starting path point pose and the target end path point pose, using a segmented weighted ICP registration algorithm to determine the global transformation matrix between the real-time point cloud and the standard point cloud, and when it is determined that the path point pose needs to be optimized according to the global transformation matrix, transforming the unexecuted path point pose to update the target starting path point pose or / and the target end path point pose and continuing the measurement until the workpiece measurement result is determined.

[0008] Furthermore, the end effector of the robotic arm driving the industrial robot performs a test on the workpiece. Based on six-dimensional force data, a hybrid ant colony-genetic optimization admittance control algorithm is used to output target admittance control parameters. Combined with a bisection iterative optimization strategy, the initial measurement waypoint of the workpiece is determined, including:

[0009] The robotic arm end effector of the industrial robot is driven by external operating force to perform a test from the test start point pose to the test end point pose of the test workpiece based on the test run admittance control parameters, and the six-dimensional force sensor at the end effector of the robotic arm collects six-dimensional force data and determines whether a collision rebound occurs.

[0010] Ant colony building is based on ant colony algorithm, which uses admittance control parameters to build ant colonies and updates ant colonies by updating pheromones. The ant colony output by the genetic algorithm when the ant colony iteration stops is met is the initial population. The fitness is calculated using six-dimensional force data to execute the genetic operator. When the genetic iteration stops, the individual with the best fitness is output as the target admittance control parameter.

[0011] When a collision rebound occurs, the starting or ending road point pose of the test is adopted, which is closer to the test rebound pose of the robotic arm end point. The geometric midpoint between the starting and ending road point poses is calculated as the new starting or ending road point pose. The robotic arm end point is then controlled by the target admittance control parameters to continue testing from the starting road point pose to the ending road point pose until no collision rebound occurs, at which point the initial starting and ending road point poses are determined.

[0012] Further, the calibration posture of the workpiece detection frame based on the test workpiece, and the correction of the initial starting waypoint pose and the initial ending waypoint pose using an adaptive registration algorithm based on the double-straight-line angle constraint according to the 2D workpiece image of the actual test workpiece of the same model, to determine the target starting waypoint pose and the target ending waypoint pose, includes:

[0013] Based on the 2D image of the test workpiece, determine the coordinates of the corner points and the center point of the test workpiece detection frame.

[0014] The image transformation matrix of the robotic arm is determined based on the test 2D workpiece image using the nine-point calibration method, and the inverse is used to obtain the image transformation matrix of the robotic arm.

[0015] The initial starting point pose and the initial ending point pose are converted into the test start pixel coordinates and test end pixel coordinates using the robotic arm image transformation matrix.

[0016] Calculate the distance between the center point coordinates of the test frame and the starting pixel coordinates and the ending pixel coordinates of the test, respectively, and the distance between the starting center point and the ending center point.

[0017] Determine the general equation of the starting line of the test test between the coordinates of the center point of the test frame and the coordinates of the starting pixel of the test test, and determine the general equation of the ending line of the test test between the coordinates of the center point of the test frame and the coordinates of the ending pixel of the test test;

[0018] Determine the general equation of the test workpiece detection frame for the transverse straight line at the centerline in the transverse direction and the general equation of the test longitudinal straight line at the centerline in the longitudinal direction; wherein the straight line direction of the test start pixel coordinates and the test end pixel coordinates is the same as the transverse direction of the test workpiece detection frame.

[0019] Calculate the included angle of the initial test line based on the general equation of the initial test line and the general equation of the longitudinal test line;

[0020] Calculate the included angle between the test endpoint lines based on the general equation of the test endpoint line and the general equation of the test longitudinal line.

[0021] Based on the measured 2D workpiece image of the actual workpiece of the same model as the test workpiece, the coordinates of the center point of the measured frame and the general equation of the measured horizontal straight line are determined.

[0022] The angle between the transverse transformation lines is calculated using the general equation of the measured transverse line and the general equation of the test transverse line.

[0023] The starting angle is constructed by the angle between the starting line and the lateral transformation line, and the ending angle is constructed by the angle between the ending line and the lateral transformation line.

[0024] Based on the measured frame center point coordinates, the starting transformation angle, the starting center point distance, the image robot transformation matrix, and the workpiece standard height, the target starting waypoint pose is determined;

[0025] Based on the coordinates of the center point of the measured frame, the endpoint transformation angle, the distance to the center point of the endpoint, the image robot transformation matrix, and the standard height of the workpiece, the pose of the target endpoint waypoint is determined.

[0026] Further, the measured workpiece is measured according to the target admittance control parameters, the target starting waypoint pose, and the target ending waypoint pose. A segmented weighted ICP registration algorithm is used to determine the global transformation matrix between the real-time point cloud and the standard point cloud. When it is determined that the waypoint pose needs to be optimized based on the global transformation matrix, the poses of the unexecuted waypoints are transformed to update the target starting waypoint pose and / or the target ending waypoint pose, and the measurement continues until the workpiece measurement result is determined. This includes:

[0027] The robotic arm end effector was driven to move from the target starting point pose to the target ending point pose according to the target admittance control parameters, and real-time point cloud data was collected.

[0028] The centroids of the standard point cloud and the real-time point cloud of the test workpiece are calculated respectively to determine the centroids of the standard point cloud and the real-time point cloud.

[0029] An aligned standard point cloud is constructed by subtracting the centroid of the standard point cloud from the standard point cloud, and an aligned real-time point cloud is constructed by subtracting the centroid of the real-time point cloud from the real-time point cloud.

[0030] The coarse registration transformation matrix is ​​output based on the alignment standard point cloud and the alignment real-time point cloud using the point-to-surface ICP algorithm.

[0031] The coarse registration real-time point cloud is obtained by transforming the coarse registration transformation matrix. The alignment standard point cloud and the coarse registration real-time point cloud are downsampled to a low resolution to obtain a low-level standard point cloud and a low-level real-time point cloud. The low-level transformation matrix is ​​output by using the weighted ICP algorithm with the goal of minimizing the weighted error function.

[0032] The low-level real-time point cloud is transformed by the low-level transformation matrix to obtain the low-level transformed real-time point cloud. The low-level standard point cloud and the low-level transformed real-time point cloud are downsampled to the next higher resolution to obtain the high-level standard point cloud and the high-level real-time point cloud. The high-level transformation matrix is ​​output by using the weighted ICP algorithm with the goal of minimizing the weighted error function.

[0033] If the current resolution is not the highest resolution and the high-level transformation matrix does not meet the convergence condition, the high-level standard point cloud and the high-level real-time point cloud are transformed by the high-level transformation matrix and then downsampled to the next higher resolution to obtain a new high-level standard point cloud and a new high-level real-time point cloud. A new high-level transformation matrix is ​​output through the weighted ICP algorithm until the current resolution is the highest resolution or the high-level transformation matrix meets the convergence condition. Then, the coarse registration transformation matrix and the high-level transformation matrix are used to synthesize the global transformation matrix.

[0034] If the global transformation matrix analysis determines that waypoint pose optimization is needed, the unexecuted waypoint poses are transformed using the global transformation matrix to obtain a new target starting waypoint pose and / or a new target ending waypoint pose. The robotic arm end effector is then driven to continue moving towards the new target ending waypoint pose according to the target admittance control parameters until the workpiece measurement result is determined.

[0035] Furthermore, it also includes:

[0036] If a collision and rebound occur during the actual measurement, the target starting road point pose or the target ending road point pose, which is closer to the measured rebound pose of the robotic arm end point, is used. The geometric midpoint between the target starting road point pose and the measured rebound pose is calculated as the new test starting road point pose or the new test ending road point pose. The robotic arm end point is then controlled to test from the test starting road point pose to the test ending road point pose using the target admittance control parameters until no collision and rebound occur. Then, the initial starting road point pose and the initial ending road point pose are determined, and the target starting road point pose and the target ending road point pose are re-determined until the workpiece measurement result is determined.

[0037] Furthermore, if the step of analyzing the global transformation matrix determines that waypoint pose optimization is needed, it includes:

[0038] Decompose the rotation matrix and translation vector from the global transformation matrix, and calculate the corresponding rotation angle and translation distance;

[0039] When the rotation angle or the translation distance is greater than the corresponding threshold, it is determined that the waypoint pose needs to be optimized.

[0040] Furthermore, the fitness calculation process includes:

[0041] ;

[0042] In the formula, Represents an individual in a genetic algorithm. Indicates fitness. Represents the expected interaction force sequence. This represents the actual interaction force sequence composed of six-dimensional force data. This represents the desired end-effector position sequence of the robotic arm. This represents the actual position sequence of the robotic arm's end effector. Indicates the response adjustment time. This represents the Euclidean distance operation. Indicates different weighting coefficients. This represents the root mean square error.

[0043] The second aspect of this invention provides a human-machine collaborative multidimensional multimodal industrial robot precision measurement device, comprising:

[0044] The waypoint testing module is used to drive the end effector of the industrial robot to test the workpiece. It outputs target admittance control parameters based on six-dimensional force data through an ant colony-genetic hybrid optimization admittance control algorithm, and determines the initial starting waypoint pose and initial ending waypoint pose of the workpiece by combining a bisection iterative optimization strategy.

[0045] The waypoint correction module is used to calibrate the posture of the workpiece detection frame based on the test workpiece, and to correct the initial starting waypoint pose and the initial ending waypoint pose using an adaptive registration algorithm based on the double straight line angle constraint according to the 2D workpiece image of the actual test workpiece of the same model, so as to determine the target starting waypoint pose and the target ending waypoint pose.

[0046] The workpiece measurement module is used to perform actual measurements on the workpiece according to the target admittance control parameters, the target starting road point pose, and the target ending road point pose. It uses a segmented weighted ICP registration algorithm to determine the global transformation matrix between the real-time point cloud and the standard point cloud. When it is determined that the road point pose needs to be optimized according to the global transformation matrix, it transforms the poses of the unexecuted road points to update the target starting road point pose and / or the target ending road point pose and continues the measurement until the workpiece measurement result is determined.

[0047] A computer device provided in a third aspect of the present invention includes a memory and a processor. The memory stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of the human-machine collaborative multidimensional multimodal industrial robot precision measurement method as described in any of the preceding claims.

[0048] The fourth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed, implements the human-machine collaborative multidimensional multimodal industrial robot precision measurement method as described in any of the preceding claims.

[0049] As can be seen from the above technical solutions, the present invention has the following advantages:

[0050] The above-described solution of the present invention provides a human-machine collaborative multidimensional multimodal industrial robot precision measurement method, comprising: driving the end effector of the industrial robot to perform a test measurement on a workpiece; outputting target admittance control parameters based on six-dimensional force data using an ant colony-genetic hybrid optimization admittance control algorithm; and determining the initial starting path point pose and initial ending path point pose of the workpiece based on a bisection iterative optimization strategy; calibrating the pose based on the workpiece detection frame of the workpiece; and using an adaptive registration algorithm based on a double-line angle constraint according to a 2D workpiece image of a measured workpiece of the same model. The initial starting waypoint pose and initial ending waypoint pose are corrected to determine the target starting waypoint pose and target ending waypoint pose. The workpiece is measured according to the target admittance control parameters, the target starting waypoint pose, and the target ending waypoint pose. A segmented weighted ICP registration algorithm is used to determine the global transformation matrix between the real-time point cloud and the standard point cloud. When it is determined that the waypoint pose needs to be optimized according to the global transformation matrix, the poses of the unexecuted waypoints are transformed to update the target starting waypoint pose and / or the target ending waypoint pose, and the measurement continues until the workpiece measurement result is determined. Based on the above scheme, multi-dimensional and multi-source perception information such as vision, force feedback, and robotic arm end-effector pose is integrated. Through human-machine collaborative trial testing to plan waypoint pose, adaptive compensation for 2D variable-height workpiece path error, and 3D real-time path closed-loop compensation control, a multi-dimensional collaborative optimization measurement path is achieved. The algorithm fusion of ant colony-genetic hybrid optimization admittance control algorithm, adaptive registration algorithm based on double straight line angle constraint, and segmented weighted ICP registration algorithm effectively offsets interference factors such as sudden workpiece pose deviation and environmental disturbance during the measurement process. At the same time, it takes into account the real-time response speed of path compensation and the high accuracy of measurement results, thereby improving the reliability of precision measurement of industrial robots in complex industrial environments. Attached Figure Description

[0051] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0052] Figure 1 This is a flowchart illustrating the steps of a human-machine collaborative multidimensional multimodal industrial robot precision measurement method provided in Embodiment 1 of the present invention.

[0053] Figure 2 This is a schematic diagram of the six-dimensional force data provided in Embodiment 1 of the present invention;

[0054] Figure 3 This is a schematic diagram comparing the transformation of the workpiece detection frame provided in Embodiment 1 of the present invention;

[0055] Figure 4This is a structural block diagram of a human-machine collaborative multidimensional multimodal industrial robot precision measurement device provided in Embodiment 2 of the present invention. Detailed Implementation

[0056] This invention provides a human-machine collaborative multidimensional multimodal industrial robot precision measurement method and device to improve the reliability of precision measurement in industrial robots.

[0057] To make the objectives, features, and advantages of this invention more apparent and understandable, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described below are only some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0058] Please see Figure 1 The present invention provides a human-machine collaborative multidimensional multimodal industrial robot precision measurement method, comprising:

[0059] Step 101: Drive the end effector of the industrial robot to test the workpiece. The target admittance control parameters are output based on the six-dimensional force data using the ant colony-genetic hybrid optimization admittance control algorithm. The initial starting point pose and initial ending point pose of the workpiece are determined by combining the bisection iterative optimization strategy.

[0060] Test workpieces refer to workpieces used for trial operation measurements.

[0061] The ant colony-genetic hybrid optimization admittance control algorithm refers to a hybrid optimization algorithm that combines ant colony optimization and genetic algorithm. This algorithm is used to tune the admittance control parameters of the end effector of an industrial robot.

[0062] Six-dimensional force data refers to measuring forces in three directions in space. , , ) and torque in three directions ( , , ).

[0063] The target admittance control parameters refer to a set of optimal admittance control parameters obtained by the ant colony-genetic hybrid optimization admittance control algorithm. These parameters are used to achieve compliant control of the robotic arm end effector during actual workpiece measurement.

[0064] The bisection iterative optimization strategy refers to the strategy of gradually approaching the safe waypoint pose without collision by repeatedly taking the midpoint between the trial waypoint pose and the trial rebound pose after collision.

[0065] The starting waypoint pose refers to the pose of the 3D camera at the end of the robotic arm used for measurement at the beginning of the scanning measurement path. The ending waypoint pose refers to the pose of the 3D camera at the end of the robotic arm used for measurement at the end of the scanning measurement path. The initial starting waypoint pose and the initial ending waypoint pose refer to the starting waypoint pose and ending waypoint pose obtained after the initial waypoint pose planning is completed during the trial testing phase.

[0066] In one specific embodiment of this example, step 101 includes the following sub-steps:

[0067] S11. Drive the end of the industrial robot arm with external operating force to perform a test from the test start point pose to the test end point pose of the test workpiece based on the test run admittance control parameters, and collect six-dimensional force data and determine whether a collision rebound occurs through the six-dimensional force sensor at the end of the robot arm.

[0068] S12. Based on the ant colony algorithm, an ant colony is constructed using the admittance control parameters and the ant colony is updated by updating the pheromones. The ant colony output when the ant colony iteration stops is met is the initial population. The fitness is calculated using six-dimensional force data to execute the genetic operator. When the genetic iteration stops, the individual with the best fitness is output as the target admittance control parameter.

[0069] S13. When a collision rebound occurs, the test start point pose or test end point pose that is closer to the test rebound pose of the robotic arm end point is adopted. The geometric midpoint between the test rebound pose and the test end point pose is calculated as the new test start point pose or the new test end point pose. The target admittance control parameters are used to control the robotic arm end point to continue the test from the test start point pose to the test end point pose until no collision rebound occurs, then the initial start point pose and the initial end point pose are determined.

[0070] The starting and ending road point poses of the test refer to the starting and ending road point poses during the initial road point pose planning in the test phase.

[0071] The admittance control parameters during trial operation refer to the admittance control parameters to be optimized during the trial testing phase.

[0072] The test rebound pose refers to the actual pose of the robotic arm when it rebounds and stabilizes after a collision.

[0073] It should be noted that before the actual measurement of the workpiece begins, a trial run measurement is usually conducted to plan and optimize the trajectory to initially determine the workpiece's scanning path. Simultaneously, the compliant robot performs dynamic collision detection and triggers compliant rebound during the trial run to optimize the admittance control parameters. Specifically:

[0074] According to the preset test run admittance control parameters, the end effector of the industrial robot is controlled to move at a low speed (e.g., 15.0~30.0 mm / s) along the preset test start point pose and test end point pose to complete the test.

[0075] During the trial, on the one hand, six-dimensional force data was acquired in real time through a six-dimensional force sensor at the end of the robotic arm, and on the other hand, the ant colony-genetic hybrid optimization admittance control algorithm was used throughout the process.

[0076] First, we build an admittance control algorithm for the robot's end effector, essentially treating it as a spring-damped system. Based on the external operating force, we calculate the position, velocity, and acceleration of the end effector.

[0077] (1)

[0078] In the formula, Indicates interactive force. Indicates virtual quality. This represents the desired acceleration at the end effector of the robotic arm. This indicates the actual acceleration at the end effector of the robotic arm. Indicates virtual damping. This indicates the desired speed at the end effector of the robotic arm. This indicates the actual speed at the end effector of the robotic arm. Indicates virtual stiffness, Indicates the desired position of the robotic arm's end effector. This indicates the actual position of the robotic arm's end effector; to balance path dragging smoothness and pose adjustment accuracy, further differentiated parameter configurations for translation and rotation are adopted:

[0079] (2)

[0080] In the formula, Indicates translational mass. Indicates rotational mass. Indicates translational damping. Indicates rotational damping, Indicates translational stiffness. Indicates rotational stiffness;

[0081] Since the actual controller at the end of the robotic arm adopts a discrete sampling mechanism, it is impossible to directly run the continuous domain differential equation. Therefore, by using bilinear transformation, equation (1) is discretized to obtain the discrete domain recursive control formula:

[0082] (3)

[0083] In the formula, Indicates the index of discrete sampling time. Representing different discretization coefficients, Indicates the controller sampling period;

[0084] In equations (1)-(3), the values ​​of virtual mass, virtual damping and virtual stiffness determine the speed and sensitivity of the response. Therefore, the ant colony-genetic hybrid optimization admittance control algorithm is used to fit a set of optimal virtual mass, virtual damping and virtual stiffness.

[0085] Secondly, initialize the ant colony algorithm parameters and genetic algorithm parameters, and define the undetermined admittance control parameters as ant colony parameters. ( (represents transpose), and requires that maximum and minimum value parameter constraints be met to prevent severely biased results (i.e.) , and The ant colony is constructed using individual ants. The ant colony is iterated through ant path selection and pheromone updates until the ant colony reaches the iteration stopping condition (e.g., the maximum number of iterations is met). The initial population of the genetic algorithm is formed by the admittance control parameters of each group in the ant colony at this time.

[0086] Next, we define the weighted fitness function based on six-dimensional force data in the genetic algorithm: , Represents an individual in a genetic algorithm. Indicates fitness. Represents the expected interaction force sequence. This represents the actual interaction force sequence composed of six-dimensional force data. This represents the desired end-effector position sequence of the robotic arm. This represents the actual position sequence of the robotic arm's end effector. It represents the response settling time (reflecting responsiveness, i.e., a measure of how fast the admittance control response is, referring to the time required from the onset of an external force to stabilize near the desired value). This represents the Euclidean distance operation. Indicates different weighting coefficients. This represents the root mean square error. Indicates the sampling point index. This indicates the number of sampling points; the genetic iteration is performed by evaluating the quality of individuals based on fitness and executing genetic operators based on fitness. The iteration stops when the genetic conditions are met (e.g., the maximum number of iterations is reached, or the change in optimal fitness is less than a threshold). , Indicates the first The optimal fitness (i.e., minimum fitness) of the next iteration. Indicates the first The optimal fitness in the next iteration. This represents the absolute value operation. When the threshold value is changed, output the parameter combination of the individual with the optimal fitness at that time. That is, the target admittance control parameters (including the target virtual mass) required for the current scenario. Virtual damping of the target and target virtual stiffness );

[0087] It is understandable that the specific optimization and iteration processes of ant colony algorithms and genetic algorithms can be found in existing technologies and will not be elaborated here; while the six-dimensional force data used in genetic algorithms, in the preferred implementation, is the raw force data collected by a six-dimensional force sensor, which is processed and output as six-dimensional force data through mechanisms such as full-attitude adaptive gravity compensation, adaptive filtering and zero-point dynamic calibration. The specific process can be found in existing technologies.

[0088] During the test, on the other hand, based on the admittance control parameters, the human-machine interaction force can be converted into motion commands at the end of the robotic arm through the admittance control algorithm. After initially determining the starting and ending road point poses of the test workpiece, the end of the robotic arm is driven to move according to the road point pose under the external operating force of the operator through the set test run admittance control parameters. If the end of the robotic arm collides unexpectedly with the working environment during the process, the admittance control algorithm will drive the end of the robotic arm to achieve impact-free flexible rebound in the opposite direction of the collision. Simultaneously, the test rebound stroke and test rebound pose during the collision process are collected and fed back with high precision. A binary iterative optimization strategy is adopted to select the road point pose that is closer to the test rebound pose from the initially determined starting or ending road point poses, and the geometric midpoint of the corresponding test rebound pose is taken as the new candidate road point pose. The above test run-collision- The rebound-data acquisition process and iterative solution ultimately yield the initial starting road point pose and the initial ending road point pose, which will not cause collisions and meet the requirements of operational safety and positioning accuracy. These are then used as the formal operational road point poses, completing the dual calibration of the initial path and operational road point poses.

[0089] Step 102: Based on the workpiece detection frame calibration posture of the test workpiece, according to the 2D workpiece image of the actual test workpiece of the same model, the initial starting road point pose and the initial ending road point pose are corrected by the adaptive registration algorithm based on the double straight line angle constraint, and the target starting road point pose and the target ending road point pose are determined.

[0090] The workpiece inspection frame calibration posture refers to the reference posture of the workpiece inspection frame obtained by placing the workpiece in the field of view of the 2D camera with the desired orientation and position. The workpiece inspection frame is in the form of a rectangle, i.e., the workpiece rectangular frame, and the entire workpiece is within the frame.

[0091] Adaptive registration algorithm refers to an algorithm that uses the change in the angle between two corresponding feature lines between the workpiece detection frame of the test workpiece and the workpiece detection frame of the actual workpiece to correct and register the pose of the waypoint.

[0092] The target starting road point pose and the target ending road point pose refer to the test starting road point pose and test ending road point pose after 2D registration, which are used as the working poses for the 3D camera to perform actual scanning.

[0093] In one specific embodiment of this example, step 102 includes the following sub-steps:

[0094] S21. Based on the 2D image of the test workpiece, determine the coordinates of the corner points and the center point of the test workpiece detection frame.

[0095] S22. Determine the image transformation matrix of the robotic arm based on the test 2D workpiece image using the nine-point calibration method, and obtain the image transformation matrix of the robotic arm by inversion;

[0096] S23. Using the robotic arm image transformation matrix, the initial starting point pose and the initial ending point pose are converted into the test starting pixel coordinates and the test ending pixel coordinates.

[0097] S24. Calculate the distance between the center point coordinates of the test frame and the starting pixel coordinates and the ending pixel coordinates of the test, respectively;

[0098] S25. Determine the general equation of the starting line of the test test between the coordinates of the center point of the test test frame and the coordinates of the starting pixel of the test test, and determine the general equation of the ending line of the test test between the coordinates of the center point of the test test frame and the coordinates of the ending pixel of the test test.

[0099] S26. Determine the general equation of the test horizontal line of the test workpiece detection frame in the horizontal direction and the general equation of the test vertical line of the test vertical line in the vertical direction; wherein, the line direction of the test start pixel coordinates and the test end pixel coordinates is the same as the horizontal direction of the test workpiece detection frame.

[0100] S27. Calculate the included angle of the initial line of the test based on the general equation of the initial line and the general equation of the longitudinal line of the test.

[0101] S28. Calculate the included angle between the lines at the test endpoint and the longitudinal line based on the general equation of the line at the test endpoint and the general equation of the longitudinal line.

[0102] S29. Based on the measured 2D workpiece image of the actual workpiece of the same model as the test workpiece, determine the coordinates of the center point of the measured frame and the general equation of the measured horizontal straight line.

[0103] S210. Calculate the angle between the transverse transformation lines using the general equation of the measured transverse line and the general equation of the trial transverse line.

[0104] S211. Construct the starting transformation angle by combining the angle between the starting line and the horizontal transformation line, and construct the ending transformation angle by combining the angle between the ending line and the horizontal transformation line.

[0105] S212. Based on the measured frame center point coordinates, starting transformation angle, starting center point distance, image robot arm transformation matrix and workpiece standard height, determine the target starting path point pose.

[0106] S213. Based on the measured center point coordinates, endpoint transformation angle, endpoint center point distance, image robot transformation matrix, and workpiece standard height, determine the target endpoint waypoint pose.

[0107] 2D workpiece image refers to the workpiece image acquired by a 2D camera; corner coordinates refer to the coordinates of the four corner points of the workpiece detection frame; center coordinates refer to the coordinates of the center point of the workpiece detection frame; starting pixel coordinates refer to the coordinates of the starting path point pose in the pixel coordinate system; ending pixel coordinates refer to the coordinates of the ending path point pose in the pixel coordinate system; general equation of the starting line refers to the general equation of the line passing through the center point of the workpiece detection frame and the starting path point pose; general equation of the ending line refers to the general equation of the line passing through the center point of the workpiece detection frame and the ending path point pose; general equation of the transverse line refers to the general equation of the line passing through the center point of the workpiece detection frame and parallel to the rectangular side of the workpiece detection frame in the transverse direction (i.e., the centerline of the workpiece detection frame in the transverse direction); general equation of the longitudinal line refers to the general equation of the line passing through the center point of the workpiece detection frame and parallel to the rectangular side of the workpiece detection frame in the longitudinal direction (the centerline of the workpiece detection frame in the longitudinal direction). It can be understood that the "transverse direction" and "longitudinal direction" of the workpiece detection frame are perpendicular to each other. Among them, "trial test" refers to the workpiece being tested, while "actual test" refers to the workpiece being measured.

[0108] The image transformation matrix of a robotic arm refers to the transformation matrix that transforms the pixel coordinate system of the workpiece image to the robotic arm coordinate system at the end of the robotic arm; the image transformation matrix of a robotic arm refers to the transformation matrix that transforms the robotic arm coordinate system at the end of the robotic arm to the pixel coordinate system of the workpiece image.

[0109] The starting center point distance refers to the Euclidean distance between the starting pixel coordinates and the center point coordinates of the bounding box; the ending center point distance refers to the Euclidean distance between the ending pixel coordinates and the center point coordinates of the bounding box.

[0110] The included angle of the starting straight line in the test refers to the angle between the straight line passing through the center point and the starting path point pose of the workpiece inspection frame and the straight line passing through the center point of the workpiece inspection frame and parallel to the rectangular side of the workpiece inspection frame in the longitudinal direction; the included angle of the ending straight line in the test refers to the angle between the straight line passing through the center point and the ending path point pose of the workpiece inspection frame and the straight line passing through the center point of the workpiece inspection frame and parallel to the rectangular side of the workpiece inspection frame in the longitudinal direction.

[0111] The lateral transformation line angle refers to the straight line angle between the centerline of the test workpiece in the lateral direction and the centerline of the actual measured workpiece in the lateral direction.

[0112] The starting transformation angle refers to the angle after the starting pixel coordinates of the test are rotated and transformed around the center point of the frame in the measured 2D workpiece image; the ending transformation angle refers to the angle after the ending pixel coordinates of the test are rotated and transformed around the center point of the frame in the measured 2D workpiece image.

[0113] The standard height of a workpiece refers to the standard height value of that model of workpiece, which is equivalent to the Z-axis coordinate of the scanning plane.

[0114] It should be noted that for workpieces of the same model whose only height parameter fluctuates, existing measurement solutions still require offline programming teaching in batches, resulting in low path switching efficiency and poor teaching repeatability. This makes it difficult to support the adaptive compensation requirements of teaching paths in human-machine collaborative scenarios. Therefore, this embodiment uses an adaptive registration algorithm based on double-straight-line angle constraints to solve the deviation between the initial calibration posture of the workpiece detection frame of the test workpiece and the actual posture of the workpiece detection frame of the measured workpiece. Then, the posture deviation is corrected and the position coordinates are adapted synchronously for the poses of the points in the initial teaching path to achieve efficient adaptive execution of the detection task under uncertain working conditions. Specifically:

[0115] During the calibration process of the test workpiece:

[0116] First, a 2D image of the test workpiece is acquired using a 2D camera. Then, a visual inspection algorithm identifies the coordinates of the four corner points of the test workpiece's detection frame in the image. Figure 3 (a) , , and The coordinates; the transformation matrix of the robotic arm is obtained by the nine-point calibration method in 2D. ,right Inverse transformation matrix of robotic arm image obtained by inversion This allows the initial starting point pose of the robotic arm's end effector to be determined. and initial and final destination pose Convert to pixel coordinates, such as Figure 3The starting and ending pixel coordinates of the test in (a) are shown in the figure.

[0117] Secondly, calculate the center point of the test frame. arrive Distance: In the formula, Indicates the coordinates of the center point of the test frame. Indicates the starting pixel coordinates of the test. This represents the distance from the starting center point. Similarly, the center point of the test frame is calculated. arrive Distance from the center point of the endpoint ;

[0118] Next, the centerline of the test workpiece inspection frame in the lateral direction is calculated. slope , Indicates the corner of the test frame coordinates Indicates the corner of the test frame The coordinates of, where, and The direction of the straight line and Parallel, in this embodiment, the lateral direction of the workpiece detection frame is defined as parallel to... and The straight lines between them are in the same direction; similarly, calculate the centerline of the test workpiece inspection frame in the longitudinal direction. The slope, and straight line The slope and the passage and straight line The slope; based on the corresponding slope, the general equation of the line can be determined, for example, the line The general equation, i.e., the general equation of the longitudinal straight line in the test, is: ,straight line The general form of the equation, i.e., the general form of the initial straight line in the trial test, is: , Represents the X-axis coordinate. Represents the Y-axis coordinate. This represents the coefficient of the X-axis coordinate in the general equation of the longitudinal straight line. This represents the coefficient of the Y-axis coordinate in the general equation of the longitudinal straight line. This represents the constant term in the general equation of the longitudinal straight line being tested. This represents the coefficient of the X-axis coordinate in the general form of the equation of the initial straight line in the trial test. This represents the coefficient of the Y-axis coordinate in the general form of the equation of the initial straight line used in the trial test. This represents the constant term in the general form equation of the initial straight line used for the trial test.

[0119] Furthermore, according to and Calculate the angle between two lines using the general equation of the straight line, i.e., measure the angle between the initial straight lines. :

[0120] ;

[0121] Similarly, according to and Calculate the angle between the two lines using the general equation of the straight line, i.e., the angle between the lines at the test endpoint. ;

[0122] During the 2D registration process of the actual workpiece:

[0123] First, a 2D image of the actual workpiece is acquired using a 2D camera. The workpiece model identifier is extracted from the image, and then the corresponding standard workpiece height can be directly retrieved based on a pre-defined workpiece model-geometric height mapping database. Since the height during scanning is fixed, there is no need to correct the Z-axis coordinates of the scanning points. Because the scanning trajectory also needs to be adjusted after the target object changes its pose, the relative pose of the target object and the scanning trajectory can be considered fixed. Figure 3 As shown in (b) in the figure, we can obtain , , as well as The process of determining the coordinates after the transformation and the corresponding general equation of the straight line can be referred to the calibration process of the test workpiece, and will not be repeated here.

[0124] Secondly, compare the transformed straight lines. and straight line The angle between the lines that change laterally can be calculated. ,and At this point, the angle between the vertical lines is transformed. Measured angle of the starting straight line Angle with the measured endpoint straight line The angle after the workpiece rotates around the center point of the frame can be calculated; due to the issue of clockwise and counterclockwise rotation, when it is clockwise rotation, the initial transformation angle is... and the endpoint transformation angle not When it is a reversal, the opposite is true;

[0125] At this point, the measured starting pixel coordinates of the workpiece after the transformation are determined based on the coordinates of the center point of the measured frame, the starting transformation angle, and the distance from the starting center point: , , Indicates the coordinates of the center point of the measured frame. This represents the absolute value operation. This represents the measured starting pixel coordinates; after transforming the measured starting pixel coordinates using the image robot arm transformation matrix, the workpiece standard height is used as the measured Z-axis coordinate. The measured initial attitude angle (i.e., the attitude angle of the initial starting point pose is constructed by adding the initial transformation angle to the initial starting point pose.) , and , This represents the rotation angle around the Z-axis in the initial waypoint pose. This represents the rotation angle around the Z-axis in the initial pose of the target waypoint. This represents the rotation angle around the Y-axis in the initial waypoint pose. This represents the rotation angle around the Y-axis in the initial waypoint pose of the target. This represents the rotation angle around the X-axis in the initial waypoint pose. This represents the rotation angle around the X-axis in the target starting waypoint pose, thus forming the target starting waypoint pose; similarly, the measured end point pixel coordinates can also be determined based on the coordinates of the measured frame center point, the end point transformation angle, and the distance to the end point center point. After transforming the measured endpoint pixel coordinates through the image robot transformation matrix, the standard height of the workpiece is used as the measured Z-axis coordinate. The attitude angle of the initial endpoint waypoint pose is added to the endpoint transformation angle to construct the measured endpoint attitude angle, and the target endpoint waypoint pose is formed, thereby completing the waypoint correction.

[0126] Step 103: Perform actual measurements on the workpiece according to the target admittance control parameters, the target starting road point pose, and the target ending road point pose. Use a segmented weighted ICP registration algorithm to determine the global transformation matrix between the real-time point cloud and the standard point cloud. When it is determined that the road point pose needs to be optimized according to the global transformation matrix, transform the unexecuted road point pose to update the target starting road point pose and / or the target ending road point pose and continue the measurement until the workpiece measurement result is determined.

[0127] The segmented weighted ICP registration algorithm refers to a weighted ICP registration algorithm that integrates multi-resolution segmented iterative registration.

[0128] Real-time point cloud refers to point cloud data collected in real time; standard point cloud refers to point cloud data used as a reference benchmark.

[0129] The global transformation matrix refers to the rigid body transformation matrix from the real-time point cloud coordinate system to the standard point cloud coordinate system.

[0130] Unexecuted waypoint poses refer to the poses of subsequent waypoints that have not yet been moved to by the end effector of the robotic arm during the current measurement process.

[0131] Workpiece measurement results refer to the measurement results of the workpiece's appearance and dimensions.

[0132] In one specific embodiment of this example, step 103 includes the following sub-steps:

[0133] S31. The robotic arm end effector is driven to move from the target starting point pose to the target ending point pose according to the target admittance control parameters, and the real-time point cloud is collected.

[0134] S32. Calculate the centroid of the standard point cloud and the real-time point cloud of the test workpiece respectively, and determine the centroid of the standard point cloud and the real-time point cloud.

[0135] S33. Construct an aligned standard point cloud by subtracting the centroid of the standard point cloud from the standard point cloud, and construct an aligned real-time point cloud by subtracting the centroid of the real-time point cloud from the real-time point cloud.

[0136] S34. Output a coarse registration transformation matrix based on the aligned standard point cloud and the aligned real-time point cloud using the point-to-surface ICP algorithm;

[0137] S35. The real-time point cloud is aligned by transforming the coarse registration transformation matrix to obtain the coarse registration real-time point cloud. The aligned standard point cloud and the coarse registration real-time point cloud are downsampled to a low resolution to obtain the low-level standard point cloud and the low-level real-time point cloud. The low-level transformation matrix is ​​output by using the weighted ICP algorithm with the goal of minimizing the weighted error function.

[0138] S36. Transform the low-level real-time point cloud using the low-level transformation matrix to obtain the low-level transformed real-time point cloud. Downsample the low-level standard point cloud and the low-level transformed real-time point cloud to the next higher resolution to obtain the high-level standard point cloud and the high-level real-time point cloud. Output the high-level transformation matrix using the weighted ICP algorithm with the goal of minimizing the weighted error function.

[0139] S37. If the current resolution is not the highest resolution and the high-level transformation matrix does not meet the convergence condition, the high-level standard point cloud and the high-level real-time point cloud are transformed by the high-level transformation matrix and then downsampled to the next higher resolution to obtain a new high-level standard point cloud and a new high-level real-time point cloud. The new high-level transformation matrix is ​​output through the weighted ICP algorithm until the current resolution is the highest resolution or the high-level transformation matrix meets the convergence condition. Then, the coarse registration transformation matrix and the high-level transformation matrix are used to synthesize the global transformation matrix.

[0140] S38. If the global transformation matrix analysis determines that the waypoint pose needs to be optimized, the unexecuted waypoint pose is transformed by the global transformation matrix to obtain a new target starting waypoint pose and / or a new target ending waypoint pose. The robotic arm end-effector is then driven to continue moving towards the new target ending waypoint pose according to the target admittance control parameters until the workpiece measurement result is determined.

[0141] It should be noted that during the measurement execution phase, this embodiment continuously tracks the real-time point cloud acquired by the 3D camera, uses the measurement path after 2D adaptive compensation correction as the motion reference, and employs a segmented weighted ICP registration algorithm based on center alignment to perform point cloud registration calculation. This allows for real-time dynamic correction of the measurement path, thereby accurately locking the target measurement area of ​​the workpiece, completing high-quality and complete 3D point cloud data acquisition, and finally outputting high-precision workpiece appearance and dimensional measurement results. Specifically:

[0142] First, the centroids of the real-time point cloud and the standard point cloud are calculated and the point cloud centers are aligned:

[0143] ;

[0144] In the formula, Indicates the real-time centroid of the point cloud. This represents the point index in the real-time point cloud. Indicates real-time point cloud, This indicates the number of points in the real-time point cloud. Indicates the centroid of the standard point cloud. This represents the point index in a standard point cloud. Represents a standard point cloud. This indicates the number of points in a standard point cloud. This indicates alignment with the real-time point cloud. Indicates alignment with standard point clouds;

[0145] Secondly, the coarse registration transformation matrix is ​​obtained using the point-to-surface ICP registration algorithm:

[0146] ;

[0147] In the formula, The transformation matrix representing the resolution of layer 0 is the coarse registration transformation matrix. Represents the point cloud transformation matrix. This represents the aligned real-time point cloud at resolution level 0. This indicates that points are projected onto a local plane aligned with a standard point cloud. This represents the aligned standard point cloud at resolution level 0. This represents the alignment standard point cloud at resolution level 0, with respect to points... The corresponding small plane containing the matching point Represents the square of the Euclidean norm;

[0148] After coarse registration is completed, Transform to Under coordinates: , This indicates coarse registration of real-time point clouds;

[0149] Next, a multi-resolution point cloud pyramid is constructed to refine the point cloud resolution layer by layer from low to high, including at least two resolution spaces. A piecewise weighted ICP iterative optimization of the transformation matrix is ​​employed: the aligned standard point cloud and the coarsely registered real-time point cloud are downsampled to a low resolution to obtain a low-level standard point cloud and a low-level real-time point cloud, respectively. These are then registered using a weighted ICP algorithm based on the constructed weighted error function, outputting a low-level transformation matrix. The low-level real-time point cloud is then transformed using the low-level transformation matrix to obtain a low-level transformed real-time point cloud. The low-level standard point cloud and the low-level transformed real-time point cloud are then downsampled to a high resolution to obtain a high-level standard point cloud and a high-level real-time point cloud. These are then registered using a weighted ICP algorithm, outputting a high-level transformation matrix. When the iteration reaches the highest resolution level or the output high-level transformation matrix satisfies the convergence condition, the high-level transformation matrix at this point is multiplied by the coarsely registered transformation matrix to synthesize the global transformation matrix.

[0150] It is understandable that a multi-resolution point cloud pyramid can include three or more resolution spaces, meaning that there can be multiple high-resolution layers between the low resolution and the highest resolution. Therefore, when the current resolution is not the highest resolution and the high-level transformation matrix does not meet the convergence condition, the high-level transformation matrix output by the previous resolution can continue to be used for point cloud transformation and downsampling to the next higher resolution. The weighted ICP algorithm is used for registration operation in a loop iteration.

[0151] In one specific implementation of this embodiment, the weighted error function includes:

[0152] ;

[0153] In the formula, Indicates the resolution level index. Indicates the first Transformation matrix of layer resolution, This represents the weighted error function. Indicates the first Alignment of real-time point clouds at layer resolution Point The weight, Indicates the first Alignment standard point cloud at layer resolution, Represented as a point exist The nearest point in;

[0154] Furthermore, the global transformation matrix determines whether the currently planned waypoint pose needs to be optimized. If so, the current motion trajectory of the robotic arm end effector is paused. Starting from the current pose of the robotic arm end effector, the global transformation matrix is ​​used to transform the poses of the subsequent unexecuted waypoints. Finally, the robotic arm end effector is controlled to start from the current pose of the robotic arm end effector and continue to execute the motion task along the corrected waypoint pose sequence, thereby completing the measurement.

[0155] In a more specific implementation of this embodiment, if the analysis of the global transformation matrix requires optimization of waypoint pose, it includes:

[0156] S3801. Decompose the rotation matrix and translation vector from the global transformation matrix, and calculate the corresponding rotation angle and translation distance;

[0157] S3802. When the rotation angle or translation distance is greater than the corresponding threshold, it is determined that the waypoint pose needs to be optimized.

[0158] It should be noted that, in one implementation, the rotation matrix and translation vector can be decomposed from the global transformation matrix, and the rotation angle and translation distance can be calculated:

[0159] ;

[0160] In the formula, This represents the X-axis component of the translation vector. This represents the Y-axis component of the translation vector. This represents the Z-axis component of the translation vector. Indicates the translation distance. Indicates the rotation angle. Represents trace operation. Represents the rotation matrix. Represents the inverse cosine function;

[0161] Waypoint pose optimization will only be performed when one of the parameters, rotation angle or translation distance, is greater than the angle threshold or translation threshold.

[0162] In one specific embodiment of this example, it further includes:

[0163] If a collision and rebound occur during the actual measurement, the target starting road point pose or the target ending road point pose, which is closer to the measured rebound pose of the robotic arm end point, is used. The geometric midpoint between the target starting road point pose and the measured rebound pose is calculated as the new test starting road point pose or the new test ending road point pose. The target admittance control parameters are used to control the robotic arm end point to test from the test starting road point pose to the test ending road point pose until no collision and rebound occur. Then, the initial starting road point pose and the initial ending road point pose are determined, and the target starting road point pose and the target ending road point pose are re-determined until the workpiece measurement result is determined.

[0164] It should be noted that during the actual test, the robotic arm end effector will enter a normal high-speed operation phase (e.g., 0.5~1.5m / s). In the event of a sudden collision, the robotic arm end effector will immediately perform an emergency stop and quickly reverse to avoid the collision, fully ensuring the safety of the equipment hardware and on-site personnel. Simultaneously, the measured rebound pose and other collision parameters will be recorded, and local iterative optimization will be initiated. That is, the same bisection iterative optimization strategy as in the trial test phase will be adopted to redetermine the initial starting point pose and the initial ending point pose after adapting to the changed environment, and 2D correction and 3D correction will continue until the measurement is completed.

[0165] In this embodiment of the invention, multi-dimensional and multi-source perception information such as vision, force feedback, and robotic arm end-effector pose is integrated. Through multi-dimensional collaborative optimization of the measurement path, including human-machine collaborative trial testing of waypoint pose, adaptive compensation for 2D variable-height workpiece path error, and 3D real-time path closed-loop compensation control, a hybrid algorithm of ant colony-genetic optimization admittance control, adaptive registration algorithm based on double-straight-line angle constraint, and segmented weighted ICP registration algorithm are used to accurately convert human-machine interaction forces into robot end-effector motion commands. This enables safe interactive operation between humans and collaborative robots, effectively offsetting interference factors such as sudden workpiece pose deviation and environmental disturbances during the measurement process. At the same time, it takes into account the real-time response speed of path compensation and the high accuracy of measurement results, achieving synergistic optimization of human-machine collaborative safety and work efficiency, and significantly improving the reliability of industrial robot precision measurement in complex industrial environments.

[0166] Please see Figure 4 The second embodiment of the present invention provides a human-machine collaborative multidimensional multimodal industrial robot precision measurement device, comprising:

[0167] The waypoint testing module 401 is used to drive the end effector of the industrial robot to test the workpiece. It outputs the target admittance control parameters based on six-dimensional force data through the ant colony-genetic hybrid optimization admittance control algorithm, and determines the initial starting waypoint pose and initial ending waypoint pose of the workpiece by combining the bisection iterative optimization strategy.

[0168] The waypoint correction module 402 is used to calibrate the posture of the workpiece detection frame based on the test workpiece. According to the 2D workpiece image of the actual test workpiece of the same model, the adaptive registration algorithm based on the double straight line angle constraint is used to correct the initial starting waypoint pose and the initial ending waypoint pose, and determine the target starting waypoint pose and the target ending waypoint pose.

[0169] The workpiece measurement module 403 is used to measure the workpiece according to the target admittance control parameters, the target starting road point pose, and the target ending road point pose. It uses a segmented weighted ICP registration algorithm to determine the global transformation matrix between the real-time point cloud and the standard point cloud. When it is determined that the road point pose needs to be optimized according to the global transformation matrix, it transforms the pose of the unexecuted road points to update the target starting road point pose and / or the target ending road point pose and continues the measurement until the workpiece measurement result is determined.

[0170] In one specific embodiment of this example, a measured collision adjustment module is further included, used for:

[0171] If a collision and rebound occur during the actual measurement, the target starting road point pose or the target ending road point pose, which is closer to the measured rebound pose of the robotic arm end point, is used. The geometric midpoint between the target starting road point pose and the measured rebound pose is calculated as the new test starting road point pose or the new test ending road point pose. The robotic arm end point is then controlled to test from the test starting road point pose to the test ending road point pose using the target admittance control parameters until no collision and rebound occur. Then, the initial starting road point pose and the initial ending road point pose are determined, and the target starting road point pose and the target ending road point pose are re-determined until the workpiece measurement result is determined.

[0172] Embodiment 3 of the present invention also provides a computer device, including a memory and a processor, wherein the memory stores a computer program; when the computer program is executed by the processor, the processor performs the steps of the human-machine collaborative multidimensional multimodal industrial robot precision measurement method as described in Embodiment 1 of the present invention.

[0173] Embodiment 4 of the present invention also provides a computer-readable storage medium storing a computer program / instruction thereon, which, when executed by a processor, implements the steps of the human-machine collaborative multidimensional multimodal industrial robot precision measurement method described in Embodiment 1 of the present invention.

[0174] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the above-described device and module can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0175] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or modules may be electrical, mechanical, or other forms.

[0176] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0177] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0178] If the integrated module is implemented as a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0179] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A human-machine collaborative multidimensional multimodal industrial robot precision measurement method, characterized in that, include: The end effector of the robotic arm of the industrial robot performs a test on the workpiece. The target admittance control parameters are output based on six-dimensional force data using an ant colony-genetic hybrid optimization admittance control algorithm. The initial starting point pose and initial ending point pose of the workpiece are determined by combining the bisection iterative optimization strategy. Based on the workpiece detection frame calibration posture of the test workpiece, and according to the 2D workpiece image of the actual test workpiece of the same model, the initial starting road point pose and the initial ending road point pose are corrected by an adaptive registration algorithm based on the double straight line angle constraint, and the target starting road point pose and the target ending road point pose are determined. The measured workpiece is measured according to the target admittance control parameters, the target starting waypoint pose, and the target ending waypoint pose. A segmented weighted ICP registration algorithm is used to determine the global transformation matrix between the real-time point cloud and the standard point cloud. When it is determined that the waypoint pose needs to be optimized according to the global transformation matrix, the unexecuted waypoint pose is transformed to update the target starting waypoint pose and / or the target ending waypoint pose, and the measurement continues until the workpiece measurement result is determined.

2. The precision measurement method for human-machine collaborative multidimensional multimodal industrial robots according to claim 1, characterized in that, The end effector of the robotic arm driving the industrial robot performs a test on the workpiece. Based on six-dimensional force data, a hybrid ant colony-genetic optimization admittance control algorithm is used to output target admittance control parameters. Combined with a bisection iterative optimization strategy, the initial measurement waypoint of the workpiece is determined, including: The robotic arm end effector of the industrial robot is driven by external operating force to perform a test from the test start point pose to the test end point pose of the test workpiece based on the test run admittance control parameters, and the six-dimensional force sensor at the end effector of the robotic arm collects six-dimensional force data and determines whether a collision rebound occurs. Ant colony building is based on ant colony algorithm, which uses admittance control parameters to build ant colonies and updates ant colonies by updating pheromones. The ant colony output by the genetic algorithm when the ant colony iteration stops is met is the initial population. The fitness is calculated using six-dimensional force data to execute the genetic operator. When the genetic iteration stops, the individual with the best fitness is output as the target admittance control parameter. When a collision rebound occurs, the starting or ending road point pose of the test is adopted, which is closer to the test rebound pose of the robotic arm end point. The geometric midpoint between the starting and ending road point poses is calculated as the new starting or ending road point pose. The robotic arm end point is then controlled by the target admittance control parameters to continue testing from the starting road point pose to the ending road point pose until no collision rebound occurs, at which point the initial starting and ending road point poses are determined.

3. The precision measurement method for human-machine collaborative multidimensional multimodal industrial robots according to claim 1, characterized in that, The calibration posture of the workpiece detection frame based on the test workpiece, and the correction of the initial starting road point pose and the initial ending road point pose using an adaptive registration algorithm based on the 2D workpiece image of the actual test workpiece of the same model, to determine the target starting road point pose and the target ending road point pose, include: Based on the 2D image of the test workpiece, determine the coordinates of the corner points and the center point of the test workpiece detection frame. The image transformation matrix of the robotic arm is determined based on the test 2D workpiece image using the nine-point calibration method, and the inverse is used to obtain the image transformation matrix of the robotic arm. The initial starting point pose and the initial ending point pose are converted into the test start pixel coordinates and test end pixel coordinates using the robotic arm image transformation matrix. Calculate the distance between the center point coordinates of the test frame and the starting pixel coordinates and the ending pixel coordinates of the test, respectively, and the distance between the starting center point and the ending center point. Determine the general equation of the starting line of the test test between the coordinates of the center point of the test frame and the coordinates of the starting pixel of the test test, and determine the general equation of the ending line of the test test between the coordinates of the center point of the test frame and the coordinates of the ending pixel of the test test; Determine the general equation of the test workpiece detection frame for the transverse straight line at the centerline in the transverse direction and the general equation of the test longitudinal straight line at the centerline in the longitudinal direction; wherein the straight line direction of the test start pixel coordinates and the test end pixel coordinates is the same as the transverse direction of the test workpiece detection frame. Calculate the included angle of the initial test line based on the general equation of the initial test line and the general equation of the longitudinal test line; Calculate the included angle between the test endpoint lines based on the general equation of the test endpoint line and the general equation of the test longitudinal line. Based on the measured 2D workpiece image of the actual workpiece of the same model as the test workpiece, the coordinates of the center point of the measured frame and the general equation of the measured horizontal straight line are determined. The angle between the transverse transformation lines is calculated using the general equation of the measured transverse line and the general equation of the test transverse line. The starting angle is constructed by the angle between the starting line and the lateral transformation line, and the ending angle is constructed by the angle between the ending line and the lateral transformation line. Based on the measured frame center point coordinates, the starting transformation angle, the starting center point distance, the image robot transformation matrix, and the workpiece standard height, the target starting waypoint pose is determined; Based on the coordinates of the center point of the measured frame, the endpoint transformation angle, the distance to the center point of the endpoint, the image robot transformation matrix, and the standard height of the workpiece, the pose of the target endpoint waypoint is determined.

4. The precision measurement method for human-machine collaborative multidimensional multimodal industrial robots according to claim 1, characterized in that, The measured workpiece is measured according to the target admittance control parameters, the target starting waypoint pose, and the target ending waypoint pose. A segmented weighted ICP registration algorithm is used to determine the global transformation matrix between the real-time point cloud and the standard point cloud. When it is determined that the waypoint pose needs to be optimized based on the global transformation matrix, the poses of the unexecuted waypoints are transformed to update the target starting waypoint pose and / or the target ending waypoint pose, and the measurement continues until the workpiece measurement result is determined, including: The robotic arm end effector was driven to move from the target starting point pose to the target ending point pose according to the target admittance control parameters, and real-time point cloud data was collected. The centroids of the standard point cloud and the real-time point cloud of the test workpiece are calculated respectively to determine the centroids of the standard point cloud and the real-time point cloud. An aligned standard point cloud is constructed by subtracting the centroid of the standard point cloud from the standard point cloud, and an aligned real-time point cloud is constructed by subtracting the centroid of the real-time point cloud from the real-time point cloud. The coarse registration transformation matrix is ​​output based on the alignment standard point cloud and the alignment real-time point cloud using the point-to-surface ICP algorithm. The coarse registration real-time point cloud is obtained by transforming the coarse registration transformation matrix. The alignment standard point cloud and the coarse registration real-time point cloud are downsampled to a low resolution to obtain a low-level standard point cloud and a low-level real-time point cloud. The low-level transformation matrix is ​​output by using the weighted ICP algorithm with the goal of minimizing the weighted error function. The low-level real-time point cloud is transformed by the low-level transformation matrix to obtain the low-level transformed real-time point cloud. The low-level standard point cloud and the low-level transformed real-time point cloud are downsampled to the next higher resolution to obtain the high-level standard point cloud and the high-level real-time point cloud. The high-level transformation matrix is ​​output by using the weighted ICP algorithm with the goal of minimizing the weighted error function. If the current resolution is not the highest resolution and the high-level transformation matrix does not meet the convergence condition, the high-level standard point cloud and the high-level real-time point cloud are transformed by the high-level transformation matrix and then downsampled to the next higher resolution to obtain a new high-level standard point cloud and a new high-level real-time point cloud. A new high-level transformation matrix is ​​output through the weighted ICP algorithm until the current resolution is the highest resolution or the high-level transformation matrix meets the convergence condition. Then, the coarse registration transformation matrix and the high-level transformation matrix are used to synthesize the global transformation matrix. If the global transformation matrix analysis determines that waypoint pose optimization is needed, the unexecuted waypoint poses are transformed using the global transformation matrix to obtain a new target starting waypoint pose and / or a new target ending waypoint pose. The robotic arm end effector is then driven to continue moving towards the new target ending waypoint pose according to the target admittance control parameters until the workpiece measurement result is determined.

5. The precision measurement method for human-machine collaborative multidimensional multimodal industrial robots according to claim 1, characterized in that, Also includes: If a collision and rebound occur during the actual measurement, the target starting road point pose or the target ending road point pose, which is closer to the measured rebound pose of the robotic arm end point, is used. The geometric midpoint between the target starting road point pose and the measured rebound pose is calculated as the new test starting road point pose or the new test ending road point pose. The robotic arm end point is then controlled to test from the test starting road point pose to the test ending road point pose using the target admittance control parameters until no collision and rebound occur. Then, the initial starting road point pose and the initial ending road point pose are determined, and the target starting road point pose and the target ending road point pose are re-determined until the workpiece measurement result is determined.

6. The precision measurement method for human-machine collaborative multidimensional multimodal industrial robots according to claim 4, characterized in that, If the global transformation matrix needs to be analyzed to determine if waypoint pose optimization is required, the following steps are included: Decompose the rotation matrix and translation vector from the global transformation matrix, and calculate the corresponding rotation angle and translation distance; When the rotation angle or the translation distance is greater than the corresponding threshold, it is determined that the waypoint pose needs to be optimized.

7. The precision measurement method for human-machine collaborative multidimensional multimodal industrial robots according to claim 2, characterized in that, The fitness calculation process includes: ； In the formula, Represents an individual in a genetic algorithm. Indicates fitness. Represents the expected interaction force sequence. This represents the actual interaction force sequence composed of six-dimensional force data. This represents the desired end-effector position sequence of the robotic arm. This represents the actual position sequence of the robotic arm's end effector. Indicates the response adjustment time. This represents the Euclidean distance operation. Indicates different weighting coefficients. This represents the root mean square error.

8. A human-machine collaborative multidimensional multimodal industrial robot precision measuring device, characterized in that, The apparatus is used to implement the human-machine collaborative multidimensional multimodal industrial robot precision measurement method as described in any one of claims 1-7, the apparatus comprising: The waypoint testing module is used to drive the end effector of the industrial robot to test the workpiece. It outputs target admittance control parameters based on six-dimensional force data through an ant colony-genetic hybrid optimization admittance control algorithm, and determines the initial starting waypoint pose and initial ending waypoint pose of the workpiece by combining a bisection iterative optimization strategy. The waypoint correction module is used to calibrate the posture of the workpiece detection frame based on the test workpiece, and to correct the initial starting waypoint pose and the initial ending waypoint pose using an adaptive registration algorithm based on the double straight line angle constraint according to the 2D workpiece image of the actual test workpiece of the same model, so as to determine the target starting waypoint pose and the target ending waypoint pose. The workpiece measurement module is used to perform actual measurements on the workpiece according to the target admittance control parameters, the target starting road point pose, and the target ending road point pose. It uses a segmented weighted ICP registration algorithm to determine the global transformation matrix between the real-time point cloud and the standard point cloud. When it is determined that the road point pose needs to be optimized according to the global transformation matrix, it transforms the poses of the unexecuted road points to update the target starting road point pose and / or the target ending road point pose and continues the measurement until the workpiece measurement result is determined.

9. A computer device, characterized in that, The system includes a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor causes the processor to perform the steps of the human-machine collaborative multidimensional multimodal industrial robot precision measurement method as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program / instructions stored thereon, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the human-machine collaborative multidimensional multimodal industrial robot precision measurement method as described in any one of claims 1-7.

Images