Hand-eye calibration method, device and system based on genetic algorithm

Abstract

The embodiment of the invention provides a hand-eye calibration method, device and system based on a genetic algorithm. The method comprises the steps that point cloud data collected by a line structure light sensor are acquired for multiple times, and corresponding feature points are determined; determining the coordinate of each feature point in a flange plate coordinate system and the coordinate of each feature point in a sensor coordinate system; determining an initial value of a hand-eye calibration matrix through an ICP algorithm based on the coordinates of the plurality of feature points in the flange plate coordinate system and the coordinates of the plurality of feature points in the sensor coordinate system; and taking the initial value of the hand-eye calibration matrix as a priori solution of a genetic algorithm, and completing the solution of the optimal hand-eye calibration matrix through multiple iterations. According to the method, the calculation efficiency is guaranteed, high-precision global optimization is achieved, then it is ensured that the robot system completes complex operation tasks based on the accurate coordinate conversion relation, and the production quality and efficiency are improved.

Description

Technical Field

[0001] The present disclosure relates to the technical field of robot vision calibration, and in particular to a hand-eye calibration method, device, and system based on a genetic algorithm. Background Art

[0002] Amid the rapid development of intelligent and robotic technologies, structured light sensors are increasingly being used in robotic vision applications due to their non-contact measurement and high precision. For example, a 3D scanning platform formed by these sensors and welding robots can measure the entire workpiece, playing a key role in scenarios such as weld seam tracking and control, adaptive welding, and part remanufacturing.

[0003] However, converting the surface coordinate data collected by the line structured light sensor into global three-dimensional data in the robot coordinate system is a key prerequisite for the robot to achieve precise operation. This process relies on hand-eye calibration of the relationship between the robot flange and the sensor, which is essentially solving the homogeneous transformation equation of the type AX=XB. For traditional 2D camera-based vision systems, the classic checkerboard image calibration method is relatively mature, and calibration is completed through the extraction and matching of image feature points. However, the output of the line laser profile sensor is three-dimensional point cloud information, which is fundamentally different from the image information. As a result, the traditional hand-eye calibration method based on image feature matching cannot be directly applied. Therefore, it is urgent to study the hand-eye calibration method suitable for line laser profile sensors to ensure the measurement accuracy and operation accuracy of the robot system and promote the further development of robot intelligent technology. Summary of the Invention

[0004] The embodiments of the present disclosure provide a hand-eye calibration method, device, and system based on a genetic algorithm to solve the related problems existing in existing technical solutions.

[0005] In view of the above problems, a first aspect is to provide a hand-eye calibration method based on a genetic algorithm, comprising:

[0006] Acquire point cloud data collected by the line structured light sensor multiple times and determine the corresponding feature points;

[0007] Determine the coordinates of each feature point in the flange coordinate system and the sensor coordinate system;

[0008] Based on the coordinates of multiple feature points in the flange coordinate system and the sensor coordinate system, the initial value of the hand-eye calibration matrix is determined using the ICP algorithm;

[0009] The initial value of the hand-eye calibration matrix is used as a priori solution of the genetic algorithm, and the optimal hand-eye calibration matrix is solved through multiple iterations.

[0010] In conjunction with the first aspect, in one possible implementation, acquiring point cloud data collected by the line structured light sensor and determining feature points includes:

[0011] Obtain point cloud data collected by the line structured light sensor;

[0012] Filtering out noise points in the point cloud data;

[0013] Fit the point cloud data after filtering out noise points to obtain the laser profile;

[0014] A preset position in the laser profile is determined as the feature point.

[0015] In combination with the first aspect, in a possible implementation manner, determining the coordinates of each feature point in the flange coordinate system includes:

[0016] For each feature point, aligning the calibrated end effector of the robot with the feature point to determine the coordinates of the feature point in the base coordinate system of the robot;

[0017] The homogeneous transformation matrix of the flange coordinate system relative to the base coordinate system is calculated using the current posture of the robot;

[0018] The coordinates of the feature points in the flange coordinate system are determined based on the homogeneous transformation matrix of the flange coordinate system relative to the base coordinate system and the coordinates of the feature points in the base coordinate system.

[0019] In combination with the first aspect, in one possible implementation, determining the initial value of the hand-eye calibration matrix using an ICP algorithm based on the coordinates of the multiple feature points in the flange coordinate system and the sensor coordinate system includes:

[0020] Determine the first relationship between the coordinates of the feature points in the flange coordinate system, the coordinates of the feature points in the sensor coordinate system, and the hand-eye calibration matrix according to the core formula of the ICP algorithm;

[0021] Substituting the coordinates of each feature point in the flange coordinate system and the coordinates of the sensor coordinate system into the first relational expression to obtain a second relational expression corresponding to the feature point;

[0022] The initial value of the hand-eye calibration matrix is obtained through the equation group composed of all second relationship expressions.

[0023] In conjunction with the first aspect, in one possible implementation, using the initial value of the hand-eye calibration matrix as a priori solution of a genetic algorithm and solving the optimal hand-eye calibration matrix through multiple iterations includes:

[0024] The fitness function is constructed by combining the soft constraint penalty function method;

[0025] Initializing a population based on a priori solutions of the genetic algorithm;

[0026] Calculating the fitness value of each individual in the population based on the fitness function;

[0027] Perform iterative operations on the population;

[0028] The iterative operation steps include:

[0029] Select individuals with high fitness values in the population as parents to perform crossover and / or mutation operations to generate new offspring individuals;

[0030] Calculating the fitness values of the new offspring individuals, and forming a new population from the parent individuals and the individuals with higher fitness values among the new offspring individuals;

[0031] Determine whether the current population meets the preset convergence conditions;

[0032] If not satisfied, return to the step of performing iterative operations on the population;

[0033] If satisfied, the solution of the optimal hand-eye calibration matrix is determined based on the individual with the highest fitness value in the current population;

[0034] The fitness value of the individual is determined based on the population size or a preset value.

[0035] In conjunction with the first aspect, in a possible implementation, constructing a fitness function by combining a soft constraint penalty function method includes:

[0036] The coordinates of the feature points in the sensor coordinate system are transformed into the base coordinate system through the hand-eye calibration matrix, and the objective function is determined based on the difference between the transformed coordinates and the coordinates of the feature points in the base coordinate system;

[0037] Taking the orthogonality constraint of the unit vector of the hand-eye calibration matrix as a constraint condition;

[0038] Substitute the objective function and the constraint conditions into the penalty function formula to form a fitness function.

[0039] In conjunction with the first aspect, in a possible implementation, initializing the population based on the priori solution of the genetic algorithm includes:

[0040] Encoding the parameters to be solved in the hand-eye calibration matrix using a preset encoding method;

[0041] Determining a spatial search range of the parameter to be solved based on an initial value of the hand-eye calibration matrix;

[0042] The population size is determined and an initial population is randomly generated based on the spatial search range.

[0043] In conjunction with the first aspect, in one possible implementation, determining the solution of the optimal hand-eye calibration matrix based on the individual with the highest fitness value in the current population includes:

[0044] Decode the solution corresponding to the individual with the highest fitness value in the current population to obtain the parameters to be solved;

[0045] Substituting the parameters to be solved into the hand-eye calibration matrix, the solution of the optimal hand-eye calibration matrix is obtained.

[0046] In a second aspect, a hand-eye calibration device based on a genetic algorithm is provided, comprising:

[0047] A data acquisition module is used to acquire point cloud data collected by the line structured light sensor multiple times and determine the corresponding feature points;

[0048] A coordinate determination module, used to determine the coordinates of each feature point in the flange coordinate system and the sensor coordinate system;

[0049] An initial value calculation module is used to determine the initial value of the hand-eye calibration matrix through the ICP algorithm based on the coordinates of multiple feature points in the flange coordinate system and the sensor coordinate system;

[0050] The genetic solution module is used to use the initial value of the hand-eye calibration matrix as a priori solution of the genetic algorithm and complete the solution of the optimal hand-eye calibration matrix through multiple iterations.

[0051] In combination with the second aspect, in a possible embodiment, the data acquisition module is used to acquire point cloud data collected by the line structured light sensor; filter out noise points in the point cloud data; fit the point cloud data after filtering out the noise points to obtain a laser contour; and determine a preset position in the laser contour as the feature point.

[0052] In combination with the second aspect, in a possible implementation, the coordinate determination module is used to align the calibrated robot's end effector with each feature point to determine the coordinates of the feature point in the robot's base coordinate system; use the robot's current posture to calculate the homogeneous transformation matrix of the flange coordinate system relative to the base coordinate system; and determine the coordinates of the feature point in the flange coordinate system based on the homogeneous transformation matrix of the flange coordinate system relative to the base coordinate system and the coordinates of the feature point in the base coordinate system.

[0053] In combination with the second aspect, in a possible implementation, the initial value calculation module is used to determine the first relationship between the coordinates of the feature point in the flange coordinate system, the coordinates of the feature point in the sensor coordinate system, and the hand-eye calibration matrix according to the core formula of the ICP algorithm; the coordinates of each feature point in the flange coordinate system and the coordinates in the sensor coordinate system are substituted into the first relationship to obtain the second relationship corresponding to the feature point; and the initial value of the hand-eye calibration matrix is obtained through a group of equations consisting of all second relationship formulas.

[0054] In conjunction with the second aspect, in a possible implementation, the genetic solution module is used to construct a fitness function in combination with a soft constraint penalty function method;

[0055] Initializing a population based on a priori solutions of the genetic algorithm;

[0056] Calculating the fitness value of each individual in the population based on the fitness function;

[0057] Perform iterative operations on the population;

[0058] The iterative operation steps include:

[0059] Select individuals with high fitness values in the population as parents to perform crossover and / or mutation operations to generate new offspring individuals;

[0060] Calculating the fitness values of the new offspring individuals, and forming a new population from the parent individuals and the individuals with higher fitness values among the new offspring individuals;

[0061] Determine whether the current population meets the preset convergence conditions;

[0062] If not satisfied, return to the step of performing iterative operations on the population;

[0063] If satisfied, the solution of the optimal hand-eye calibration matrix is determined based on the individual with the highest fitness value in the current population;

[0064] The fitness value of the individual is determined based on the population size or a preset value.

[0065] In combination with the second aspect, in a possible implementation, the genetic solution module is used to transform the coordinates of the feature points in the sensor coordinate system into the base coordinate system through the hand-eye calibration matrix, and determine the objective function based on the difference between the transformed coordinates and the coordinates of the feature points in the base coordinate system; use the orthogonal constraints of the unit vectors of the hand-eye calibration matrix as constraints; and substitute the objective function and the constraints into the penalty function formula to form a fitness function.

[0066] In combination with the second aspect, in a possible implementation, the genetic solution module is used to encode the parameters to be solved in the hand-eye calibration matrix using a preset encoding method; determine the spatial search range of the parameters to be solved based on the initial value of the hand-eye calibration matrix; determine the population size and randomly generate an initial population based on the spatial search range.

[0067] In combination with the second aspect, in one possible implementation, the genetic solution module is used to decode the solution corresponding to the individual with the highest fitness value in the current population to obtain the parameters to be solved; and substitute the parameters to be solved into the hand-eye calibration matrix to obtain the solution of the optimal hand-eye calibration matrix.

[0068] In a third aspect, a hand-eye calibration system based on a genetic algorithm is provided, comprising: a calibration plate, a robot, and a hand-eye calibration device based on a genetic algorithm as described in the second aspect, or in combination with any possible implementation of the second aspect;

[0069] The robot comprises: a linear structured light sensor, a flange and an end effector;

[0070] The robot is used to adjust its own posture multiple times so that the light plane projected by the line structured light sensor intersects with the edge of the calibration plate;

[0071] The hand-eye calibration device based on the genetic algorithm is used to repeatedly obtain the point cloud data collected by the structured light sensor after the robot adjusts its own posture line and determine the corresponding feature points; determine the coordinates of each feature point in the flange coordinate system and the sensor coordinate system; based on the coordinates of multiple feature points in the flange coordinate system and the sensor coordinate system, determine the initial value of the hand-eye calibration matrix through the ICP algorithm; use the initial value of the hand-eye calibration matrix as the prior solution of the genetic algorithm, and complete the solution of the optimal hand-eye calibration matrix through multiple iterations.

[0072] The beneficial effects of the embodiments of the present disclosure include:

[0073] The embodiments of the present disclosure provide a hand-eye calibration method, device, and system based on a genetic algorithm, which are applied in the field of robot vision to solve the problem of calibrating the posture relationship between the visual sensor and the robot's end effector (such as a flange).

[0074] The method disclosed herein first repeatedly acquires point cloud data from a line structured light sensor and identifies feature points, providing sufficient data samples for the subsequent calculation of the initial values of the hand-eye calibration matrix using the ICP (Iterative Closest Point) algorithm. Multiple sets of point cloud data and their corresponding feature points comprehensively reflect the spatial relationship between the sensor and the flange in different poses, making the initial values calculated based on this data more global and reliable. Next, the coordinates of each feature point in both the flange and sensor coordinate systems are determined, clarifying the positional representation of the same feature point in each coordinate system. This establishes data associations for calculating the transformation relationship between the two, forming the core data foundation for hand-eye calibration. The ICP algorithm is then used to determine the initial values of the hand-eye calibration matrix. Through iterative optimization, the distance between the two sets of coordinates is minimized, quickly obtaining an approximate transformation matrix. This provides an optimal starting point for the subsequent genetic algorithm, reducing the search space and improving computational efficiency. Finally, the initial values are used as the prior solution for the genetic algorithm, and the optimal hand-eye calibration matrix is solved through multiple iterations. By simulating natural selection, the genetic algorithm searches for the optimal solution globally, overcoming the problem of traditional methods easily falling into local optimality and improving calibration accuracy.

[0075] This method solves the technical problem that line structured light sensors cannot directly apply traditional hand-eye calibration methods. By combining the ICP algorithm with the genetic algorithm, it not only ensures computational efficiency but also achieves high-precision global optimization, ensuring that the robot system can complete complex tasks based on accurate coordinate transformation relationships, thereby improving work quality and efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0076] Figure 1 A flowchart of a hand-eye calibration method based on a genetic algorithm provided in an embodiment of the present disclosure;

[0077] Figure 2 A schematic diagram of a robot hand-eye calibration model provided in an embodiment of the present disclosure;

[0078] Figure 3 A schematic diagram illustrating the coordinate transformation relationship between the robot and the sensor in different postures provided by an embodiment of the present disclosure;

[0079] Figure 4 This is a diagram showing the effect of feature point extraction provided by an embodiment of the present disclosure;

[0080] Figure 5 One of the flow charts for solving the optimal hand-eye calibration matrix through multiple iterations provided in an embodiment of the present disclosure;

[0081] Figure 6 The second flowchart of solving the optimal hand-eye calibration matrix through multiple iterations provided in an embodiment of the present disclosure;

[0082] Figure 7 A field calibration diagram of the verification experiment provided by the embodiment of the present disclosure;

[0083] Figure 8 This is a diagram showing the iterative effect of the genetic algorithm provided in the embodiment of the present disclosure;

[0084] Figure 9 A comparison chart of the effects of the embodiment of the present disclosure and the traditional method;

[0085] Figure 10 A structural diagram of a hand-eye calibration device based on a genetic algorithm provided in an embodiment of the present disclosure;

[0086] Figure 11 This is a structural diagram of a hand-eye calibration system based on a genetic algorithm provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0087] The present disclosure provides a genetic algorithm-based hand-eye calibration method, device, and system. Preferred embodiments of the present disclosure are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are intended only to illustrate and explain the present disclosure and are not intended to limit the present disclosure. Furthermore, the embodiments and features within the embodiments may be combined with one another unless there is a conflict.

[0088] The present disclosure provides a hand-eye calibration method based on genetic algorithm. Figure 1 Shown, including:

[0089] S101, acquiring point cloud data collected by the line structured light sensor multiple times and determining corresponding feature points;

[0090] S102, determining the coordinates of each feature point in the flange coordinate system and the sensor coordinate system;

[0091] S103, determining an initial value of a hand-eye calibration matrix using an ICP algorithm based on the coordinates of the multiple feature points in the flange coordinate system and the sensor coordinate system;

[0092] S104: Using the initial value of the hand-eye calibration matrix as a priori solution of the genetic algorithm, and completing the solution of the optimal hand-eye calibration matrix through multiple iterations.

[0093] To facilitate understanding, we first provide a schematic diagram of a possible robot hand-eye calibration model, as shown in Figure 2 As shown, Figure 2 {B} represents the robot base coordinate system, {E} represents the flange coordinate system, and {L} represents the sensor coordinate system. B P represents the coordinates of the feature point in the {B} coordinate system, L P represents the coordinates of the feature point in the {L} coordinate system, Represents the homogeneous transformation matrix of {E} relative to {B}, which can be calculated from the robot position and posture. It is the homogeneous transformation matrix of {L} relative to {E}, that is, the hand-eye calibration matrix.

[0094] When a robot works collaboratively with a visual sensor, the three-dimensional point cloud data collected by the sensor needs to be converted to the robot's base coordinate system to achieve precise positioning and operation of the robot's end effector. The conversion relationship between the flange (end effector) and the robot's base coordinate system can be obtained through robot posture calculation, so it is necessary to accurately determine the conversion relationship between the flange and the sensor coordinate system, which is called hand-eye calibration.

[0095] In the embodiment of the present disclosure, each time the point cloud data is acquired, the robot changes its posture and the end flange with the sensor aligns the laser stripe with the straight edge of the calibration plate, and then stops and collects the data to obtain a set of point cloud data. Figure 3 As shown, changing the robot's posture can involve only translation (changing position), or changing both the position and posture simultaneously. This is not a limitation, but generally, for ease of operation, only changing the position suffices. The coordinate transformation relationship between the robot and the sensor at different postures is used to determine the initial value of a stable hand-eye calibration matrix, achieving precise alignment between the visual coordinate system and the robot coordinate system.

[0096] Based on this, a hand-eye calibration method combining the iterative closest point (ICP) algorithm and the genetic algorithm is proposed, and the hand-eye calibration matrix is obtained through multi-step processing.

[0097] In the disclosed embodiment, in step S101, point cloud data collected by the line structured light sensor is acquired multiple times and corresponding feature points are determined. This is achieved by scanning from multiple perspectives and multiple positions to obtain multiple sets of point cloud data, and corresponding feature points are determined for each set of point cloud data. The specific number of data acquisition times can be preset during implementation.

[0098] Step S102 determines the flange coordinates of each feature point in the flange coordinate system and the sensor coordinates in the sensor coordinate system. By using sensor measurement principles and the robot's kinematic model, the positions of the same feature point in different coordinate systems are quantified, establishing data associations and providing the necessary data for calculating the transformation relationship.

[0099] In step S103, the ICP algorithm is used to iteratively optimize the distance between the flange coordinates and the sensor coordinates of multiple feature points to determine the initial value of the hand-eye calibration matrix. This initial value provides a starting point for subsequent optimization.

[0100] In step S104, the initial value of the hand-eye calibration matrix is used as the prior solution for the genetic algorithm. By simulating the natural selection mechanism, the global optimal solution is searched through multiple iterations to obtain an accurate hand-eye calibration matrix. This process avoids the problem of traditional methods falling into local optimality and accurately and efficiently obtains the optimal hand-eye calibration matrix.

[0101] This method combines the rapid initial value calculation capabilities of the ICP algorithm with the global optimization properties of the genetic algorithm, addressing the problem of linear structured light sensors being unable to directly use traditional hand-eye calibration methods. By collecting multiple sets of data and collaborating with the two algorithms, the optimal hand-eye calibration matrix is solved, thereby improving calibration accuracy and stability, ensuring that the robot system completes tasks based on accurate coordinate transformation relationships, and effectively improving production quality and efficiency.

[0102] In another embodiment provided by the present disclosure, the above step S101, acquiring point cloud data collected by the line structured light sensor multiple times and determining corresponding feature points, can be implemented as follows:

[0103] Step 1: Obtain point cloud data collected by the line structured light sensor;

[0104] Step 2: Filter out noise points in the point cloud data;

[0105] Step 3: Fit the point cloud data after filtering out noise points to obtain the laser profile;

[0106] Step 4: Determine the preset position in the laser profile as a feature point.

[0107] In the hand-eye calibration method based on genetic algorithms, step S101 extracts feature points using point cloud data collected by a line structured light sensor. However, during the sensor imaging process, due to environmental interference and measurement noise, the acquired raw point cloud data often contains outliers. These noise points usually exhibit irregular and sparse distribution characteristics, which can easily introduce deviations and data distortion in local areas, thereby adversely affecting the subsequent point cloud processing process. Therefore, after acquiring the three-dimensional point cloud data, the outlier noise points must be effectively filtered out. Step S101 is the basic link for data acquisition and feature point extraction, and its specific implementation process directly affects the subsequent calibration accuracy. Therefore, effective measures must be taken to eliminate noise interference.

[0108] In the embodiment of the present disclosure, step S101 includes the following steps when implemented:

[0109] Step 1: Obtain point cloud data collected by the line structured light sensor. This is achieved by emitting a laser beam from the sensor to scan the target, receiving the reflected light, and converting it into a collection of three-dimensional points. This is the raw data source for the entire calibration process.

[0110] The point cloud data obtained by a linear structured light sensor contains both large pulse interference and small high-frequency noise. Therefore, the collected data should be processed first. Step 2 removes noise points from the point cloud data. Due to factors such as sensor measurement errors and environmental interference, the raw point cloud data may contain outliers that deviate from the true surface. These noise points can be removed through algorithms such as statistical filtering and voxel filtering. For example, statistical filtering can be used to remove points that deviate significantly from the average distance, ensuring that subsequent calculations are based on valid data. In addition, in one possible implementation, the DBSCAN clustering algorithm can be used to process the point cloud data, which can both remove outliers and prevent the influence of noise near the contour on the fitted line.

[0111] Step 3: Fit the point cloud data after noise filtering to obtain the laser profile. A curve fitting algorithm (such as the least squares method for fitting a line or curve) is used to fit the discrete point cloud data into a continuous laser profile to facilitate subsequent feature extraction. Alternatively, in one possible implementation, a RANSAC algorithm can be used to fit the point cloud data after noise filtering to obtain a more ideal laser profile.

[0112] Step 4 determines the preset position in the laser profile as the feature point, and the extraction effect is as follows Figure 4 As shown in the figure, the left endpoint of the contour can be selected as the preset position as the feature point because its position is fixed and easy to identify, providing a stable reference for coordinate system transformation. In actual applications, the preset position in the laser contour can be selected as the left endpoint, or the contour midpoint, right endpoint, outer corner vertex, or other points with geometric characteristics as required.

[0113] The method provided by the disclosed embodiments effectively improves the accuracy and stability of feature points through a systematic data processing process. It converts raw point cloud data into valid features that can be used for calibration, avoiding errors caused by data quality issues. This provides a high-quality data foundation for the subsequent ICP algorithm to determine the initial value of the hand-eye calibration matrix and for genetic algorithm optimization, thereby improving the accuracy and reliability of the entire hand-eye calibration method.

[0114] In another embodiment provided by the present disclosure, the above step S102, determining the coordinates of each feature point in the flange coordinate system, can be implemented as follows:

[0115] Step 1: For each feature point, align the calibrated robot end effector with the feature point to determine the coordinates of the feature point in the robot's base coordinate system;

[0116] Step 2: Calculate the homogeneous transformation matrix of the flange coordinate system relative to the base coordinate system using the current posture of the robot;

[0117] Step 3: Based on the homogeneous transformation matrix of the flange coordinate system relative to the base coordinate system and the coordinates of the feature points in the base coordinate system, determine the coordinates of the feature points in the flange coordinate system.

[0118] In the hand-eye calibration process based on the genetic algorithm, step S102 aims to achieve accurate mapping of feature points from the robot base coordinate system to the flange coordinate system, providing basic data for subsequent hand-eye calibration matrix calculation. This step is completed by the following operations:

[0119] First, perform step 1. After the robot adjusts its posture, it aligns the reference point of the end effector (e.g., the welding gun tip in a welding robot) with the extracted feature points. During this process, encoder data from each robot joint is collected in real time. Using a kinematic forward solution algorithm, the 3D coordinates of the feature points in the robot's base coordinate system are calculated and output. For example, when calibrating a workpiece, the welding gun tip is precisely aligned with the laser contour feature points to obtain their coordinates in the base coordinate system.

[0120] Next, proceed to step 2. Based on the current robot pose data, a preset kinematic model (such as the Denavit-Hartenberg model) is called to perform matrix operations. By combining the rotation matrix and the translation vector, a homogeneous transformation matrix is generated that describes the position relationship between the flange coordinate system and the base coordinate system. This matrix fully incorporates the flange's rotation and translation information.

[0121] Finally, execute step 3 to convert the coordinates obtained in step 1 in the base coordinate system into homogeneous coordinates and multiply them with the homogeneous transformation matrix generated in step 2. Through the coordinate transformation calculation, the three-dimensional coordinate values of the feature point in the flange coordinate system are output, completing the coordinate transformation of the feature point from the global space to the local space of the end effector.

[0122] This implementation establishes a systematic coordinate transformation mechanism through autonomous robot motion control and coordinate calculation. Using kinematic models and matrix operations to achieve coordinate space mapping, it effectively avoids coordinate transformation errors and provides high-precision data support for subsequent ICP algorithm determination of the hand-eye calibration matrix initial value and genetic algorithm optimization, thereby improving the overall accuracy and reliability of the hand-eye calibration method.

[0123] In another embodiment provided by the present disclosure, the above step S103, determining the initial value of the hand-eye calibration matrix by the ICP algorithm based on the coordinates of the multiple feature points in the flange coordinate system and the sensor coordinate system, can be implemented as follows:

[0124] Step 1: Determine the first relationship between the coordinates of the feature points in the flange coordinate system, the coordinates of the feature points in the sensor coordinate system, and the hand-eye calibration matrix according to the core formula of the ICP algorithm;

[0125] Step 2: Substitute the coordinates of each feature point in the flange coordinate system and the sensor coordinate system into the first relational expression to obtain a second relational expression corresponding to the feature point;

[0126] Step 3: Obtain the initial value of the hand-eye calibration matrix through the equation group composed of all the second relations.

[0127] In the disclosed embodiment of the genetic algorithm-based hand-eye calibration process, step S103 is the key step in obtaining the initial value of the calibration matrix, connecting the feature point coordinate transformation and the global optimization process. The mathematical relationship model is constructed and solved using the ICP algorithm, providing a reliable initial solution for subsequent genetic algorithm iterations. This is achieved through the following steps.

[0128] During step 1, the built-in ICP algorithm core formula is invoked. Based on the principles of rigid body kinematics, a mathematical mapping relationship is established between the flange coordinate system, the sensor coordinate system, and the desired hand-eye calibration matrix, forming the first equation. This equation, in the form of a matrix transformation, describes the constraints for transforming the coordinates of the same feature point in the two coordinate systems using the hand-eye calibration matrix, providing a theoretical basis for subsequent calculations.

[0129] Step 2: Substitute the coordinate values of each acquired feature point in the flange coordinate system and the sensor coordinate system into the first relational expression one by one. Through matrix multiplication and equation operations, a corresponding second relational expression is derived for each feature point. For example, for multiple edge feature points on a workpiece, the equation constraints for their coordinate transformation are calculated separately. Each second relational expression reflects the positional relationship between the two coordinate systems for a single feature point.

[0130] Step 3 integrates the second relational expressions corresponding to all feature points to construct a system of equations. This system of equations is solved using numerical methods such as the least squares method or singular value decomposition (SVD). This process uses optimization calculations to find the hand-eye calibration matrix that minimizes the coordinate transformation error for all feature points, thereby determining the initial value. This initial value comprehensively considers the spatial distribution information of multiple feature points and preliminarily establishes the transformation relationship between the two coordinate systems.

[0131] This implementation systematically applies the ICP algorithm to the calculation of initial values for hand-eye calibration. Using data from multiple feature points, the constrained equations are constructed, avoiding the one-sided nature of single-point solutions. Compared to traditional random initialization, the initial values obtained using the ICP algorithm are closer to the true solution, providing a high-quality starting point for the subsequent global optimization of the genetic algorithm. This effectively reduces the number of iterations and improves the accuracy of the hand-eye calibration matrix, thereby ensuring that the robot system executes tasks based on reliable coordinate transformation relationships.

[0132] In a possible implementation, the determination of the first and second relationship equations is as follows:

[0133] According to the kinematic theory of multi-degree-of-freedom industrial robots, the following formula is given:

[0134]

[0135] Iterative Closest Point (ICP) is a point cloud matching algorithm that transforms a set of point clouds A = {a1, ..., a n} after rotation and translation with the second set of point clouds B = {b1, ..., b n} overlap algorithm. Its core formula is: a i =Rb i +t. Use SVD algorithm to solve the solution of R and t. R, t corresponds to Initial value.

[0136] Further transformation of formula (1) can be obtained:

[0137]

[0138] This formula is equivalent to:

[0139]

[0140] Click P i To fix a point on the calibration plate, point B P i The coordinates in the base coordinate system are fixed. Adjust the robot position, then point L P i The coordinates in the sensor coordinate system follow the matrix changes, so the first relation is:

[0141]

[0142] To estimate Through multiple measurements with the line structured light sensor, the specific coordinate data is substituted into the first equation to obtain the corresponding second equation. All the second equations are combined to obtain the following equation group:

[0143]

[0144] Because during the measurement (Hand-eye calibration matrix) always remains unchanged, so its estimation can be completed through ICP.

[0145] In another embodiment provided by the present disclosure, the above step S104, using the initial value of the hand-eye calibration matrix as the priori solution of the genetic algorithm and completing the solution of the optimal hand-eye calibration matrix through multiple iterations, can be implemented as follows:

[0146] Step 1: Construct a fitness function by combining the soft constraint penalty function method;

[0147] Step 2: Initialize the population based on the prior solution of the genetic algorithm;

[0148] Step 3: Calculate the fitness value of each individual in the population based on the fitness function;

[0149] Step 4: Perform iterative operations on the population;

[0150] The steps of the iterative operation include:

[0151] Select individuals with high fitness values in the population as parents to perform crossover and / or mutation operations to generate new offspring individuals;

[0152] Calculate the fitness value of the new offspring individuals, and form a new population with the individuals with higher fitness values among the parent individuals and the new offspring individuals;

[0153] Step 5: Determine whether the current population meets the preset convergence conditions;

[0154] If not satisfied, return to step 4 and perform iterative operations on the population;

[0155] Step 6: If satisfied, determine the solution of the optimal hand-eye calibration matrix based on the individual with the highest fitness value in the current population;

[0156] Among them, the fitness value of an individual is determined based on the population size or preset value.

[0157] In the embodiment of the present disclosure, in the hand-eye calibration process based on the genetic algorithm, step S104 is the core link for obtaining a high-precision hand-eye calibration matrix. Based on the initial value obtained in step S103, the hand-eye calibration matrix is further optimized through the global optimization characteristics of the genetic algorithm. Figure 5 The process in , through the steps of constructing fitness function, initializing population, performing iterative operations and judging convergence conditions, realizes the transformation from initial value to optimal solution. The specific implementation is as follows:

[0158] During step 1, a fitness function is constructed using a soft constraint penalty function to measure the quality of different hand-eye calibration matrix solutions. This function considers the error between the feature point coordinates in the flange coordinate system and their coordinates in the sensor coordinate system after transformation using the hand-eye calibration matrix, as well as other constraints (such as the orthogonality of the rotation matrix). The error is then mapped to a fitness value. Smaller errors are associated with higher fitness values, providing a quantitative criterion for subsequent selection of high-quality solutions.

[0159] Step 2: Based on the initial value of the hand-eye calibration matrix obtained in step S103, multiple similar individuals are generated through random perturbation or neighborhood search to initialize the population of the genetic algorithm. Each individual represents a potential solution to the hand-eye calibration matrix, forming the initial search space.

[0160] In step 3, the fitness function is constructed and calculated for each individual in the population to obtain the corresponding fitness value. For example, for multiple individual hand-eye calibration matrix solutions in a workpiece calibration scenario, the fitness corresponding to their coordinate transformation errors is calculated. A higher value indicates that the solution is closer to the true transformation relationship.

[0161] During the iterative process in step 4, individuals with higher fitness values are first selected as parents. Crossover operations are used to recombinant gene fragments and mutation operations are used to introduce new genetic features, generating new offspring individuals. The fitness values of the offspring individuals are then calculated, and the individuals with higher fitness values from both the parent and offspring generations are retained to form a new population, achieving evolutionary selection of the population.

[0162] During step 5, the algorithm can be evaluated for convergence by judging the current population's fitness distribution, the maximum number of iterations, and other preset conditions. If the conditions are not met, the algorithm returns to step 4 and continues iterating. If the conditions are met, the algorithm proceeds to step 6, where the individual with the highest fitness value in the current population is selected as the solution to the optimal hand-eye calibration matrix.

[0163] This implementation combines a genetic algorithm with soft-constrained optimization, using the initial values obtained by the ICP algorithm as the search starting point to avoid the blindness of traditional random search. Through an adaptive population iteration and screening mechanism, it effectively overcomes the problem of traditional methods easily falling into local optimality. It can globally optimize in complex solution spaces, significantly improving the accuracy and stability of the hand-eye calibration matrix, and providing a reliable coordinate transformation basis for precise robot operation.

[0164] In a possible implementation, the specific evolutionary operations in the genetic algorithm include:

[0165] Calculate individual fitness: Evaluate the fitness value of each individual and perform evolutionary operations accordingly.

[0166] Crossover operation: Use uniform crossover to exchange genes at a preset crossover rate (such as 0.6) to increase the diversity of the population.

[0167] Mutation operation: impose a preset mutation rate (such as 0.01) on offspring individuals to improve search ability and prevent premature convergence.

[0168] Elite retention strategy: select elite individuals whose fitness values are ranked within the preset range in the population or those ranked within the limit of the population size to directly enter the next generation in order to retain the better solutions in the optimization process.

[0169] Penalty item setting: In order to balance optimization efficiency and accuracy, the penalty coefficient M is set (the value of M can be set according to actual conditions) to ensure the effectiveness of the constraints while avoiding deviation in results caused by excessive penalties.

[0170] Termination criteria: Set a maximum number of iterations and / or introduce a number of no updates as termination criteria. If multiple iterations fail to yield a better solution, the optimization process ends prematurely. The final output is the accurate hand-eye matrix.

[0171] In another embodiment provided by the present disclosure, the above step 1, constructing a fitness function in combination with a soft constraint penalty function method, can be implemented as follows:

[0172] Step (1), transforming the coordinates of the feature points in the sensor coordinate system into the base coordinate system through the hand-eye calibration matrix, and determining the objective function based on the difference between the transformed coordinates and the coordinates of the feature points in the base coordinate system;

[0173] Step (2), taking the orthogonality constraint of the unit vector of the hand-eye calibration matrix as a constraint condition;

[0174] Step (3): Substitute the objective function and constraints into the penalty function formula to form the fitness function.

[0175] In the disclosed embodiment, in the genetic algorithm-based hand-eye calibration step S104, constructing a fitness function is key to measuring the quality of the hand-eye calibration matrix solution, directly affecting the optimization direction and accuracy of the genetic algorithm. By combining the soft constraint penalty function method, the coordinate transformation error and the matrix constraint conditions are quantitatively integrated. The specific implementation is as follows:

[0176] When step (1) is executed, the coordinate data of the feature points in the sensor coordinate system is obtained, and the coordinates are transformed using the current hand-eye calibration matrix. The coordinates obtained after the transformation in the base coordinate system are interpolated with the coordinates of the feature points actually measured in the base coordinate system, and the error between the two is used as the objective function. For example, in a workpiece calibration scenario, the coordinates of multiple laser contour feature points are transformed, and the distance error between the transformed coordinates and the coordinates of the actual feature points in the base coordinate system is calculated. This error value directly reflects the accuracy of the current hand-eye calibration matrix. The smaller the error, the closer the matrix is to the true transformation relationship.

[0177] Step (2) takes into account the orthogonality constraint that the rotation part of the hand-eye calibration matrix must satisfy (i.e., the unit vectors of the rotation matrix are perpendicular to each other and have a modulus of 1). This constraint is taken into consideration. This constraint relationship is described by constructing a mathematical expression, such as calculating whether the product of the rotation matrix and its transposed matrix is close to the identity matrix. This is used as one of the conditions for measuring the validity of the matrix to avoid solving a matrix that does not conform to physical meaning.

[0178] Step (3) substitutes the objective function in step (1) and the constraints in step (2) into the penalty function formula. The penalty function "penalizes" the current hand-eye calibration matrix solution based on the error of the objective function and the degree of violation of the constraints, and finally generates a fitness function. If a matrix solution has a small error in coordinate transformation and satisfies the orthogonal constraints, the fitness function assigns it a higher fitness value; otherwise, the fitness value is reduced.

[0179] This implementation method systematically combines coordinate transformation accuracy with matrix physical constraints, innovatively constructing a fitness function that conforms to the characteristics of hand-eye calibration. Compared to approaches that solely consider coordinate errors, this method can ensure both the accuracy and physical plausibility of the solution during the genetic algorithm search process, effectively avoiding the generation of invalid calibration matrix solutions. By quantifying constraints and errors, it provides a clear optimization guide for the genetic algorithm, significantly improving the global search efficiency of hand-eye calibration and the reliability of the final results, laying the foundation for robots to perform tasks based on accurate coordinate transformations.

[0180] In genetic algorithms, the fitness function measures the performance of individuals and serves as the core guide for population evolution. For optimization problems subject to constraints, relying solely on the objective function often fails to achieve both accurate and feasible solutions. Therefore, a penalty function mechanism is often employed, incorporating constraints into the fitness function as penalty terms, thereby transforming constrained optimization problems into unconstrained ones for solution.

[0181] This disclosure takes the optimization of homogeneous transformation matrix parameters in hand-eye calibration as an example to construct a fitness function containing target terms and constraint terms. In the hand-eye calibration process, ideally, the feature points observed by the sensor should theoretically completely coincide with the position of the point in the robot base coordinate system after continuous transformation of the hand-eye matrix and the robot end posture. Therefore, minimizing the spatial distance between the two points can be used as the optimization goal to establish the objective function. At the same time, considering that the rotation part in the hand-eye matrix must satisfy the unit orthogonality constraint, a penalty term is introduced to appropriately punish individuals that violate the constraint. For the specific process, please refer to Figure 6 .

[0182] To clearly express the relationship between the above objectives and constraints, the present disclosure provides a spatial transformation process that combines the kinematics of industrial robots with hand-eye calibration, mathematically derives the positions of sensor feature points in the robot base coordinate system, and provides a theoretical basis for the construction of the fitness function.

[0183] In a possible implementation, according to the kinematics theory of multi-degree-of-freedom industrial robots, the following formula is obtained:

[0184]

[0185] Taking the case of changing the robot's position when changing the robot's posture as an example, in the sensor coordinate system, since the Y direction is the scanning direction of the line structured light sensor, and the line structured light sensor is stationary when collecting data during calibration, the first set of scanned point cloud data is taken as an example. The Y coordinate in the obtained XYZ data is 0 (the value of the Y coordinate will change with the change of the robot's position), so let P L =(x l ,O,z l ) T , P B =(x b ,y b , z b ) T , according to the data structure characteristics of the homogeneous matrix, it can be transformed into:

[0186]

[0187] Among them and The data is the Euler angle pose transformation from the robot to the homogeneous matrix, where The result of hand-eye calibration matrix is and in Translate and rotate the parts separately.

[0188] Arrange formula (6) to get:

[0189]

[0190] Will Substituting into formula (7) we get:

[0191]

[0192] The robot's known The data is brought into the above formula, and the identified feature point data is substituted into formula (8). After transformation, the representation P of the sensor feature point in the base coordinate is obtained. B ′, and P B Perform distance difference calculation as the objective function in the subsequent penalty function:

[0193]

[0194] Since the Y-axis coordinate value in the sensor coordinate system is always 0, Elements in It has no effect on the solution result, so the parameters to be solved are:

[0195]

[0196] The unit vector constraint in formula (10) is:

[0197]

[0198] The penalty function method is a classic algorithm for solving constrained optimization problems. The basic idea is to introduce a penalty factor based on the constraints and combine it with the objective function, thereby transforming the optimization problem from constrained to unconstrained, achieving the penalty objective. The optimal solution to the constrained optimization problem is obtained by solving a series of unconstrained optimization problems. The core idea of the penalty function is to penalize individuals that violate the constraints.

[0199] The general form of the mathematical model of the optimization constraint problem involved in the disclosure can be expressed by the following formula:

[0200] minf(X),X=[X1,X2,X3,…,X n ]∈R

[0201] sth i (X)=0,i=1,2,3,…,p (12)

[0202] Where: n represents the population size, h i (X) represents the i-th equality constraint, p represents the number of equality constraints, f(X) serves as the objective function, and X can be represented as an n-dimensional vector:

[0203] X n =(x n1 , x n2 , x n3 ,…,x nm ) (13)

[0204] The quadratic penalty function method constructs an objective function and introduces a penalty factor, taking the degree of constraint violation as a penalty term and gradually adding it to the objective function to form a soft constraint penalty function. In this way, the algorithm allows for a certain degree of constraint violation and imposes penalties based on the degree of violation, thereby guiding the solution to converge in the direction of satisfying the constraints. The penalty function form is:

[0205] F(x)=f(x)+P(x) (14)

[0206] Among them: F(x) is the penalty function. In the composition of the penalty term P(x), let

[0207] H i ={h1(x)-ε,h1(x)-3ε,h2(x)-ε,h2(x)-3ε,…h i (x)-ε,h i (x)-3ε} (15)

[0208] Where ε is a small tolerance value that allows the solution to violate the constraint within a certain range. The penalty term can be written as:

[0209]

[0210] There will be two penalty operations here and both will be included in the penalty term, where h i It is not necessarily exactly 0. Instead, it leaves a certain amount of slack space. If h i (x) is less than ε, then (h i (x)-ε)≤0, there will be no penalty; only when h i When (x) exceeds ε, max{0,h i (x)-ε} will become a positive value and be added to the penalty. i (x)<3ε is subject to a penalty. i (x)>3ε will be subject to additional penalties, which makes the solutions with a larger degree of violation be "quadratically penalized" and thus less likely to be retained by the genetic algorithm.

[0211] In summary, by combining the spatial transformation relationship between robot kinematic modeling and hand-eye calibration, minimizing the feature point coordinate error as the objective function, and converting constraints such as the orthogonality of the rotation matrix into penalty terms, and introducing them into the fitness function of the genetic algorithm, it not only effectively improves the accuracy and physical feasibility of the solution, but also realizes the unconstrained solution of the constrained optimization problem.

[0212] In another embodiment provided by the present disclosure, the above step 2, initializing the population based on the priori solution of the genetic algorithm, can be implemented as follows:

[0213] Step (1) encoding the parameters to be solved in the hand-eye calibration matrix using a preset encoding method;

[0214] Step (2), determining the spatial search range of the parameters to be solved based on the initial value of the hand-eye calibration matrix;

[0215] Step (3), determine the population size and randomly generate the initial population based on the spatial search range.

[0216] In the disclosed embodiment, in the genetic algorithm-based hand-eye calibration process, step 2 of step S104 serves as the foundation for algorithm startup. This process converts the initial values of the hand-eye calibration matrix into an initial population for the genetic algorithm to search, providing a starting point for subsequent iterative optimization. This process constructs an initial solution space that meets the optimization requirements through three steps: encoding, determining the search range, and randomly generating individuals. The specific implementation is as follows:

[0217] When executing step (1), a preset encoding method such as binary encoding or real number encoding can be selected to encode the unknown rotation matrix parameters (such as Euler angles and quaternions) and translation vector parameters in the hand-eye calibration matrix. For example, by using real number encoding, the 9 elements of the rotation matrix and the 3 elements of the translation vector are directly mapped to real number encoding strings, so that the matrix parameters are converted into a gene sequence form that can be processed by the genetic algorithm, which is convenient for subsequent crossover and mutation operations.

[0218] Step (2) is based on the initial value of the hand-eye calibration matrix obtained in step S103, and its parameter distribution is analyzed to determine the spatial search range of the parameters to be solved. By floating the initial value up and down by a certain proportion (such as a range of ±10% based on the initial value), or setting boundaries based on the robot kinematics and sensor measurement range, a search space containing potential optimal solutions is defined. For example, if the value of a parameter of the rotation matrix in the initial value is 0.8, the search range of the parameter is set to [0.72, 0.88] to ensure that the initial population covers a reasonable solution space area.

[0219] Step (3) generates multiple individuals within the parameter search range determined in step (2) using a random number generation algorithm based on the preset population size parameters (e.g., setting the number of individuals in the population to 50 or 100) to form an initial population. Each individual corresponds to a potential hand-eye calibration matrix solution, and the differences between individuals provide a diverse search starting point for the genetic algorithm. For example, in the workpiece calibration scenario, 50 individuals that meet the parameter range are randomly generated to form the initial population for subsequent iterative optimization.

[0220] This implementation transforms the initial values of the hand-eye calibration matrix into an operational initial population for the genetic algorithm through a systematic encoding strategy, a reasonable search range definition, and randomized individual generation. Compared to blindly generating initial solutions, this method leverages the prior information provided by the initial values to narrow the search range and reduce ineffective searches. Furthermore, encoding and randomization ensure population diversity, providing a high-quality starting point for the genetic algorithm to efficiently optimize globally. This initialization strategy effectively balances search efficiency and solution accuracy, significantly improving both the efficiency and final accuracy of the hand-eye calibration matrix solution, thereby ensuring the reliability of the robot's coordinate system transformation.

[0221] In one possible implementation, in order to enable the parameters in the problem space to be searched in a genetic algorithm, they need to be converted into a gene string structure in the genetic space. This process is called chromosome encoding. For example, binary encoding of continuous variables is used for encoding.

[0222] First, set the search space range of a parameter to [L, U], where U is the upper limit of the search space and L is the lower limit of the search space. For example, if the string length is n and the binary encoding parameter is used, there are 2 nThere are different encoding methods. The corresponding relationships are as follows:

[0223] 000...0000=0→L

[0224] 000...0001=1→L+δ

[0225] 000...0010=2→L+2δ

[0226]

[0227] 111...1111=2”-1→U (17)

[0228] in:

[0229]

[0230] In another embodiment provided by the present disclosure, in the above step 6, determining the solution of the optimal hand-eye calibration matrix based on the individual with the highest fitness value in the current population can be implemented as follows:

[0231] Step (1), decoding the solution corresponding to the individual with the highest fitness value in the current population to obtain the parameters to be solved;

[0232] Step (2): Substitute the parameters to be solved into the hand-eye calibration matrix to obtain the solution of the optimal hand-eye calibration matrix.

[0233] In the disclosed embodiment of the hand-eye calibration process based on the genetic algorithm, step 6 of step S104 is the key step in ultimately obtaining a high-precision calibration matrix. This process processes the population after the genetic algorithm iterative convergence and converts the optimal individual found by the algorithm search into a practically usable hand-eye calibration matrix. This process achieves the mapping from the algorithm solution to the physical space coordinate transformation relationship through two steps: decoding and parameter substitution. The specific implementation is as follows:

[0234] When step (1) is executed, the individual with the highest fitness value in the current population is subjected to a reverse operation, i.e., decoding, based on the preset encoding method (e.g., binary encoding or real number encoding) adopted in step (1) in step S104. For example, if real number encoding is used to encode the hand-eye calibration matrix parameters into a real number string, then by parsing the real number string, the gene fragments therein are restored into rotation matrix elements and translation vector parameters to obtain the original parameter values to be solved. This step converts the gene sequence optimized by the algorithm into physical parameters that can be used to describe the coordinate system transformation relationship, providing a data basis for the subsequent generation of the calibration matrix.

[0235] Step (2) combines the parameters to be solved obtained by decoding step (1) according to the mathematical structure of the hand-eye calibration matrix. The rotation parameters are constructed as a rotation matrix, and the translation parameters are constructed as a translation vector. The final combination forms a complete homogeneous transformation matrix, that is, the solution of the optimal hand-eye calibration matrix.

[0236] This implementation method converts the genetic algorithm optimization results into a practical calibration matrix through systematic decoding and parameter reorganization operations. Compared with directly using the algorithm search results, this method, through the bidirectional encoding-decoding operation, not only ensures the effectiveness of the algorithm optimization process, but also ensures that the final solution conforms to the physical meaning and mathematical structure of the hand-eye calibration matrix. The optimal hand-eye calibration matrix obtained through this process can effectively eliminate coordinate conversion errors based on the results of previous multi-step data processing and algorithm optimization, provide a high-precision coordinate system conversion basis for the robot system, significantly improve the accuracy and stability of task execution, and achieve accurate calibration of the spatial relationship between the robot and the sensor.

[0237] The decoding process may refer to the following embodiments:

[0238] Assume that the binary code of an individual is b n b n-1 b n-2 ...b2b1, the corresponding decoding formula is as follows:

[0239]

[0240] Assume parameter X∈[2,4], encode X with 5-bit binary, with a total of 2 5 There are encoding schemes. Assume that one of the encoding schemes is x=10101, and the corresponding decimal number is:

[0241]

[0242] The corresponding parameter X value is:

[0243]

[0244] The present disclosure provides an embodiment to verify the accuracy and versatility of the hand-eye calibration method based on genetic algorithm provided by the present disclosure.

[0245] The robot used in the experiment is a six-axis ER-20 model produced by Efort. The positioning accuracy is 0.06mm. The line structured light camera model is MV-DP2120-01H produced by Hikvision. The on-site calibration diagram is shown in the figure below. Figure 7 Among them: 1- calibration plate; 2- line laser camera; 3- welding gun; 4- EFORT ER-20 robot; 5- workbench.

[0246] Using the on-site experimental device, three sets of data of 15 characteristics were collected. The flange position is shown in Table 1.

[0247] Table 1 Flange attitude data

[0248]

[0249] The calibration plate is fixed, so the position of the feature points under the base coordinate system does not change. The feature points are measured using the calibrated end tool. The positions of the feature points in the base coordinate system are shown in Table 2:

[0250] Table 2 Coordinates of feature points in the base coordinate system

[0251]

[0252]

[0253] The data collected multiple times are subjected to endpoint identification, and the endpoints are shown in Table 3:

[0254] Table 3 Feature points recognized by the sensor

[0255]

[0256] Based on the ICP algorithm, the initial value of the hand-eye matrix is:

[0257]

[0258] After calculation by genetic algorithm, the hand-eye matrix is obtained as follows:

[0259]

[0260] The iterative effect is as follows Figure 8 As shown, from Figure 8 It can be seen that the value of formula (14) gradually decreases during the iteration process, satisfying the trend of the penalty function being 0.

[0261] To verify the accuracy of the results, an experiment was conducted. 15 points of data were collected in a pose outside the calibration process. The feature points were transformed using formula (1) to obtain their representations in the {B} coordinate system. The transformed points were then compared with the feature points in the base coordinate system {B}. Table 4 shows the transformed representations of the feature points in the camera in {B} coordinates and their original representations in {B}. This shows that the data transformation results are relatively accurate.

[0262] Table 4 Comparison of conversion points and original feature points

[0263]

[0264] In order to verify the optimization of the results, the formula is used:

[0265] f(X)=|P i ′-P| (9)

[0266] Define the distance between two points, and perform distance calculation on the feature points after they are transformed by formula (1) with the points under the base coordinates.

[0267] The disclosed method is compared with the traditional method through the corresponding point distance error results. The comparison results are as follows Figure 9 ,The results show that the distance between points is within 0.2mm, and the proposed method is more ,effective than the original method.

[0268] In summary, this disclosure provides a hand-eye calibration method based on a penalty function genetic algorithm. This method directly utilizes straight edges as calibration features, eliminating the need for complex calibration templates and simplifying implementation. Experimental results demonstrate that this method achieves high calibration accuracy and good stability, making it suitable for on-site calibration. The calibration error is kept within 0.2 mm, meeting common requirements.

[0269] In addition, the present disclosure uses a welding robot as an example for illustration and experimental verification. The hand-eye calibration method based on genetic algorithm provided by the present disclosure is not limited to welding robots. It can be widely applied to various industrial robots with hand-eye collaborative operation requirements, including but not limited to assembly robots, spraying robots, handling robots, robotic arms, etc., as well as other automated equipment involving vision-actuator calibration.

[0270] Based on the same disclosed concept, the embodiments of the present disclosure also provide a hand-eye calibration device and system based on a genetic algorithm. Since the principles of the problems solved by these devices and systems are similar to those of the aforementioned hand-eye calibration method based on a genetic algorithm, the implementation of the device and system can refer to the implementation of the aforementioned method, and the repeated parts will not be repeated.

[0271] With the above Figure 1 Corresponding to the method shown, the embodiment of the present disclosure also provides a hand-eye calibration device based on genetic algorithm, such as Figure 10 Shown, including:

[0272] The data acquisition module 1001 is used to acquire point cloud data collected by the line structured light sensor multiple times and determine the corresponding feature points;

[0273] A coordinate determination module 1002 is used to determine the coordinates of each feature point in the flange coordinate system and the sensor coordinate system;

[0274] An initial value calculation module 1003 is used to determine the initial value of the hand-eye calibration matrix through an ICP algorithm based on the coordinates of multiple feature points in the flange coordinate system and the sensor coordinate system;

[0275] The genetic solution module 1004 is used to use the initial value of the hand-eye calibration matrix as the priori solution of the genetic algorithm and complete the solution of the optimal hand-eye calibration matrix through multiple iterations.

[0276] In another embodiment provided by the present disclosure, the data acquisition module 1001 is used to acquire point cloud data collected by a line structured light sensor; filter out noise points in the point cloud data; fit the point cloud data after filtering out the noise points to obtain a laser profile; and determine a preset position in the laser profile as a feature point.

[0277] In another embodiment provided by the present disclosure, the coordinate determination module 1002 is used to align the calibrated end effector of the robot with the feature point for each feature point to determine the coordinates of the feature point in the base coordinate system of the robot; use the current posture of the robot to calculate the homogeneous transformation matrix of the flange coordinate system relative to the base coordinate system; based on the homogeneous transformation matrix of the flange coordinate system relative to the base coordinate system and the coordinates of the feature point in the base coordinate system, determine the coordinates of the feature point in the flange coordinate system.

[0278] In another embodiment provided by the present disclosure, the initial value calculation module 1003 is used to obtain the first relationship between the coordinates of the feature point in the flange coordinate system, the coordinates of the feature point in the sensor coordinate system, and the hand-eye calibration matrix according to the core formula of the ICP algorithm; substitute the coordinates of each feature point in the flange coordinate system and the coordinates in the sensor coordinate system into the first relationship to obtain the second relationship corresponding to the feature point; and obtain the initial value of the hand-eye calibration matrix through the equation group composed of all the second relationship.

[0279] In another embodiment provided by the present disclosure, the genetic solution module 1004 is used to construct a fitness function in combination with a soft constraint penalty function method;

[0280] Initialize the population based on the prior solution of the genetic algorithm;

[0281] Calculate the fitness value of each individual in the population based on the fitness function;

[0282] Perform iterative operations on the population;

[0283] The steps of the iterative operation include:

[0284] Select individuals with high fitness values in the population as parents to perform crossover and / or mutation operations to generate new offspring individuals;

[0285] Calculate the fitness value of the new offspring individuals, and form a new population with the individuals with higher fitness values among the parent individuals and the new offspring individuals;

[0286] Determine whether the current population meets the preset convergence conditions;

[0287] If not, return to the step of performing iterative operations on the population;

[0288] If satisfied, the solution of the optimal hand-eye calibration matrix is determined based on the individual with the highest fitness value in the current population;

[0289] Among them, the fitness value of an individual is determined based on the population size or preset value.

[0290] In another embodiment provided by the present disclosure, the genetic solution module 1004 is used to transform the coordinates of the feature points in the sensor coordinate system into the base coordinate system through the hand-eye calibration matrix, and determine the objective function based on the difference between the transformed coordinates and the coordinates of the feature points in the base coordinate system; use the orthogonal constraints of the unit vectors of the hand-eye calibration matrix as constraints; and substitute the objective function and the constraints into the penalty function formula to form a fitness function.

[0291] In another embodiment provided by the present disclosure, the genetic solution module 1004 is used to encode the parameters to be solved in the hand-eye calibration matrix using a preset encoding method; determine the spatial search range of the parameters to be solved based on the initial value of the hand-eye calibration matrix; determine the population size and randomly generate an initial population based on the spatial search range.

[0292] In another embodiment provided by the present disclosure, the genetic solution module 1004 is used to decode the solution corresponding to the individual with the highest fitness value in the current population to obtain the parameters to be solved; and substitute the parameters to be solved into the hand-eye calibration matrix to obtain the solution of the optimal hand-eye calibration matrix.

[0293] With the above Figure 1 Corresponding to the method shown, the embodiment of the present disclosure also provides a hand-eye calibration system based on genetic algorithm, such as Figure 11 As shown, it includes: a calibration plate 1101, a robot 1102 and a hand-eye calibration device 1103 based on a genetic algorithm as provided in any of the above embodiments;

[0294] The robot 1102 includes: a linear structured light sensor, a flange, and an end effector;

[0295] The robot 1102 is used to adjust its own posture multiple times so that the light plane projected by the line structured light sensor intersects with the edge of the calibration plate 1101;

[0296] A hand-eye calibration device 1103 based on a genetic algorithm is used to repeatedly obtain point cloud data collected by a structured light sensor after the robot adjusts its own posture line and determine the corresponding feature points; determine the coordinates of each feature point in the flange coordinate system and the sensor coordinate system; determine the initial value of the hand-eye calibration matrix through the ICP algorithm based on the coordinates of multiple feature points in the flange coordinate system and the sensor coordinate system; use the initial value of the hand-eye calibration matrix as a priori solution of the genetic algorithm, and complete the solution of the optimal hand-eye calibration matrix through multiple iterations.

[0297] In the disclosed embodiment, a hand-eye calibration system based on a genetic algorithm is constructed to achieve precise coordinate conversion between the robot and the line structured light sensor. This system, through the coordinated operation of a calibration plate 1101, a robot 1102, and a hand-eye calibration device 1103 based on a genetic algorithm, converts the point cloud data collected by the sensor into global three-dimensional data in the robot coordinate system. The specific workflow is as follows:

[0298] Calibration plate 1101 serves as a feature reference, its straight edge providing a stable measurement target for the linear structured light sensor. Robot 1102, equipped with the linear structured light sensor, flange, and end effector, controls joint motion and adjusts its position multiple times to ensure that the light plane projected by the linear structured light sensor intersects the straight edge of calibration plate 1101.

[0299] The genetic algorithm-based hand-eye calibration device 1103, serving as the core processing unit, first repeatedly acquires point cloud data collected by the structured light sensor after the robot 1102 adjusts its posture and extracts feature points from it, providing basic data for calibration. The coordinates of each feature point in the flange coordinate system and the sensor coordinate system are then determined, establishing a data association. Based on the coordinates of these multiple feature points, the ICP algorithm is used to calculate the initial values of the hand-eye calibration matrix, quickly obtaining an approximate transformation relationship. Finally, this initial value is used as the genetic algorithm's prior solution, and through multiple iterative optimizations, the optimal hand-eye calibration matrix is solved.

[0300] This system combines hardware acquisition with algorithmic processing. Robot 1102 actively adjusts its posture to collect multiple sets of data. This system combines the global optimization capabilities of the genetic algorithm with the rapid initial value calculation characteristics of the ICP algorithm to achieve high-precision hand-eye calibration. The straight edges of the calibration plate 1101 are directly used as calibration features, eliminating the need for complex calibration templates and simplifying implementation. Compared to traditional calibration methods, this system effectively overcomes the calibration challenges posed by the characteristics of line structured light sensor data. Through multi-step collaborative processing, it significantly improves the robot's coordinate conversion accuracy during operation, ensuring high consistency between robot 1102's operations and sensor measurements.

[0301] Through the description of the above embodiments, those skilled in the art can clearly understand that the embodiments of the present disclosure can be implemented through hardware or by means of software plus a necessary general hardware platform. Based on this understanding, the technical solutions of the embodiments of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.), including a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in the various embodiments of the present disclosure.

[0302] Those skilled in the art will understand that the accompanying drawings are merely schematic diagrams of a preferred embodiment, and the modules or processes in the accompanying drawings are not necessarily required for implementing the present disclosure.

[0303] Those skilled in the art will appreciate that the modules in the devices of the embodiments may be distributed in the devices of the embodiments as described in the embodiments, or may be located in one or more devices different from the embodiments with corresponding changes. The modules of the above embodiments may be combined into one module or further split into multiple submodules.

[0304] The serial numbers of the above-mentioned embodiments of the present disclosure are for description only and do not represent the advantages or disadvantages of the embodiments.

[0305] Obviously, those skilled in the art may make various changes and modifications to the present disclosure without departing from the spirit and scope of the present disclosure. Thus, if these modifications and variations of the present disclosure fall within the scope of the claims of the present disclosure and their equivalents, the present disclosure is intended to include these modifications and variations.

Claims

1. A hand-eye calibration method based on genetic algorithm, characterized in that: include: Acquire point cloud data collected by the line structured light sensor multiple times and determine the corresponding feature points; Determine the coordinates of each feature point in the flange coordinate system and the sensor coordinate system; Based on the coordinates of multiple feature points in the flange coordinate system and the sensor coordinate system, the initial value of the hand-eye calibration matrix is determined using the ICP algorithm; The initial value of the hand-eye calibration matrix is used as a priori solution of the genetic algorithm, and the optimal hand-eye calibration matrix is solved through multiple iterations.

2. The method according to claim 1, wherein The step of obtaining point cloud data collected by the line structured light sensor and determining feature points includes: Obtain point cloud data collected by the line structured light sensor; Filtering out noise points in the point cloud data; Fit the point cloud data after filtering out noise points to obtain the laser profile; A preset position in the laser profile is determined as the feature point.

3. The method according to claim 1, wherein Determining the coordinates of each feature point in the flange coordinate system includes: For each feature point, aligning the calibrated end effector of the robot with the feature point to determine the coordinates of the feature point in the base coordinate system of the robot; The homogeneous transformation matrix of the flange coordinate system relative to the base coordinate system is calculated using the current posture of the robot; The coordinates of the feature points in the flange coordinate system are determined based on the homogeneous transformation matrix of the flange coordinate system relative to the base coordinate system and the coordinates of the feature points in the base coordinate system.

4. The method according to claim 1, wherein The method of determining the initial value of the hand-eye calibration matrix by using the ICP algorithm based on the coordinates of the multiple feature points in the flange coordinate system and the sensor coordinate system includes: Determine the first relationship between the coordinates of the feature points in the flange coordinate system, the coordinates of the feature points in the sensor coordinate system, and the hand-eye calibration matrix according to the core formula of the ICP algorithm; Substituting the coordinates of each feature point in the flange coordinate system and the coordinates of the sensor coordinate system into the first relational expression to obtain a second relational expression corresponding to the feature point; The initial value of the hand-eye calibration matrix is obtained through the equation group composed of all second relationship expressions.

5. The method according to claim 1, wherein The initial value of the hand-eye calibration matrix is used as a priori solution of the genetic algorithm, and the optimal hand-eye calibration matrix is solved through multiple iterations, including: The fitness function is constructed by combining the soft constraint penalty function method; Initializing a population based on a priori solutions of the genetic algorithm; Calculating the fitness value of each individual in the population based on the fitness function; Perform iterative operations on the population; The iterative operation steps include: Select individuals with high fitness values in the population as parents to perform crossover and / or mutation operations to generate new offspring individuals; Calculating the fitness values of the new offspring individuals, and forming a new population from the parent individuals and the individuals with higher fitness values among the new offspring individuals; Determine whether the current population meets the preset convergence conditions; If not satisfied, return to the step of performing iterative operations on the population; If satisfied, the solution of the optimal hand-eye calibration matrix is determined based on the individual with the highest fitness value in the current population; The fitness value of the individual is determined based on the population size or a preset value.

6. The method according to claim 5, wherein The method of constructing a fitness function by combining a soft constraint penalty function includes: The coordinates of the feature points in the sensor coordinate system are transformed into the base coordinate system through the hand-eye calibration matrix, and the objective function is determined based on the difference between the transformed coordinates and the coordinates of the feature points in the base coordinate system; Taking the orthogonality constraint of the unit vector of the hand-eye calibration matrix as a constraint condition; Substitute the objective function and the constraint conditions into the penalty function formula to form a fitness function.

7. The method according to claim 5, wherein Initializing the population based on the priori solution of the genetic algorithm includes: Encoding the parameters to be solved in the hand-eye calibration matrix using a preset encoding method; Determining a spatial search range of the parameter to be solved based on an initial value of the hand-eye calibration matrix; The population size is determined and an initial population is randomly generated based on the spatial search range.

8. The method according to claim 5, wherein The method of determining the solution of the optimal hand-eye calibration matrix according to the individual with the highest fitness value in the current population includes: Decode the solution corresponding to the individual with the highest fitness value in the current population to obtain the parameters to be solved; Substituting the parameters to be solved into the hand-eye calibration matrix, the solution of the optimal hand-eye calibration matrix is obtained.

9. A hand-eye calibration device based on genetic algorithm, characterized in that: include: A data acquisition module is used to acquire point cloud data collected by the line structured light sensor multiple times and determine the corresponding feature points; A coordinate determination module, used to determine the coordinates of each feature point in the flange coordinate system and the sensor coordinate system; An initial value calculation module is used to determine the initial value of the hand-eye calibration matrix through the ICP algorithm based on the coordinates of multiple feature points in the flange coordinate system and the sensor coordinate system; The genetic solution module is used to use the initial value of the hand-eye calibration matrix as a priori solution of the genetic algorithm and complete the solution of the optimal hand-eye calibration matrix through multiple iterations.

10. A hand-eye calibration system based on genetic algorithm, characterized in that: include: A calibration plate, a robot, and a hand-eye calibration device based on a genetic algorithm as claimed in claim 9; The robot comprises: a linear structured light sensor, a flange and an end effector; The robot is used to adjust its own posture multiple times so that the light plane projected by the line structured light sensor intersects with the edge of the calibration plate; The hand-eye calibration device based on the genetic algorithm is used to repeatedly obtain the point cloud data collected by the structured light sensor after the robot adjusts its own posture line and determine the corresponding feature points; determine the coordinates of each feature point in the flange coordinate system and the sensor coordinate system; based on the coordinates of multiple feature points in the flange coordinate system and the sensor coordinate system, determine the initial value of the hand-eye calibration matrix through the ICP algorithm; use the initial value of the hand-eye calibration matrix as the prior solution of the genetic algorithm, and complete the solution of the optimal hand-eye calibration matrix through multiple iterations.

Images