Multi-robot joint calibration method, system and computer equipment
The features of the calibration feature block are acquired through external structural parts and acquisition devices, and the conversion relationship between the multi-robot camera coordinate system and the world coordinate system is directly calculated, solving the problem of the complexity and accuracy of the multi-level coordinate system conversion in the existing technology, realizing the High-precision multi-robot joint calibration.
Patent Information
- Application Number
- CN202410971595.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-19
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2044-07-19
AI Technical Summary
The existing multi-robot joint calibration method is complex and has limited accuracy. The errors are introduced in the multi-level coordinate system conversion, and the deviation of internal parameters of the robot affects the calibration accuracy.
Using external structural parts as calibration device, the surface features of calibration feature blocks are obtained through the target acquisition device and the reference acquisition device, the calibration matrix is calculated, and the multi-level coordinate system conversion is bypassed, and the conversion relationship between the camera coordinate system and the world coordinate system is directly solved to avoid Use robot body parameters to improve calibration accuracy.
The multi-robot joint calibration process is simplified, the accuracy of end-position calibration is improved, the influence of matrix calculation errors is reduced, and the accuracy and robustness of calibration results are enhanced.
Smart Images

Figure CN118514083B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of automatic control technology, and in particular to a multi-robot joint calibration method, system and computer equipment. Background Art
[0002] In current large-scale online measurement, multiple robot terminals equipped with multiple measurement cameras are required to perform mobile multi-point imaging at several spatial positions, and the multi-point measurement data is converted into a unified world coordinate system for overall measurement. In this process, there is a spatial posture relationship between several measurement points of a single robot, and there is also a spatial coordinate system conversion relationship between multiple robots. These spatial conversion relationships need to be solved and obtained through the joint calibration of the multi-robot terminal posture.
[0003] At present, a multi-level coordinate system conversion method is usually used. First, a single robot equipped with a single measurement camera is calibrated by hand and eye to obtain the conversion relationship between the camera coordinate system and the robot terminal center coordinate system. Then, the robot model and the angles of each joint are used to convert the robot terminal center coordinate system to the robot base coordinate system. Finally, the base coordinate systems of multiple robots are converted to a unified world coordinate system. However, the above method has many calibration steps and the operation is relatively complicated. In addition, multiple link errors are introduced during the conversion of multiple coordinate systems, which reduces the overall calibration effect. At the same time, when the coordinate system of the center point of the robot terminal is converted to the coordinate system of the robot base, parameters such as the joint angle and connecting rod length read inside the robot are required. However, the parameters read inside the robot often deviate from the true values, which limits the calibration accuracy.
[0004] Therefore, how to simplify the multi-robot joint calibration process while improving the accuracy of robot terminal position and posture calibration is a problem that needs to be solved at this stage. Summary of the invention
[0005] Based on this, the purpose of this application is to provide a method, system and computer equipment that can both simplify the multi-robot joint calibration process and improve the accuracy of robot end position calibration, so as to solve the technical problems mentioned in the above background technology.
[0006] In a first aspect, the present application provides a multi-robot joint calibration method. Each robot in the multi-robot is equipped with a target acquisition device at the end, including:
[0007] Determine a calibration feature block located at each spatial measuring point in the calibration device;
[0008] When each robot determines the corresponding running trajectory according to the multiple spatial measurement points, the first surface feature of the calibration feature block is obtained through the target acquisition device;
[0009] Acquire a second surface feature of each calibration feature block by a reference acquisition device;
[0010] According to the first surface feature and the second surface feature, a calibration matrix corresponding to each spatial measuring point is obtained;
[0011] According to the calibration matrix, the error index corresponding to each spatial measurement point is determined, and when the error index meets the preset conditions, the calibration result is obtained; when the error index does not meet the preset conditions, the calibration feature block data is updated and the calibration matrix is re-determined until the calibration matrix meets the preset conditions or reaches the iteration termination conditions, and the calibration result is obtained.
[0012] In a second aspect, the present application also provides a multi-robot joint calibration system, the system comprising multiple robots, a terminal, a calibration device and a reference acquisition device, and the terminal of each robot in the multiple robots is equipped with a target acquisition device, wherein:
[0013] A terminal, used to determine a calibration feature block located at each spatial measurement point in the calibration device;
[0014] The target acquisition device is used to send the first surface feature of the acquired calibration feature block to the terminal when each robot determines the corresponding running trajectory according to the multiple spatial measurement points;
[0015] A reference collection device is used to send the collected second surface features of each calibration feature block to the terminal;
[0016] A terminal, used for obtaining a calibration matrix corresponding to each spatial measurement point according to the first surface feature and the second surface feature;
[0017] The terminal is used to determine the error index corresponding to each spatial measurement point according to the calibration matrix, and obtain the calibration result when the average error meets the preset conditions; when the error index does not meet the preset conditions, the calibration feature block data is updated and the calibration matrix is re-determined until the calibration matrix meets the preset conditions or reaches the iteration termination conditions to obtain the calibration result.
[0018] In a third aspect, the present application further provides a computer device, which includes a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps in the multi-robot joint calibration method when executing the computer program.
[0019] The above-mentioned multi-robot joint calibration method, system, and computer device determine the calibration feature block located at each spatial measurement point in the calibration device, and when each robot determines the corresponding running trajectory according to multiple spatial measurement points, the first surface feature of the calibration feature block can be obtained through the target acquisition device, and then the second surface feature of each calibration feature block can be obtained through the reference acquisition device. In this way, the calibration matrix corresponding to each spatial measurement point can be obtained according to the first surface feature and the second surface feature. Therefore, by designing an external structural part as a calibration device, the characteristic size and position constraints of the calibration device in the world coordinate system corresponding to the reference acquisition device are used to solve the conversion relationship between the camera coordinate system corresponding to the target acquisition device at the end of the robot at the spatial measurement point and the world coordinate system. Only through this conversion relationship, that is, the calibration matrix, the camera coordinate system of multiple robots at several spatial measurement points can be converted to a unified world coordinate system, which simplifies the joint calibration of the terminal posture of multiple robots in large scenes.
[0020] In addition, when the error index corresponding to each spatial measurement point is determined according to the calibration matrix, and when the error index meets the preset conditions, the calibration result can be obtained. Since the parameter information of the robot body is not used in the calibration process, the robot is only regarded as the clamping and moving device of the target acquisition device, and a coordinate system conversion relationship calculation is performed, the influence of the matrix calculation accuracy on the overall calibration accuracy is avoided to the greatest extent, and the accuracy of the robot terminal position and posture calibration is improved. BRIEF DESCRIPTION OF THE DRAWINGS
[0021] Figure 1 is an application environment diagram of a multi-robot joint calibration system in one embodiment;
[0022] Figure 2 is a schematic structural diagram of a calibration device in one embodiment;
[0023] Figure 3 A schematic diagram of coordinate transformation in one embodiment;
[0024] Figure 4 Schematic diagram of a multi-robot joint calibration method in one embodiment;
[0025] Figure 5 A schematic diagram of the structure of a calibration feature block in an embodiment;
[0026] Figure 6 is a schematic flow chart of a multi-robot joint calibration method in another embodiment;
[0027] Figure 7 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION
[0028] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.
[0029] The multi-robot joint calibration method provided in the embodiment of the present application can be applied to Figure 1 In the multi-robot joint calibration system shown in the figure. The multi-robot joint calibration system includes a terminal 1, multiple robots 2, a calibration device 4 and a reference acquisition device; the end of each robot in the multi-robot is equipped with a target acquisition device 3, and is fixedly connected to the end flange of the multi-robot; the terminal can be a computing processing unit, the multi-robot can be multiple six-axis robots, and the target acquisition device includes but is not limited to a measuring camera. The calibration device is also a combined special calibration device, and the structural diagram is shown in FIG. Figure 2 As shown, the calibration device includes a support structure, such as a rigid and stable bracket 5 and a plurality of calibration feature blocks 6. The calibration device is placed at the required operating position according to the measurement requirements, and the plurality of calibration feature blocks are respectively installed at the preset spatial measurement points on the calibration device, that is, the calibration device is used to set a corresponding calibration feature block at each spatial measurement point; the calibration feature blocks are used to provide different surface features. The base of the multi-robot is fixedly installed around the calibration device; the terminal can be placed around the measurement station and communicated with the multi-robot and the target acquisition device through cables.
[0030] The data collected by the target acquisition device is in the camera coordinate system at the end position of the robot, that is, Figure 3 In Coordinate system; the data collected by the reference acquisition device is in the world coordinate system, that is, Figure 3 In Coordinate system. In order to achieve absolute measurement of multiple robots in a large scene, it is necessary to convert the camera coordinate system of several robot end positions into a unified world coordinate system. Therefore, it is necessary to solve the camera coordinate system of different robot end positions. With the unified world coordinate system The conversion relationship between them can realize the joint calibration of multi-robot terminal poses.
[0031] In one embodiment, Figure 4 As shown, a multi-robot joint calibration method is provided, comprising the following steps:
[0032] Step 401, determining a calibration feature block located at each spatial measurement point in a calibration device.
[0033] Specifically, the terminal plans the measurement points according to the spatial distribution of the features to be measured in the preset measurement task and the target acquisition device carried by the multi-robot terminal, determines the number of spatial measurement points n in the calibration device and the spatial position of the corresponding target acquisition device, and numbers each spatial measurement point i=1, 2, ...n. A calibration feature block is set at each spatial measurement point on the rigid and stable bracket, and the calibration feature block is required to be within the viewing angle of the target acquisition device corresponding to the point, at which time the calibration device is built.
[0034] In one embodiment, the terminal determines the number of robots m and the placement of the multi-robot bases based on all spatial measurement points and the spatial positions corresponding to each target acquisition device, and in combination with the arm spans and workspaces of different robots, so that at least one robot end can be identified at each spatial measurement point, and the robots are numbered j = 1, 2, ... m. When each robot end is equipped with a target acquisition device, the number of the target acquisition device is the same as the corresponding robot number.
[0035] Step 402: When each robot determines the corresponding running trajectory according to a plurality of spatial measurement points, a first surface feature of the calibration feature block is acquired through a target acquisition device.
[0036] The first surface feature includes but is not limited to a point cloud data.
[0037] Specifically, after the terminal assigns each spatial measurement point to a different robot for identification, the robot pre-plans the corresponding running trajectory according to the assigned spatial measurement point, so that the surface of the calibrated feature block corresponding to all spatial measurement points can be within the field of view of the target acquisition device. For example, the terminal assigns n spatial measurement points to m robots and plans the running trajectory. Each robot j is responsible for The measurement points in space, the measurement point corresponding to robot j is expressed as ,satisfy and Each spatial measuring point corresponds to a calibration feature block, so the number of the calibration feature block is the same as the spatial measuring point number and can be represented by i.
[0038] Then, when the robot receives the operation instruction from the terminal, it can run to each spatial measurement point one by one according to the pre-planned operation trajectory, thereby triggering the target acquisition device on the robot to collect the first surface feature of the calibration feature block at the spatial measurement point. The terminal selects one of the robots j and sends the operation instruction to the robot j through the cable. After receiving the operation instruction, the robot j drives the target acquisition device to move to each spatial measurement point. And feedback the arrival signal to the terminal. After confirming the arrival signal of the target acquisition device, the terminal sends a trigger signal to the target acquisition device j through the cable. After receiving the trigger signal, the target acquisition device j can collect the first surface features of the calibration feature block at the spatial measurement point one by one. , and the first surface feature will be The data is transmitted back to the terminal through the cable so that the terminal can record the number of each spatial measurement point.
[0039] Step 403: Acquire the second surface feature of each calibration feature block by using a reference acquisition device.
[0040] The reference acquisition device includes but is not limited to a three-dimensional coordinate measuring machine; specifically, after receiving the trigger instruction from the terminal, the reference acquisition device acquires the second surface features of each calibration feature block one by one. Preferably, the second surface features include but are not limited to a point cloud data.
[0041] Step 404: Obtain a calibration matrix corresponding to each spatial measurement point according to the first surface feature and the second surface feature.
[0042] Specifically, after completing the collection of calibration feature blocks at all spatial measurement points, the terminal compares the first surface feature with the second surface feature, and solves the transformation relationship between the camera coordinate system and the world coordinate system of each spatial measurement point. The transformation relationship can be a calibration matrix, so that in subsequent actual measurements, all spatial measurement point data can be converted to a unified world coordinate system according to the transformation relationship to complete the joint calibration of the multi-robot terminal posture.
[0043] Step 405, according to the calibration matrix, determine the error index corresponding to each spatial measurement point, and when the error index meets the preset conditions, obtain the calibration result; when the error index does not meet the preset conditions, update the calibration feature block data, and redetermine the calibration matrix until the calibration matrix meets the preset conditions or reaches the iteration termination conditions, and obtain the calibration result.
[0044] Specifically, the terminal determines the feature point data of each feature unit on the surface of the calibration feature block. Taking the feature unit as a feature hole as an example, the feature point data may optionally include the center coordinates; the center coordinates include the first feature center coordinates extracted from the first surface feature and the second feature center coordinates extracted from the second surface feature. The terminal uses the calibration matrix to transform the center coordinates of the feature hole, and adopts a method of reducing the error of solving the conversion matrix between coordinate systems to obtain the average error, covariance and other error indicators before and after the conversion of the corresponding calibration feature block. If the error indicator is greater than the preset threshold E, it returns to the step of re-determining the feature hole on the surface of the calibration feature block, and implements the update iterative processing of different feature point data until the error indicator is less than or equal to the preset threshold or the iteration termination condition is reached. At this point, the terminal realizes the relevant data recording of the calibration process and the calculation of the calibration results.
[0045] In the above-mentioned multi-robot joint calibration method, the world coordinate system corresponding to the reference acquisition device is used as the reference coordinate system, and the position of the spatial measurement point on the combined special calibration device in the world coordinate system is used as the conversion target value to solve the conversion relationship between the camera coordinate system and the world coordinate system under different terminal postures of the robot. Since the world coordinate system is the coordinate system of the workpiece to be measured or a virtual space point, the camera coordinate systems at different robot spatial measurement points can be unified into the same coordinate system through the above-mentioned conversion relationship to complete the cross-robot terminal posture joint calibration. The calibration process of the present application bypasses the existing multi-level coordinate system conversion method, omits the original intermediate multiple coordinate system transfer process, and obtains the calibration result of the corresponding spatial measurement point by calculating the calibration matrix only once, without calculating multiple coordinate system conversions. In this way, the multi-robot joint calibration process is greatly simplified.
[0046] In the conversion relationship between the two coordinate systems of this application, that is, the process of solving the calibration matrix, the calibration matrix obtained is affected by the data structure and the accuracy of numerical calculation, and there are calculation errors in the calibration matrix. In the process of obtaining the calibration result through multi-level coordinate system conversion in the existing method, it is necessary to solve the conversion relationship between multiple coordinate systems, for example: the calibration matrix between the camera coordinate system-the robot end center point coordinate system, and the calibration matrix between the robot base coordinate system-the world coordinate system. Since the calibration matrix between multiple coordinate systems is used for matrix operations, the calculation error of the calibration matrix of each level of coordinate system conversion will be accumulated and amplified. In addition, the conversion relationship between the robot end center point coordinate system and the robot base coordinate system is returned through the robot body, and the joint angles and connecting rod lengths of each level of the robot have certain deviations from the true values, which affects the conversion accuracy of the coordinate system at this level. The above two factors cause the calibration accuracy of the existing multi-level coordinate system conversion method to be limited.
[0047] Therefore, the present application does not use the parameter information of the robot body during the calibration process, but only regards the robot as a clamping and moving device of the target acquisition device, and only performs one coordinate system transformation relationship calculation, so as to minimize the influence of the matrix calculation accuracy on the overall calibration accuracy, thereby improving the accuracy of the robot terminal posture calibration; and through the judgment of the error index, the calibration feature block data is updated to further determine the calibration matrix, which can greatly improve the calibration accuracy.
[0048] In addition, the present application also has the advantage that the calibration device is highly robust to the scene environment. Due to the existing method of using external tracking devices to obtain the position and posture of multiple robots at specific points, it is necessary to place the external tracking device at a fixed position around the robot arm. At the same time, the external device needs to track the markers at the end of the robot in real time, and there must be no external physical obstruction between the tracking device and the end of the robot. These two points put forward certain requirements for the environment of the calibration scene, and if there is physical interference during the calibration process, such as object movement or staff walking, it may affect the calibration results. In addition to the fixed position of the robot itself, the calibration device of the present application can be placed directly at the planned position of the workpiece to be measured, without occupying additional site space, and the calibration equipment can be quickly built and modified to meet the calibration needs of different measurement scenarios.
[0049] In one embodiment, obtaining a calibration matrix corresponding to each spatial measurement point according to the first surface feature and the second surface feature includes:
[0050] For the current spatial measurement point traversed, the first feature coordinates of each feature unit are extracted through the first surface features of the current calibration feature block; the second feature coordinates of each feature unit are extracted through the second surface features of the current calibration feature block; and the calibration matrix corresponding to the current spatial measurement point is determined through multiple first feature coordinates and multiple second feature coordinates.
[0051] The calibration feature block includes a plurality of feature units, and a plurality of feature holes may be used as an example for explanation. Figure 5 As shown, there are 9 characteristic holes processed on it Of course, the type, quantity, position, etc. of the feature unit can be set arbitrarily to provide features.
[0052] Specifically, for the spatial measurement point The first surface feature of the calibration feature block , the terminal extracts the first surface feature Medium Feature Hole The terminal fits the center position of each characteristic hole on the plane where the surface of the calibration feature block is located, and records the coordinates of the first characteristic circle center of the characteristic hole fitted on the plane where the surface of the calibration feature block is located as , which is the first characteristic coordinate. Among them, ,and It is a 3*1 matrix, and each row stores the x, y, and z coordinates of the center of the circle in sequence. Similarly, for the spatial measurement points obtained by the reference sampling device The second surface feature of the calibration feature block The terminal fits the center position of each feature hole on the plane where the calibration feature block surface is located in the order of the feature hole number, and records the coordinates of the second feature center of the feature hole on the plane where the calibration block surface is located as , is the second characteristic coordinate. Among them, It is also a 3*1 matrix and is stored in the internal storage device of the terminal.
[0053] The coordinates of the center of the first characteristic circle are the spatial measurement points The three-dimensional coordinates of the center of the circle in the upper camera coordinate system, and the second characteristic center coordinates are the three-dimensional coordinates of the center of the circle in the world coordinate system. Thus, the spatial measurement point can be determined by multiple first characteristic center coordinates and multiple second characteristic center coordinates. The corresponding calibration matrix is .
[0054] in, Indicates spatial measurement points The rotation matrix between the camera coordinate system and the world coordinate system, Indicates spatial measurement points The translation matrix between the camera coordinate system and the world coordinate system. Then, the terminal can transform the transformation relationship between the two coordinate systems into the objective function:
[0055]
[0056] Among them, U represents the calibration matrix of the calibration feature block of the spatial measurement point involved in calculating the coordinate system transformation A collection of characteristic unit numbers.
[0057] In this embodiment, the coordinates of the feature points are used instead of the local measurement point cloud to match the space points and solve the coordinate system transformation matrix, which reduces the computational complexity and improves the calibration processing speed. For example, the calibration matrix of the space measurement points is calculated by comparing multiple sets of feature circle center coordinates, thereby improving the calibration accuracy. It is worth noting that the feature hole is only an adaptive example, and the feature unit can also be selected but not limited to a triangle, a slot, etc., with its corner point coordinates as the feature coordinates, which will not be repeated here.
[0058] In one embodiment, the calibration feature block also includes a plurality of positioning units, and the positioning holes are also used as an example for explanation. Of course, the type, quantity, and position of the positioning units can be set arbitrarily. Figure 5As shown, it also includes 2 positioning holes ; Preferably, it also includes two connecting holes. The positioning hole is used for guiding the assembly of the positioning pin, and the connecting hole is used for installing and fixing the calibration feature block and the rigid stable bracket. When the positioning hole in the calibration feature block adopts an asymmetric design, it is used to uniquely determine the position of the calibration feature block, that is, the calibration device is a structural design including a rigid stable bracket and a plurality of asymmetric calibration feature blocks.
[0059] The method of the present application also includes: obtaining the positioning coordinates of each positioning unit, such as the positioning circle center coordinates, through the first surface features of the current calibration feature block; matching the order between multiple first feature coordinates with the order between multiple feature units based on the relative distance between each positioning coordinate and each first feature coordinate.
[0060] Specifically, the terminal extracts the first surface feature Center positioning hole The terminal fits the center position of each positioning hole on the plane where the surface of the calibration feature block is located, and records the first positioning circle center coordinates of the positioning hole on the plane where the surface of the calibration feature block is located as , which is the first positioning coordinate. Among them, ,and and They are all in the form of a 3*1 matrix, and each row stores the x, y, and z coordinates of the center of the circle in sequence. When the positioning holes on the surface of the calibration feature block are designed asymmetrically, the terminal can locate the center coordinates of the circle. and the first characteristic circle center coordinates The relative position between them determines the order of multiple feature holes and adjusts the coordinates of the center of the first feature circle The order of storage in the storage device makes it Figure 5 Medium Feature Hole The order corresponds.
[0061] In one embodiment, determining a calibration matrix corresponding to a current spatial measurement point through a plurality of first characteristic coordinates and a plurality of second characteristic coordinates includes:
[0062] Respectively obtain the first centroid coordinates corresponding to the first characteristic coordinates and the second centroid coordinates corresponding to the second characteristic coordinates;
[0063] De-centering the first characteristic coordinates and the second characteristic coordinates according to the first centroid coordinates and the second centroid coordinates to obtain a plurality of de-centering first characteristic coordinates and a plurality of de-centering second characteristic coordinates;
[0064] Determining feature data when a plurality of de-centroided first feature coordinates and a plurality of de-centroided second feature coordinates are associated;
[0065] Determine the calibration matrix corresponding to the current spatial measurement point based on the feature data.
[0066] Similarly, the feature unit is a feature hole as an example for explanation. Specifically, the first feature center coordinate and the second feature center coordinate are in different coordinate systems. The terminal is for the first feature coordinate in the camera coordinate system , that is: the coordinates of the center of the first characteristic circle; and the coordinates of the second characteristic circle in the world coordinate system , that is, the coordinates of the second characteristic circle center, and the coordinates of the first centroid corresponding to the coordinates of the first characteristic circle center are obtained respectively The second centroid coordinates corresponding to the second characteristic circle center coordinates for:
[0067]
[0068]
[0069] in, Represents the number of elements of the characteristic unit in the set U. The terminal performs de-centroiding on the three-dimensional points in two different coordinate systems to obtain the de-centroided first characteristic coordinates and the de-centroided second eigencoordinate , taking the characteristic hole as an example, the first characteristic hole center coordinates and the second characteristic hole center coordinates. If the characteristic unit is a rectangle, triangle, ellipse, etc., its corner point, center, intersection of the major axis and the minor axis, etc. can be selected as the first characteristic coordinates and the second characteristic coordinates.
[0070] Specifically, the de-centroided first eigencoordinate and the de-centroided second eigencoordinate They are:
[0071]
[0072]
[0073] More preferably, the terminal may optionally but not limited to adopt the formula:
[0074]
[0075] Determine multiple de-centroided first feature coordinates and the de-centroided second eigencoordinate Feature data when associated , and determine the calibration matrix corresponding to the current spatial measurement point according to the characteristic data. It is worth noting that using covariance as characteristic data is only an example and is not limited to this.
[0076] In this embodiment, the first characteristic coordinate and the second characteristic coordinate are de-centred to obtain the de-centred first characteristic coordinate and the de-centred second characteristic coordinate. Taking the characteristic unit as a characteristic hole as an example, the position of the characteristic hole center relative to the center of mass is extracted by de-centred processing to eliminate the influence of the overall translation of different characteristic hole centers on the calibration result. Correlating multiple sets of characteristic hole center coordinates after de-centred processing can reduce the uncertainty caused by a single measurement error, so that when determining the calibration matrix later, feature point matching can be performed through asymmetric feature design.
[0077] In one embodiment, a calibration matrix corresponding to a current spatial measurement point is determined based on feature data, including: determining an eigenvector matrix after decomposing the feature data, and determining a rotation matrix corresponding to the current spatial measurement point through the eigenvector matrix; and obtaining a translation matrix through the first centroid coordinates, the second centroid coordinates and the rotation matrix.
[0078] Among them, the calibration matrix includes the rotation matrix and the translation matrix, that is, the calibration matrix is [ , ].
[0079] Specifically, the terminal obtains the characteristic data Perform singular value decomposition and get:
[0080]
[0081] in for The eigenvalue matrix of and for Therefore, the terminal determines the spatial measurement point through the eigenvector matrix. The rotation matrix between the camera coordinate system and the world coordinate system It can be expressed as:
[0082]
[0083] The terminal obtains the spatial measurement point through the first centroid coordinates, the second centroid coordinates and the rotation matrix. The translation matrix between the camera coordinate system and the world coordinate system , which can be expressed as:
[0084]
[0085] In this embodiment, by decomposing characteristic data, determining the rotation matrix and calculating the translation matrix, an accurate description of the rotation relationship and translation relationship between spatial measuring points can be achieved, thereby improving the accuracy and stability of the measurement and eliminating the influence of the rotation and translation between different spatial measuring points on the measurement results.
[0086] In one embodiment, the error index corresponding to each spatial measurement point is determined according to the calibration matrix, including: for the current spatial measurement point traversed to, the distance deviation associated with each first feature coordinate and the corresponding second feature coordinate is determined according to the calibration matrix corresponding to the current spatial measurement point; the number of feature units in the current calibration feature block is determined, and the error index corresponding to the current spatial measurement point is determined according to multiple distance deviations and the number of feature units.
[0087] Specifically, taking the characteristic hole as an example, after obtaining the spatial measurement point The corresponding rotation matrix in the calibration matrix With the translation matrix After that, the terminal calculates The transformed first feature coordinates The second characteristic coordinates corresponding to The distance deviation is:
[0088]
[0089] Where u represents the spatial measurement point The terminal determines the number of feature units in the current calibration feature block. , and determine the spatial measurement points based on the distance deviation and number of feature units of all feature units The average error of coordinate system transformation is:
[0090]
[0091] In this embodiment, by determining The distance deviation between each transformed first feature coordinate and the corresponding second feature coordinate represents the distance error between the actual measuring point and the ideal feature block. Through multiple distance deviations and the number of feature units, an average error that measures the accuracy of measuring point positioning can be determined so that the overall accuracy and error range of measuring point positioning can be evaluated. It is worth noting that the average error is only an adaptive example of the error index, and other error indicators such as variance and covariance can be optionally calculated through distance deviation for judgment.
[0092] In one embodiment, when the error index does not meet the preset condition, the calibration feature block data is updated, including: for each feature unit in the current calibration feature block, when the error index does not meet the preset condition, the maximum distance deviation is screened out from the distance deviations corresponding to each feature unit; the feature unit corresponding to the maximum distance deviation is eliminated to subsequently re-determine the calibration matrix. Specifically, the feature unit corresponding to the maximum distance deviation is deleted, and the number of remaining feature holes in the current calibration feature block is reduced by 1 to update the calibration feature block data.
[0093] Specifically, the terminal can preset a coordinate system conversion deviation threshold E to determine the spatial measurement point The coordinate system conversion error index, such as whether the average error is greater than the threshold E. If it is greater than the threshold E, it is considered that the current calibration matrix Therefore, the terminal needs to participate in the calculation of the calibration matrix Find the feature unit corresponding to the maximum distance deviation in the feature unit set U and get the feature unit number as:
[0094]
[0095] The terminal obtains the characteristic unit number After that, the feature unit is removed from the feature unit set U, so that The value is reduced by 1 accordingly to obtain the updated feature unit set U. Therefore, the terminal continues to use the updated feature unit set to determine the spatial measurement point Each first characteristic coordinate and each second characteristic coordinate are processed to remove the centroid, and then the spatial measurement point is obtained New calibration matrix, up to the spatial measurement point The coordinate system conversion error index is less than or equal to the preset threshold E, or the number of elements in the feature unit set U When the number is less than a preset number, such as 4, or other iteration termination conditions, such as time, are met, the loop stops.
[0096] In this embodiment, by considering that there may be some feature data collection anomalies or coordinate extraction errors in multiple feature units on the calibration feature block, a more accurate calibration matrix can be obtained by eliminating the corresponding feature point data and then iteratively calculating the surface features of the feature block.
[0097] In one embodiment, Figure 6As shown, it is a flow chart of the multi-robot joint calibration method in another embodiment, and the characteristic hole is used as an example for explanation. The terminal first determines the fixed installation position of the base of the multi-robot, and allocates the measurement point position of each robot, that is, the spatial measurement point that can be identified by the target acquisition device, and at the same time builds a calibration device according to the spatial measurement point, and collects the second surface feature of the calibration feature block through the reference acquisition device. For the current robot traversed to, all the spatial measurement points corresponding to the current robot and the running tracks corresponding to the multiple spatial measurement points are determined, so that the current robot traverses each spatial measurement point one by one. The first surface feature of the calibration feature block is obtained by the target acquisition device carried on the current robot, and multiple first feature circle center coordinates and multiple second feature circle center coordinates are extracted from the first surface feature and the second surface feature of the calibration feature block, and then the first feature hole center coordinate and the second feature hole center coordinate are determined. The terminal solves the coordinate system conversion matrix according to the first feature hole center coordinate and the second feature hole center coordinate, that is, the calibration matrix corresponding to the current spatial measurement point. The terminal determines the average error of each calibration feature block before and after coordinate transformation based on the calibration matrix, and when the average error is greater than the preset threshold, determines whether the number of feature holes is less than the preset number. If not, the feature holes corresponding to the maximum distance deviation are eliminated, and the calibration matrix corresponding to the current spatial measurement point is recalculated. If or the average error is less than or equal to the preset threshold, the calibration matrix corresponding to each of the final spatial measurement points is determined as the current spatial measurement point. The coordinate system transformation matrix of the spatial measurement point The terminal completes the spatial measurement point After the data is extracted, the robot j is instructed to move to the next spatial measuring point through the cable. , until all the spatial measurement points that the current robot j is responsible for have collected the data of the calibration feature blocks. After the terminal completes all the measurement points that robot j is responsible for, it updates the robot number and sets , until the coordinate system transformation relationship calculation of all m robots is completed and stored.
[0098] At this point, the conversion relationship between the camera coordinate system and the world coordinate system corresponding to the n spatial measurement points of the m robots is obtained and recorded, and the joint calibration of the multi-robot terminal pose is completed. In the actual measurement after the calibration, the corresponding external pose transformation matrix is used at each measurement point to convert the relative measurement value of the camera coordinate system into the overall absolute measurement value.
[0099] In one embodiment, the characteristic hole design form on the calibration characteristic block of the combined special calibration device in the present application can be replaced by a combination of other characteristic structural forms.
[0100] In one embodiment, the positioning hole design form on the asymmetric design of the calibration feature block of the combined special calibration device in the present application can be replaced by a combination of other structural forms.
[0101] In one embodiment, the calibration device of the present application can be replaced by other structural forms or the object to be measured itself, and the rigid stable bracket therein can be replaced by other supporting structures.
[0102] In one embodiment, the calculation process of solving the coordinates of feature points in the camera coordinate system and the coordinates of corresponding feature points in the world coordinate system in the present application can be replaced by the Gauss-Newton method.
[0103] In one embodiment, the coordinate matching method corresponding to the serial number of the characteristic hole in different coordinate systems of the present application can be replaced by other search methods such as traversal method.
[0104] It should be understood that, although the steps in the flowcharts involved in the above embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.
[0105] Based on the same inventive concept, the embodiment of the present application also provides a multi-robot joint calibration device for implementing the multi-robot joint calibration method involved above. The implementation solution provided by the device to solve the problem is similar to the implementation solution recorded in the above method, so the specific limitations in one or more multi-robot joint calibration device embodiments provided below can refer to the limitations of the multi-robot joint calibration method above, and will not be repeated here.
[0106] In one embodiment, a multi-robot joint calibration device is provided, comprising: a feature block determination module, a surface feature determination module and a calibration matrix determination module, wherein:
[0107] The feature block determination module is used to determine the calibration feature block located at each spatial measurement point in the calibration device.
[0108] The surface feature determination module is used to obtain the first surface feature of the calibration feature block through the target acquisition device when each robot determines the corresponding running trajectory according to multiple spatial measurement points; and obtain the second surface feature of each calibration feature block through the reference acquisition device.
[0109] The calibration matrix determination module is used to obtain the calibration matrix corresponding to each spatial measurement point according to the first surface feature and the second surface feature; determine the average error corresponding to each spatial measurement point according to the calibration matrix, and obtain the calibration result when the average error meets the preset condition.
[0110] Each module in the above multi-robot joint calibration can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each module.
[0111] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 7 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store the surface features of the calibration feature block. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a multi-robot joint calibration method is implemented.
[0112] Those skilled in the art will understand that Figure 7 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.
[0113] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.
[0114] In one embodiment, a computer-readable storage medium is provided, storing a computer program, which implements the steps in the above method embodiments when executed by a processor.
[0115] In one embodiment, a computer program product or computer program is provided, the computer program product or computer program includes computer instructions, the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the steps in the above-mentioned method embodiments.
[0116] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.
[0117] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0118] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.
Claims
1. A multi-robot joint calibration method, characterized in that: The end of each robot in the multi-robot is equipped with a target acquisition device; the method includes: Determine a calibration feature block located at each spatial measuring point in the calibration device; When each robot determines the corresponding running trajectory according to the multiple spatial measurement points, the first surface feature of the calibration feature block is obtained through the target acquisition device; Acquire the second surface feature of each calibration feature block by referring to the acquisition device to acquire data in the world coordinate system; According to the first surface feature and the second surface feature, a calibration matrix corresponding to each spatial measuring point is obtained; According to the calibration matrix, the error index corresponding to each spatial measurement point is determined, and when the error index meets the preset conditions, the calibration result is obtained; when the error index does not meet the preset conditions, the calibration feature block data is updated, and the calibration matrix is re-determined until the calibration matrix meets the preset conditions or reaches the iteration termination conditions, and the calibration result is obtained; Determining the error index corresponding to each spatial measurement point according to the calibration matrix, including: for the current spatial measurement point traversed to, determining the distance deviation associated with each first characteristic coordinate and the corresponding second characteristic coordinate according to the calibration matrix corresponding to the current spatial measurement point; The number of feature units in the current calibration feature block is determined, and the error index corresponding to the current spatial measurement point is determined based on multiple distance deviations and the number of feature units.
2. The method according to claim 1, characterized in that The calibration feature block includes a plurality of feature units; According to the first surface feature and the second surface feature, a calibration matrix corresponding to each spatial measurement point is obtained, including: For the current spatial measurement point traversed, the first feature coordinates of each feature unit are extracted through the first surface feature of the current calibration feature block; Extracting the second feature coordinates of each feature unit through the second surface feature of the current calibration feature block; A calibration matrix corresponding to the current spatial measurement point is determined through the multiple first characteristic coordinates and the multiple second characteristic coordinates.
3. The method according to claim 2, characterized in that The calibration feature block also includes a plurality of positioning units, and the method further includes: Extracting the positioning coordinates of each positioning unit through the first surface feature of the current calibration feature block; According to the relative distance between each positioning coordinate and each first characteristic coordinate, the order between the plurality of first characteristic coordinates is matched with the order between the plurality of characteristic units.
4. The method according to claim 2, characterized in that: Determining a calibration matrix corresponding to a current spatial measurement point through a plurality of first characteristic coordinates and a plurality of second characteristic coordinates includes: Respectively obtain the first centroid coordinates corresponding to the first characteristic coordinates and the second centroid coordinates corresponding to the second characteristic coordinates; De-centering the first characteristic coordinates and the second characteristic coordinates according to the first centroid coordinates and the second centroid coordinates to obtain a plurality of de-centering first characteristic coordinates and a plurality of de-centering second characteristic coordinates; Determining feature data when a plurality of de-centroided first feature coordinates and a plurality of de-centroided second feature coordinates are associated; Determine the calibration matrix corresponding to the current spatial measurement point based on the feature data.
5. The method according to claim 4, characterized in that The calibration matrix includes the rotation matrix and the translation matrix; Determine the calibration matrix corresponding to the current spatial measurement point based on the feature data, including: Determine the eigenvector matrix after the eigendata is decomposed, and determine the rotation matrix corresponding to the current spatial measurement point through the eigenvector matrix; The translation matrix is obtained through the first centroid coordinates, the second centroid coordinates and the rotation matrix.
6. The method according to claim 5, characterized in that The calculation formulas of the de-centroided first characteristic coordinates and the de-centroided second characteristic coordinates include: in, is the first characteristic coordinate, is the second characteristic coordinate, is the first centroid coordinate, is the coordinate of the second centroid, is the de-centroided first eigencoordinate, is the de-centroided second characteristic coordinate, is the number of elements in the feature unit set U.
7. The method according to claim 6, characterized in that Determine feature data The formula includes: in, Represents feature data.
8. The method according to claim 7, characterized in that The formula for determining the calibration matrix includes: in, for The eigenvalue matrix of and for The left and right eigenvector matrix of is the rotation matrix, is the translation matrix.
9. The method according to any one of claims 1 to 8, characterized in that: When the error index does not meet the preset condition, the calibration feature block data is updated, including: for each feature unit in the current calibration feature block, when the error index does not meet the preset condition, the maximum distance deviation is screened out from the distance deviations corresponding to each feature unit; The feature unit corresponding to the maximum distance deviation is eliminated.
10. A multi-robot joint calibration system, characterized in that: For executing the method according to any one of claims 1 to 9, the system comprises a terminal, multiple robots, a calibration device and a reference acquisition device, and the end of each robot in the multiple robots is equipped with a target acquisition device, wherein: A terminal, used to determine a calibration feature block located at each spatial measurement point in the calibration device; The target acquisition device is used to send the first surface feature of the acquired calibration feature block to the terminal when each robot determines the corresponding running trajectory according to the multiple spatial measurement points; A reference collection device is used to send the collected second surface features of each calibration feature block to the terminal; The terminal is used to obtain the calibration matrix corresponding to each spatial measurement point according to the first surface feature and the second surface feature; the terminal is used to determine the error index corresponding to each spatial measurement point according to the calibration matrix, and obtain the calibration result when the average error meets the preset condition; when the error index does not meet the preset condition, the calibration feature block data is updated, and the calibration matrix is re-determined until the calibration matrix meets the preset condition or reaches the iteration termination condition, and the calibration result is obtained.
11. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When a processor executes the computer program, the steps of the method according to any one of claims 1 to 9 are implemented.
