A quick calibration method for a connecting rod type dual-arm direct-drive manipulator

Abstract

The present application relates to the technical field of mechanical hand calibration, and especially to a quick calibration method of a connecting rod type double-arm direct-drive mechanical hand; when detecting joint image data and extracting feature point coordinates of the marker, a dynamic threshold Canny edge detection algorithm is used to detect the joint image data, the detection precision and robustness of the Canny edge detection algorithm to the reflective point marker are improved by combining local image statistical characteristics and a dynamic threshold adjustment strategy, and the algorithm is particularly suitable for scenes with large changes in illumination and uneven contrast; furthermore, an adjustment coefficient is determined based on global gradient, global variance and global average brightness of the image, that is, the influence of the local variance and the local average brightness of each pixel point in the image on the high and low thresholds is constrained by the global characteristics of the image, the sensitivity of the algorithm to different image characteristics can be flexibly controlled, and the dynamic threshold Canny edge detection algorithm is more flexible and universal.

Description

Technical Field

[0001] This invention relates to the field of robot calibration technology, and in particular to a rapid calibration method for a linkage-type dual-arm direct-drive robot. Background Technology

[0002] With the rapid development of industrial automation and intelligent manufacturing, robotic arms, as an important type of automated equipment, have been widely used in various fields such as manufacturing, logistics and warehousing, medical rehabilitation, and aerospace. Robotic arms can replace manual labor in completing repetitive, high-intensity, and high-precision tasks, improving production efficiency, product quality, and production safety. Among them, the linkage-type dual-arm direct-drive robotic arm has received particular attention due to its unique structural advantages, such as compact structure, flexible movement, high control precision, and fast response speed. Linkage-type dual-arm direct-drive robotic arms typically consist of multiple joints, each directly driven by a direct-drive motor, reducing transmission links and improving transmission efficiency and control precision. Its dual-arm structure enables the robotic arm to perform complex collaborative operations, such as dual-arm collaborative grasping and assembly. However, since the motion accuracy and repeatability of the robotic arm directly affect its working performance and application effect, precise calibration is essential before the robotic arm is put into use.

[0003] In existing technologies, there are solutions that attach reflective dots to the joints of a robotic arm and use Canny edge detection to obtain the joint coordinates of the robotic arm. However, for the specific application scenario of a linkage-type dual-arm direct-drive robotic arm, which has large lighting changes and uneven contrast, the traditional Canny algorithm results in insufficient accuracy in recognizing reflective dots and weak robustness.

[0004] Meanwhile, existing technologies employ deep learning models to obtain the theoretical position of the robot arm. This approach typically uses the Nesterov accelerated gradient method to train the deep learning model and optimize the training process. However, the optimization speed of the Nesterov accelerated gradient method does not meet the expected requirements, resulting in a long calibration time for the robot arm. Therefore, developing a fast, efficient, and high-precision calibration method is of great significance for improving the practicality and production efficiency of linkage-type dual-arm direct-drive robot arms. Summary of the Invention

[0005] To address the aforementioned technical problems, this invention provides a rapid calibration method for a linkage-type dual-arm direct-drive manipulator, which solves the problems existing in the prior art.

[0006] This invention provides a rapid calibration method for a linkage-type dual-arm direct-drive manipulator, comprising the following steps:

[0007] S1: Initialize the structural parameters of the linkage-type dual-arm direct-drive manipulator;

[0008] S2: Use a vision sensor to acquire the actual position data of the joints of the linkage-type dual-arm direct-drive manipulator;

[0009] In step S2, the joint image data is detected using the dynamic threshold Canny edge detection algorithm;

[0010] S3: Establish the kinematic model of the linkage-type dual-arm direct-drive manipulator;

[0011] The kinematic model is a deep neural network model. During the training process of the deep neural network model, the improved Nesterov accelerated gradient method is used to optimize the training process of the deep neural network model. An adaptive learning rate is introduced into the momentum term, and the momentum term is adjusted.

[0012] S4: Calculate the theoretical positions of the joints of the linkage-type dual-arm direct-drive manipulator based on the kinematic model of the manipulator.

[0013] S5: The linkage-type dual-arm direct-drive manipulator is calibrated based on the actual and theoretical positions of its joints.

[0014] Preferably, in step S2, acquiring the joint position data of the linkage-type dual-arm direct-drive manipulator using a vision sensor specifically involves:

[0015] S2.1: Install multiple vision sensors in the workspace of the linkage-type dual-arm direct-drive manipulator and calibrate them;

[0016] S2.2: Install markers on each joint of the linkage-type dual-arm direct-drive manipulator;

[0017] S2.3: Use a vision sensor to collect joint image data of the linkage-type dual-arm direct-drive manipulator;

[0018] S2.4: Detect the joint image data, extract the feature point coordinates of the marker, convert the feature point coordinates of the marker into world coordinates, and calculate the actual position data of the joint.

[0019] Preferably, the dynamic threshold Canny edge detection algorithm for detecting the joint image data specifically involves:

[0020] For each pixel in the joint image data, calculate the local variance σ and the local average brightness L in its neighborhood;

[0021] The high and low thresholds of the Canny algorithm are dynamically adjusted based on the local variance and average brightness of each pixel in its neighborhood.

[0022] Edge detection is performed on the joint image data using the calculated high and low thresholds.

[0023] Preferably, the high and low thresholds of the Canny algorithm are dynamically adjusted based on the local variance and average brightness of each pixel in its neighborhood, specifically as follows:

[0024] Set a base threshold T base Then, based on the local variance of each pixel s and average brightness L The relative size is used to adjust the high and low thresholds of the Canny algorithm;

[0025] Specifically, the high threshold T of the Canny algorithm is calculated using the following formula. high and low threshold T low :

[0026]

[0027] ;

[0028] In the formula, T base It is the basic threshold. k 1 and k 2 represents the adjustment coefficient used to adjust the influence of local variance and average brightness on the threshold. d for An adjustment coefficient that controls the difference between high and low thresholds.

[0029] Preferably, the adjustment coefficient is calculated. k 1 and k The specific steps in step 2 are as follows:

[0030] Calculate the global gradient G of the joint image;

[0031] Calculate the global average brightness of the joint image data. and global variance ;

[0032] Based on the global gradient G and global average brightness of the joint image and global variance Calculate the adjustment coefficient k 1 and k The value of 2.

[0033] Preferably, the calculation formula is as follows:

[0034]

[0035] ;

[0036] In the formula, c1 is a constant used to adjust the global gradient. k The degree of influence of 1 c 2 and c 3 is a constant used to adjust the global variance and global average brightness. k The extent of the impact of 2.

[0037] Preferably, S3 specifically comprises:

[0038] S3.1: Establish a deep neural network model;

[0039] S3.2 Obtain the dataset used for training the deep neural network model;

[0040] S3.3: Train the deep neural network model using the dataset.

[0041] Preferably, the formula for the improved Nesterov accelerated gradient method is:

[0042]

[0043] ;

[0044] In the formula, t is the number of iterations. v t Let be the momentum term in the t-th iteration. v t+1 This refers to the momentum term calculated before the (t+1)th iteration. β It is the momentum coefficient. or t It's the learning rate. It is the gradient of the loss function E. i t Indicates the first t Model parameters at the next iteration i t+1 It is in the t The model parameters updated at +1 iteration G t It is in the t The cumulative gradient during the next iteration. It is a constant.

[0045] Preferably, the learning rate or t Calculated in the following way:

[0046] .

[0047] The embodiments of the present invention have the following technical effects:

[0048] This invention detects joint image data, extracts the feature point coordinates of the markers, converts these coordinates into world coordinates, and calculates the actual position data of the joint. It then employs a dynamic threshold Canny edge detection algorithm to detect the joint image data. By combining local image statistical characteristics with a dynamic threshold adjustment strategy, the detection accuracy and robustness of the Canny edge detection algorithm for reflective markers are improved, making it particularly suitable for scenes with large illumination variations and uneven contrast. Furthermore, the adjustment coefficient is determined based on the global gradient, global variance, and global average brightness of the image. This means that the influence of the local variance and local average brightness of each pixel in the image on high and low thresholds is constrained by the global features of the image. This allows for flexible control of the algorithm's sensitivity to different image characteristics, making the dynamic threshold Canny edge detection algorithm more flexible and versatile.

[0049] During the training of the motion model, an improved Nesterov accelerated gradient method is adopted. An adaptive learning rate is introduced into the momentum term, and iterative optimization is achieved by adjusting the momentum term, i.e., through the accumulation of gradients. G t To adjust the learning rate or This approach allows for a decrease in the learning rate when gradient changes are large, and an increase in the learning rate when gradient changes are small. This method helps to achieve rapid convergence in the early stages of training and allows for more refined adjustments in the later stages. The goal is to better control the changes in the learning rate during training, thereby improving the efficiency and stability of optimization, and ultimately increasing the calibration speed of the linkage-type dual-arm direct-drive manipulator. Attached Figure Description

[0050] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0051] Figure 1 This is a flowchart of a rapid calibration method for a linkage-type dual-arm direct-drive manipulator provided in an embodiment of the present invention;

[0052] Figure 2 This is a flowchart of an embodiment of the present invention, which uses a visual sensor to acquire the joint position data of the linkage-type dual-arm direct-drive manipulator;

[0053] Figure 3 This is a flowchart illustrating the kinematic model of the linkage-type dual-arm direct-drive manipulator provided in an embodiment of the present invention. Detailed Implementation

[0054] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0055] Example 1, Appendix Figure 1 A flowchart illustrating a rapid calibration method for a linkage-type dual-arm direct-drive manipulator is shown in the attached diagram. Figure 1 As shown, a rapid calibration method for a linkage-type dual-arm direct-drive manipulator includes the following steps:

[0056] S1: Initialize the structural parameters of the linkage-type dual-arm direct-drive manipulator;

[0057] Before calibrating the linkage-type dual-arm direct-drive manipulator, its structural parameters need to be initialized. Specifically, the manipulator is placed in a known, repeatable initial position and posture to provide a reference for subsequent measurements. Simultaneously, the initial position and state information of each joint of the manipulator are acquired. These data will serve as reference data during the calibration process for subsequent calculations and error compensation. Initialization sets the manipulator's motion speed and joint angles within a safe range to avoid collisions or other accidents during calibration.

[0058] The structural parameters of the link-type dual-arm direct-drive manipulator include geometric parameters and kinematic parameters. The geometric parameters include link lengths, joint offsets, and link twist angles. The kinematic parameters include joint angles, joint velocities, and joint accelerations.

[0059] Furthermore, the initialization of the link length parameters of the link-type dual-arm direct-drive manipulator is specifically as follows: the actual length of each link is measured using a laser rangefinder, and the measured link length is recorded in the control system of the link-type dual-arm direct-drive manipulator as an initial parameter.

[0060] The initialization of the joint offset parameters of the linkage-type dual-arm direct-drive manipulator is specifically as follows: the offset of each joint is measured using a laser tracker, and the measured joint offset is recorded in the control system of the linkage-type dual-arm direct-drive manipulator.

[0061] The initialization of the link torsion angle parameters of the link-type dual-arm direct-drive manipulator is specifically as follows: the torsion angle of each link is measured using a laser tracker, and the measured link torsion angle is recorded in the control system of the link-type dual-arm direct-drive manipulator.

[0062] The initialization of the joint angle parameters of the linkage-type dual-arm direct-drive manipulator is specifically as follows: the control system of the linkage-type dual-arm direct-drive manipulator sets the angle of each joint to a known initial value (such as 0 degrees), and then records the set joint angle in the control system of the linkage-type dual-arm direct-drive manipulator.

[0063] The initialization of the joint speed and acceleration parameters of the linkage-type dual-arm direct-drive manipulator is specifically as follows: the speed and acceleration of each joint are set to a low-speed mode (such as 10% to 20% of the maximum speed) through the control system of the linkage-type dual-arm direct-drive manipulator, and then the set joint speed and acceleration are recorded in the control system of the linkage-type dual-arm direct-drive manipulator.

[0064] The above method allows for the accurate initialization of the structural parameters of the linkage-type dual-arm direct-drive manipulator. This parameter initialization ensures the accuracy and efficiency of subsequent calibration.

[0065] S2: Use a vision sensor to acquire the actual position data of the joints of the linkage-type dual-arm direct-drive manipulator;

[0066] Joint position measurement based on vision sensors involves acquiring image information of the robot's joint positions using vision sensors (such as cameras) and then calculating the precise joint positions using image processing algorithms. This method offers advantages such as being non-contact, highly accurate, and capable of real-time operation.

[0067] Specifically, as shown in the attached document Figure 2 As shown, the position data of the joints of the linkage-type dual-arm direct-drive manipulator obtained using a vision sensor are as follows:

[0068] S2.1: Install multiple vision sensors in the workspace of the linkage-type dual-arm direct-drive manipulator and calibrate them;

[0069] Multiple vision sensors are installed in the workspace of the robot to ensure that the field of view of the vision sensors can cover all joints of the robot. The vision sensors are calibrated by a calibration plate to determine the intrinsic parameters (such as focal length and principal point) and extrinsic parameters (such as the position and orientation of the sensor in the world coordinate system).

[0070] S2.2: Install markers on each joint of the linkage-type dual-arm direct-drive manipulator;

[0071] In this step, the marker can be a reflective dot. The marker is installed on each joint of the linkage-type dual-arm direct-drive manipulator to ensure that the position of the marker is fixed and easy to identify.

[0072] S2.3: Use a vision sensor to collect joint image data of the linkage-type dual-arm direct-drive manipulator;

[0073] In this step, the visual sensor is a binocular camera.

[0074] S2.4: Detect the joint image data, extract the feature point coordinates of the marker, convert the feature point coordinates of the marker into world coordinates, and calculate the actual position data of the joint;

[0075] In the detection of joint graphic data, extracting the markers in the joint graphic is crucial for extracting the coordinates of the markers. In the prior art, edge detection is a key step in extracting the precise position of reflective markers. Canny edge detection is a classic edge detection algorithm that is widely used due to its good detection effect and computational efficiency. However, for the specific application scenario of a linkage-type dual-arm direct-drive manipulator with large lighting changes and uneven contrast, the traditional Canny algorithm results in insufficient accuracy in the recognition of reflective points and weak robustness. Therefore, this embodiment proposes a dynamic threshold Canny edge detection algorithm to detect the joint image data in order to improve the detection accuracy and robustness of reflective markers on the joints of the linkage-type dual-arm direct-drive manipulator.

[0076] Specifically, the dynamic threshold Canny edge detection algorithm detects the joint image data as follows:

[0077] For each pixel in the joint image data, calculate the local variance σ and the local average brightness L in its neighborhood;

[0078] The size of the neighborhood is 3×3;

[0079] The high and low thresholds of the Canny algorithm are dynamically adjusted based on the local variance and average brightness of each pixel in its neighborhood.

[0080] One of them is to set a basic threshold. T base Then, based on the local variance of each pixel s and average brightness L The relative size is used to adjust the high and low thresholds of the Canny algorithm;

[0081] Specifically, the high threshold T of the Canny algorithm is calculated using the following formula. high and low threshold Tlow :

[0082]

[0083] ;

[0084] In the formula, T base It is the basic threshold. k 1 and k 2 represents the adjustment coefficient used to adjust the influence of local variance and average brightness on the threshold. d for An adjustment coefficient that controls the difference between high and low thresholds.

[0085] Edge detection is performed on the joint image data using the calculated high and low thresholds; this step is existing technology and will not be discussed in detail here.

[0086] Furthermore, this embodiment also provides a method for calculating the adjustment coefficient. k 1 and k Option 2 is as follows:

[0087] Calculate the global gradient G of the joint image;

[0088] The horizontal gradient of the joint image is calculated using the Sobel operator. G x and vertical gradient G y Then, the global gradient of the joint image is calculated; the specific formula is:

[0089] ;

[0090] Calculate the global average brightness of the joint image data. and global variance ;

[0091] Based on the global gradient G and global average brightness of the joint image and global variance Calculate the adjustment coefficient k 1 and k The value of 2.

[0092] The calculation formula is as follows:

[0093]

[0094] ;

[0095] In the formula, c 1 is a constant used to adjust the global gradient. k The degree of influence of 1 c 2 and c3 is a constant used to adjust the global variance and global average brightness. k The extent of the impact of 2.

[0096] By combining local image statistical characteristics with a dynamic threshold adjustment strategy, this scheme can improve the detection accuracy and robustness of the Canny edge detection algorithm for reflective markers, making it particularly suitable for scenes with large illumination variations and uneven contrast. Furthermore, the adjustment coefficient is determined based on the global gradient, global variance, and global average brightness of the image. That is, by constraining the influence of the local variance and local average brightness of each pixel in the image on the high and low thresholds through the global features of the image, the sensitivity of the algorithm to different image characteristics can be flexibly controlled, making the algorithm more flexible and versatile.

[0097] Coordinate transformation is the process of converting detected image coordinates into world coordinates to calculate the three-dimensional position of the joint. This process is a key step in mapping two-dimensional image information to three-dimensional space. In this step, the intrinsic and extrinsic parameters of the two cameras of the binocular camera are determined by using markers. Then, by matching the images of the two cameras, the disparity of the markers is calculated. The feature point coordinates of the markers are calculated using the disparity and the baseline distance of the cameras. The feature point coordinates of the markers are then converted into world coordinates to obtain the position data of the joint.

[0098] S3: Establish the kinematic model of the linkage-type dual-arm direct-drive manipulator;

[0099] Traditional manipulator kinematic models rely on precise geometric parameters and assumptions to establish the kinematic model. However, actual manipulators may have manufacturing and assembly errors. Therefore, in order to improve the adaptability and accuracy of the kinematic model, this embodiment adopts a data-driven method to establish the kinematic model. That is, a large amount of data on the end position and attitude of the linkage-type dual-arm direct-drive manipulator at different joint angles is collected. This data is used to train a data-driven kinematic model. The kinematic model uses a neural network model to establish the mapping relationship between the input (joint position data) and the output (end position and attitude).

[0100] In this step, the kinematic model of the linkage-type dual-arm direct-drive manipulator is a deep neural network (DNN) model. The DNN model trains a neural network model using a large amount of sample data, enabling it to quickly and accurately estimate the manipulator's kinematic parameters. This method is particularly suitable for complex nonlinear systems, automatically learning the mapping relationship between inputs (joint angles) and outputs (end-effector position and attitude), reducing the complex geometric modeling and parameter estimation steps required in traditional kinematic models.

[0101] Among them, as attached Figure 3 As shown, S3 specifically includes:

[0102] S3.1: Establish a deep neural network model;

[0103] The deep learning network model includes an input layer, a hidden layer, and an output layer; the number of neurons in the input layer is equal to the number of joint angles, n. For example, for a 6-joint linkage-type dual-arm direct-drive manipulator, the input layer has 6 neurons, corresponding to 6 joint angles.

[0104] The hidden layer is a key part of the deep learning network model. It is generally a multi-layer structure, with each layer containing multiple neurons. The activation function of the hidden layer is generally the ReLU function.

[0105] Furthermore, the hidden layer in this embodiment has a 3-layer architecture, with 128, 64 and 32 neurons in each layer, respectively.

[0106] The number of neurons in the output layer is equal to the dimensions of the end-effector position and orientation of the linkage-type dual-arm direct-drive manipulator. For example, for a 6-dimensional output [x, y, z, roll, pitch, yaw] of position and orientation, the output layer has 6 neurons; since the end-effector position and orientation of the linkage-type dual-arm direct-drive manipulator are continuous values, the activation function of the output layer is a linear activation function.

[0107] S3.2: Obtain the dataset used for training the deep neural network model;

[0108] Among them, the joint angles of the linkage-type dual-arm direct-drive manipulator under different working states are recorded. i =[ i 1, i 2,…, θn Furthermore, the position of the end effector of the linkage-type dual-arm direct-drive manipulator is recorded using high-precision measuring equipment (such as a vision sensor). x , y , z The data samples are [roll, pitch, yaw] and [pitch, yaw]. Sufficient data samples are collected to cover the workspace and range of motion of the robot, and these data samples are used as a dataset.

[0109] S3.3: Train the deep neural network model using the dataset;

[0110] The weights and biases of the deep neural network model are randomly set, the dataset used for training is input, the predicted value is calculated by the deep neural network model, the error between the predicted value and the true value is calculated using the loss function, the gradient is calculated by the backpropagation algorithm, the network parameters are updated, and the above process is repeated until the loss function converges or the preset number of training rounds is reached.

[0111] In the process of model training, optimizers are generally used to accelerate the convergence speed of the model. In the prior art, Nesterov gradient acceleration method is generally used for training. However, the optimization speed of the above method cannot meet the expected requirements during the optimization process. Therefore, this embodiment proposes an improved optimization algorithm to reduce the number of iterations.

[0112] Specifically, in order to further improve the performance of the Nesterov accelerated gradient method, this embodiment adopts an improved Nesterov accelerated gradient method, which introduces an adaptive learning rate into the momentum term and adjusts the momentum term.

[0113] The improved Nesterov accelerated gradient method is expressed as follows:

[0114]

[0115] ;

[0116] In the formula, t is the number of iterations. v t Let be the momentum term in the t-th iteration, representing the current "velocity". v t+1 The momentum term, calculated up to the (t+1)th iteration, is derived based on the parameters and gradient of the t-th iteration. β This is the momentum coefficient, which typically ranges from 0.8 to 0.99. or t It's the learning rate. It is the gradient of the loss function E. i t Indicates the first t Model parameters at the next iteration i t+1 It is in the t The model parameters updated at +1 iteration G t It is in the t The accumulated gradient over each iteration is used to calculate the learning rate. It is a very small constant used to prevent division by zero errors.

[0117] In each iteration, the improved Nesterov accelerated gradient method will adjust the gradient based on the current gradient. and previous momentum v t To calculate the new momentum v t+1 Then use this momentum to update the model parameters. i t+1This process will repeat until a stopping condition is met, such as reaching the maximum number of iterations or the value of the loss function no longer decreasing significantly.

[0118] Among them, learning rate or t Calculated in the following way:

[0119]

[0120] According to the above learning rate formula, the learning rate is dynamically adjusted based on each iteration t, that is, through the accumulation of gradients. G t To adjust the learning rate or This approach allows for a decrease in the learning rate when the gradient changes significantly, and an increase in the learning rate when the gradient changes slightly. This helps to achieve rapid convergence in the early stages of training and allows for finer adjustments in the later stages. The goal is to better control the changes in the learning rate during training, thereby improving the efficiency and stability of the optimization process.

[0121] Therefore, the process of training the deep neural network model using the dataset is as follows: by inputting the dataset into the deep neural network model, the deep neural network model can calculate the current parameters. i t Predicted values Then calculate the gradient of the loss function E. Next, update the gradient accumulator; after calculating the learning rate for each iteration t, update the momentum term. v t+1 and parameters i t+1 Thus, a training session was completed.

[0122] S4: Calculate the theoretical positions of the joints of the linkage-type dual-arm direct-drive manipulator based on the kinematic model of the manipulator.

[0123] S4 involves: acquiring the angle data of the joints of the linkage-type dual-arm direct-drive manipulator, using it as input data for the kinematic model of the linkage-type dual-arm direct-drive manipulator; the output of the kinematic model of the linkage-type dual-arm direct-drive manipulator is the theoretical position of the joints of the linkage-type dual-arm direct-drive manipulator; through the above process, the theoretical position of the manipulator's joints can be quickly acquired.

[0124] S5: The linkage-type dual-arm direct-drive manipulator is calibrated based on the actual and theoretical positions of its joints.

[0125] S5 involves: calculating the interpolation between the actual and theoretical positions of the joints of the linkage-type dual-arm direct-drive manipulator as the error of the control system of the linkage-type dual-arm direct-drive manipulator; adjusting the joint control parameters of the control system based on the error so that it can compensate for the error in subsequent movements, thereby achieving the calibration of the linkage-type dual-arm direct-drive manipulator.

[0126] Example 2: The present invention also provides an electronic device, including one or more processors and a memory.

[0127] A processor can be a central processing unit (CPU) or other form of processing unit with data processing and / or instruction execution capabilities, and can control other components in an electronic device to perform desired functions.

[0128] The memory may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and a processor may execute the program instructions to implement a rapid calibration method for a linkage-type dual-arm direct-drive manipulator as described above in any embodiment of this application, and / or other desired functions. Various contents such as initial extrinsic parameters and thresholds may also be stored in the computer-readable storage medium.

[0129] In one example, the electronic device may also include input and output devices, which are interconnected via a bus system and / or other forms of connection (not shown). The input device may include, for example, a keyboard, a mouse, etc. The output device may output various information to the outside, including warning messages, braking force, etc. The output device may include, for example, a display, a speaker, a printer, and a communication network and its connected remote output devices, etc.

[0130] Of course, for simplicity, components such as buses and input / output interfaces have been omitted. In addition, depending on the specific application, the electronic device may include any other appropriate components.

[0131] In addition to the methods and devices described above, embodiments of this application may also be computer program products, which include computer program instructions that, when executed by a processor, cause the processor to perform the function of a rapid calibration method for a linkage-type dual-arm direct-drive manipulator provided in any embodiment of this application.

[0132] The computer program product can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of this application. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0133] Furthermore, embodiments of this application may also be computer-readable storage media storing computer program instructions, which, when executed by a processor, cause the processor to implement a rapid calibration method for a linkage-type dual-arm direct-drive manipulator provided in any embodiment of this application.

[0134] The computer-readable storage medium may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.

[0135] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the technical solutions of the embodiments of the present invention.

Claims

1. A rapid calibration method of a linkage type dual-arm direct-drive manipulator, characterized by, The method comprises the following steps: S1: initializing the structure parameters of the linkage type double-arm direct drive manipulator; S2: acquiring actual position data of the joints of the linkage type double-arm direct drive manipulator by using a visual sensor; In the S2, a dynamic threshold Canny edge detection algorithm is used to detect the joint image data; specifically: For each pixel point in the joint image data, the local variance σ and the local average brightness L in its neighborhood are calculated; According to the local variance and average brightness of each pixel point in its neighborhood, the high threshold and low threshold of the Canny algorithm are dynamically adjusted; The calculated high threshold and low threshold are used to perform edge detection on the joint image data; According to the local variance and average brightness of each pixel point in its neighborhood, the high threshold and low threshold of the Canny algorithm are dynamically adjusted specifically as follows: Setting a base threshold T base Then adjust the high and low threshold of Canny algorithm according to the relative size of local variance σ and average brightness L of each pixel point. In particular, the high threshold T of the Canny algorithm is calculated using the following formula high and the low threshold T low : ； ； wherein T base is a base threshold, k 1 and k 2 are adjustment coefficients for adjusting the influence of local variance and average luminance on the threshold, d is an adjustment coefficient controlling the difference between the high and low thresholds. S3: establishing a kinematics model of the linkage type double-arm direct drive manipulator; The kinematics model is a deep neural network model, and in the training process of the deep neural network model, an improved Nesterov accelerated gradient method is used to optimize the training process of the deep neural network model, an adaptive learning rate is introduced into the momentum term, and the momentum term is adjusted; S4: calculating the theoretical position of the joints of the linkage type double-arm direct drive manipulator according to the kinematics model of the linkage type double-arm direct drive manipulator; S5: calibrating the linkage type double-arm direct drive manipulator according to the actual position and the theoretical position of the joints of the linkage type double-arm direct drive manipulator.

2. The rapid calibration method of the linkage type double-arm direct drive manipulator according to claim 1, wherein: In the S2, acquiring the position data of the joints of the linkage type double-arm direct drive manipulator by using a visual sensor specifically comprises: S2.1: installing a plurality of visual sensors in the working space of the linkage type double-arm direct drive manipulator and calibrating them; S2.2: installing markers on each joint of the linkage type double-arm direct drive manipulator; S2.3: collecting joint image data of the linkage type double-arm direct drive manipulator by using a visual sensor; S2.4: detecting the joint image data, extracting feature point coordinates of the markers, converting the feature point coordinates of the markers into world coordinates, and calculating the actual position data of the joints.

3. The rapid calibration method of the linkage type double-arm direct drive manipulator according to claim 2, wherein: Computing the adjustment coefficient k 1 and k 2 are as follows: calculating the global gradient G of the joint image; calculating a global mean intensity of the joint image data and a global variance ; Based on the global gradient G and global average brightness of the joint image and global variance Calculate the adjustment coefficient k 1 and k The value of 2; the calculation formula is: ； ； wherein c 1 is a constant to adjust the degree of influence of the global gradient on k 1, c 2 and c 3 are constants to adjust the degree of influence of the global variance and the global average luminance on k 2.

4. The rapid calibration method of the linkage type double-arm direct drive manipulator according to claim 1, wherein: The S3 specifically comprises: S3.1: establishing a deep neural network model; S3.2: acquiring a data set for training the deep neural network model; S3.3: training the deep neural network model by using the data set.

5. The rapid calibration method of the linkage type double-arm direct drive manipulator according to claim 1, wherein: The formula of the improved Nesterov accelerated gradient method is: ； ； where t is the iteration number, v t is the momentum term for the tth iteration, v t+1 is the momentum term calculated before the t+1th iteration, β is the momentum coefficient, η t is the learning rate, is the gradient of the loss function E, θ t denotes the model parameters at the tth iteration, t θ t+1 denotes the updated model parameters at the t+1th iteration, t G t denotes the accumulation of the gradient at the tth iteration, t is a constant.​​​ 6. The rapid calibration method of the linkage type double-arm direct drive manipulator according to claim 5, wherein: learning rate η t By the following way: 。

Images