A method for correcting the 3D placement error of a robot based on feature matching

Abstract

This invention belongs to the technical field of industrial image processing and discloses a method for correcting the three-dimensional placement error of a robot based on feature matching. The method includes the following steps: S1 capturing a reference image; S2 capturing an actual image; S3 comparing the actual image with the reference image and calculating the descriptor for each detection key point in each image; S4 calculating the two most matching detection key points in the reference image and the actual image; S5 calculating the positional deviation between the current position and the initial position of the robotic arm based on the relative position between the most matching detection key points, and moving the robotic arm according to the positional deviation; S6 repeating steps S1 to S5 a preset number of times to achieve the position correction of the robotic arm. This invention achieves the correction of three-dimensional automatic placement errors, improving the accuracy and stability of workpiece quality inspection.

Description

Technical Field

[0001] This invention belongs to the technical field of industrial image processing, and more specifically, relates to a method for correcting the three-dimensional placement error of a robot based on feature matching. Background Technology

[0002] In the industrial manufacturing sector, efficient and accurate workpiece quality inspection is crucial for ensuring product quality and improving production efficiency. With technological advancements, image processing technology has been widely applied in industrial product inspection. Industrial image processing technology, with its high speed and high precision, provides strong support for product quality control on automated production lines. In industrial product quality inspection, it is typically necessary to align different curved surfaces or specific locations of the product with a camera to obtain clear and accurate images for defect detection. However, in actual operation, due to factors such as workpiece placement, fixture fixation, and the movement of a six-axis multi-joint robotic arm, the workpiece may exhibit slight positional and orientation deviations. These minor deviations can lead to a decrease in the quality of the inspection images, thereby affecting the accuracy of subsequent defect detection.

[0003] To address this problem, researchers have proposed various methods, including laser detection, tactile detection, and image matching detection. Among these methods, feature matching has emerged as an effective approach. Feature matching technology extracts feature points from a reference image and the actual acquired image, and then matches these feature points to obtain the translation and rotation transformation relationships between the two images. By analyzing these transformation relationships, the positional and pose deviations between the images can be calculated.

[0004] Existing technologies primarily suffer from insufficient accuracy. For example, tactile methods use tactile sensors to detect contact between a robot and a workpiece or other objects. By measuring contact force and position, positional error can be estimated. However, this method can be affected by sensor accuracy and contact conditions. Similarly, laser-based methods use laser distance sensors or laser scanners to measure the position of a robot or workpiece. The accuracy of this method is affected by the performance of the laser sensor and environmental factors. Furthermore, current image matching methods have significant room for improvement, as they remain highly susceptible to environmental influences. Summary of the Invention

[0005] In view of the above-mentioned defects or improvement needs of the existing technology, the present invention provides a method for correcting the three-dimensional placement error of a robot based on feature matching, which solves the problems of insufficient accuracy and poor stability in the existing error correction process.

[0006] To achieve the above objectives, according to the present invention, a method for correcting the three-dimensional placement error of a robot based on feature matching is provided, the method comprising the following steps:

[0007] The S1 robotic arm carries a camera and moves to the initial position to capture an image of the workpiece as a reference image.

[0008] The S2 robotic arm moves and captures images of the workpiece during actual work to obtain actual images;

[0009] S3 compares the actual image with the reference image, inputs the two images into the neural network to obtain an intermediate feature map, selects the detection key points corresponding to each image in the intermediate feature map, and calculates the descriptor of each detection key point in each image respectively.

[0010] S4 calculates the matching score between each detection keypoint in the reference image and the actual image, and obtains the two most matching detection keypoints in the reference image and the actual image based on the matching score.

[0011] S5 calculates the positional deviation between the current position of the robotic arm and the initial position based on the relative positions between the most matched detection key points, and moves the robotic arm according to the positional deviation;

[0012] S6 repeats steps S1 to S5 a preset number of times to achieve the position correction of the robotic arm.

[0013] More preferably, in step S3, the selection of the detection key points is calculated according to the following probability formula:

[0014] P(x,y)=σ(f(x,y))

[0015] Where σ is the sigmoid activation function, f(x, y) is the output of the last convolutional layer for pixels x and y, and P(x, y) is the probability of the keypoint.

[0016] More preferably, in step S3, the descriptor calculation for the detected key points is performed according to the following formula:

[0017] D M (x,y)=VectorFusion(D mid (x,y),D grad (x,y),D diff (x,y))

[0018] VectorFusion is a complex vector fusion method, D mid (x, y) is the intermediate feature map output descriptor, D grad (x, y) is a multidimensional gradient descriptor, D diff (x, y) is the difference descriptor, D M (x, y) is the descriptor corresponding to each detection key point.

[0019] More preferably, the intermediate feature map output descriptor D mid (x,y) is obtained in the following way:

[0020] g i (z)=α i ×tanh(β i ×z+γ i )+δ i

[0021] Where z is the output value of the intermediate feature map, α i ,β i γ i δ i It is the function g i The parameter is tanh, which is the hyperbolic tangent function.

[0022] More preferably, the multidimensional gradient descriptor D grad (x, y) is obtained as follows: Select a 16x16 neighborhood around the keypoint and divide it into 4x4 sub-blocks; For each sub-block, calculate the gradient histogram in 8 directions to obtain the 8-dimensional vector corresponding to each sub-block; Concatenate the vectors of all sub-blocks to obtain the required multi-dimensional gradient descriptor.

[0023] More preferably, the difference descriptor D diff (x, y) is obtained in the following way:

[0024]

[0025] Among them, b j It detects the binary string of the j-th sub-block surrounding the keypoint. It is a concatenation operation, where N is the number of sub-blocks.

[0026] More preferably, in step S4, the matching score is calculated according to the following formula:

[0027] S total =FusionFunction(S cos S dot S euc )

[0028] Among them, S cos It is the cosine similarity score, S dot It is a point integral, S euc It is the Euclidean distance fraction, S total It's the matching score.

[0029] More preferably, the cosine similarity, point integral, and Euclidean distance score distribution are calculated according to the following formula:

[0030]

[0031] S dot (D M , D′ M ) = D M *D′ M

[0032]

[0033] Where W is a learned weight matrix, D M It is a descriptor from the reference image, D′ M It is a descriptor from the current image, and k is the length of the descriptor.

[0034] More preferably, after obtaining the matching score, the matching score needs to be normalized and converted into a matching probability. The matching probability is then compared with a preset threshold, and the two detection key points that meet the preset conditions are the most matching detection key points.

[0035] More preferably, in step S5, after calculating the position deviation, the position deviation needs to be transformed to the coordinate system of the robotic arm.

[0036] In summary, the technical solutions conceived by this invention have the following beneficial effects compared with the prior art:

[0037] 1. This invention proposes a method for correcting 3D automatic placement errors in industrial product inspection based on template feature matching, aiming to overcome the limitations of traditional methods. By employing a six-axis multi-joint robotic arm with a 2D industrial color camera at its end effector, after the workpiece is placed, the movement of the six-axis multi-joint robotic arm allows the camera to focus on and acquire images of various curved surfaces of the workpiece from different perspectives. Using the proposed MegaMatch method, the acquired images are matched with reference images to calculate the offset between them. Subsequently, based on the offset, the initial position and orientation of the workpiece are adjusted by controlling the six-axis multi-joint robotic arm, and multiple cycles are performed until the rotational and translational offset is less than a set threshold, thereby achieving the correction of 3D automatic placement errors.

[0038] 2. This invention employs the MegaMatch feature matching method for descriptor calculation, combining the calculation methods of intermediate feature map output descriptors, difference descriptors, and multi-dimensional gradient descriptors to obtain a ensemble descriptor. This comprehensive descriptor method ensures stable and accurate matching results under various conditions. Specifically, the intermediate feature map output descriptor captures global information of the image, the difference descriptor provides local detail information, and the multi-dimensional gradient descriptor provides directional information around key points. The combination of these three descriptors ensures stable matching results under different scales, rotations, and lighting conditions, because a single descriptor may not provide sufficient information for accurate matching in certain specific situations, while combining multiple descriptors provides more comprehensive information, thereby improving matching accuracy. Then, the MegaMatch feature matching method calculates three matching scores for this ensemble descriptor and fuses these scores to form a comprehensive matching score. This method utilizes multiple calculation methods such as cosine similarity, dot product, and Euclidean distance to ensure matching accuracy. Through this fusion method, we can obtain a more accurate and robust matching result. In general, this method not only improves the accuracy of image matching, but also ensures reliable matching results under various different scenarios and conditions.

[0039] 3. This invention provides a three-dimensional automatic placement error correction method for industrial product inspection using the MegaMatch method for image matching. This method overcomes the shortcomings of traditional methods in correcting workpiece placement errors. For example, methods based on tactile sensing or laser detection have low accuracy and are greatly affected by the environment. In addition, other feature matching methods may also have low matching accuracy due to environmental influences. By integrating image processing, machine vision, and mechanical control technologies, this method effectively improves the accuracy and stability of workpiece quality inspection, providing strong support for product quality control on industrial production lines. Attached Figure Description

[0040] Figure 1 This is a method for correcting the three-dimensional placement error of a robot based on feature matching, constructed according to a preferred embodiment of the present invention.

[0041] Figure 2 This is an actual error correction effect diagram constructed according to a preferred embodiment of the present invention. Detailed Implementation

[0042] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0043] like Figure 1 As shown, a method for automatic 3D placement error correction in industrial product inspection based on multi-method feature matching is presented. This method includes the following steps:

[0044] 1) Obtain the standard reference image:

[0045] Before placing the workpiece, a reference image needs to be acquired as a template image. This reference image will be used for subsequent image matching to calculate the translational and rotational offsets between different curved surfaces of the workpiece. First, a fixed position is determined as the initial position. Subsequent 3D defect detection uses this initial position to control a six-axis multi-joint robotic arm to acquire images of the cover at various angles. This ensures the uniformity of the acquired images; the image acquired at this position becomes the reference image.

[0046] 2) Workpiece placement and image acquisition:

[0047] The workpiece is placed in the fixture, and the workpiece is imaged at the current position of the six-axis multi-joint robotic arm using a six-axis multi-joint robotic arm and an industrial camera.

[0048] 3) Image matching:

[0049] One of the core steps of this invention is image matching to achieve automatic workpiece correction. To accurately calculate the translational and rotational offsets between images, a novel feature matching method, called the MegaMatch method, is employed for feature extraction, descriptor calculation, and feature matching. This method improves upon the descriptor calculation and feature matching capabilities of existing algorithms. By performing feature matching between the acquired image and a reference image, the translational and rotational offsets between the images are calculated. This algorithm can extract stable feature points at different scales and rotation angles, providing a reliable foundation for subsequent offset calculations.

[0050] A. MegaMatch descriptor calculation:

[0051] The MegaMatch neural network is primarily designed for detecting keypoints and computing their descriptors. Its architecture can be divided into the following parts:

[0052] 1. Input layer:

[0053] The network input is a grayscale image of size (H×W), where H is the height of the image and W is the width of the image.

[0054] 2. Convolutional layer:

[0055] The initial layers of the network are convolutional layers. These layers are responsible for extracting low-level features from the input image, such as edges, corners, and textures. Each convolutional layer is typically followed by a non-linear activation function, such as a rectified linear unit (ReLU).

[0056] 3. Intermediate feature map:

[0057] As image data is processed through convolutional layers, the spatial dimension decreases, but the depth (number of channels) increases. This results in a series of intermediate feature maps that capture increasingly abstract representations of the input image.

[0058] 4. Key point detection head:

[0059] This part of the network focuses on detecting keypoints in images. It processes intermediate feature maps and generates a heatmap of keypoint probabilities. Each pixel in this heatmap represents the probability that the location is a keypoint.

[0060] The keypoint probability P(x,y) is given by: P(x,y)=σ(f(x,y)), where σ is the sigmoid activation function and f(x,y) is the output of the last convolutional layer for pixels x and y.

[0061] 5. Descriptor Computation Header:

[0062] In the MegaMatch network, we employ a weighted combination method to merge multiple descriptors into a set descriptor.

[0063] a) Calculation of intermediate feature map output descriptors:

[0064] In parallel with the keypoint detection head, another part of the network computes descriptors for the detected keypoints. These descriptors capture local image information around each keypoint. The descriptor Dmid for a keypoint at location (x, y) is a vector whose components are computed as: Dmid i (x, y) = g i (f(x, y)). Where g i It is a function that processes the intermediate feature map output to compute the i-th component of the descriptor. Specifically:

[0065] g i (z)=α i ×tanh(β i ×z+γ i )+δ i

[0066] In the above formula: z is the output value of the intermediate feature map. α i ,β i γ i δ i It is the function g i The parameters, obtained through network training, are used to adjust the shape of the function to best fit the data. tanh is the hyperbolic tangent function, which maps input values ​​to between -1 and 1.

[0067] The choice of this transformation function is based on its ability to capture nonlinear characteristics while maintaining computational simplicity. This is achieved by adjusting the parameter α. i ,β i γ i δ i The network can learn an optimal transition function shape for each component.

[0068] b) Calculation of multidimensional gradient descriptors:

[0069] This descriptor assigns one or more orientations to each keypoint. This is achieved by calculating the direction and magnitude of the local image gradient around the keypoint. This ensures the descriptor's rotation invariance. At the scale of the keypoint, the gradients of the image in the x and y directions are calculated:

[0070]

[0071]

[0072] Here, I represents the grayscale value of the image. Using these gradient values, the gradient magnitude and direction for each pixel are calculated. A 16x16 neighborhood is selected around the keypoint and divided into 4x4 sub-blocks. For each sub-block, a gradient histogram is calculated in 8 directions. This results in an 8-dimensional vector for each sub-block. The vectors from all sub-blocks are concatenated to obtain a 128-dimensional descriptor vector: D grad = [d1, d2, ..., d 128 ].

[0073] c) Calculation of difference descriptors:

[0074] Constructing a nonlinear scale space: This descriptor constructs its scale space based on a diffusion method. This involves repeatedly applying a diffusion filter to the image at different scales. Mathematically, this can be represented as:

[0075]

[0076] Where L is the image at a given scale t, and c is the conductivity coefficient, and This is the gradient of the image. For each detected keypoint, the descriptor computes a local feature descriptor that calculates the gradient of the image in the x and y directions at the scale of the keypoint.

[0077]

[0078]

[0079] I represents the image grayscale value. For each pixel, the direction and magnitude of its gradient are calculated:

[0080]

[0081]

[0082] The gradients around the keypoints are then used to form a descriptor. This descriptor captures the differences between gradient value pairs in binary format. A direction is assigned to each pixel using the gradient direction. This is typically achieved by dividing the 360 ​​degrees into several directional intervals and assigning the gradient direction of each pixel to the nearest interval.

[0083] Select a region around the keypoint and divide it into multiple sub-blocks. For each sub-block, compare the gradient magnitude of the center pixel with that of its surrounding pixels. If the magnitude of the center pixel is greater than that of its neighbors, the bit is set to 1; otherwise, it is set to 0. In this way, each sub-block produces a binary string.

[0084] For a keypoint at position (x, y), its descriptor D diff (x, y) can be represented as:

[0085]

[0086] Among them, b j It is the binary string of the j-th sub-block. This represents a concatenation operation, where N is the number of sub-blocks. The binary representations of all sub-blocks are concatenated to form a long binary descriptor. This descriptor can be further converted into an integer vector, which is the difference descriptor.

[0087] d) Descriptor fusion

[0088] Given a keypoint location (x, y), we first obtain the corresponding descriptor using three methods:

[0089] D mid (x,y): Intermediate feature map output descriptor

[0090] D grad (x,y): Multidimensional gradient descriptor

[0091] Ddiff (x, y): Difference descriptor

[0092] We use vector fusion to combine these three descriptors, resulting in the combined descriptor D. M The formula for calculating (x, y) is:

[0093] D M (x, y) = VectorFusion(D mid (x, y), D grad (x, y), D diff (x, y))

[0094] VectorFusion is a complex vector fusion method, and its steps are as follows:

[0095] First, each input descriptor is L2 normalized to have a unit norm. This ensures that all descriptors are on the same scale.

[0096] Secondly, for each descriptor, we assign it a weight, which is determined based on its performance in past matching tasks. Then, we calculate the weighted average descriptor:

[0097] D avg =w mid ×D mid +w grad ×D grad +w diff ×D diff

[0098] Next, to enhance the characteristics of the descriptors, we perform element-wise multiplication between the weighted average descriptor and each original descriptor:

[0099] D fusion =D avg ⊙D mid ⊙D grad ⊙D diff

[0100] Finally, we perform L2 normalization again to obtain the final fusion descriptor D. M (x, y).

[0101] 6. Output:

[0102] The network produces two main outputs: a heatmap of keypoint probabilities and a set of descriptors for each detected keypoint.

[0103] B. MegaMatch Feature Matching:

[0104] MegaMatch feature matching takes as input a set descriptor by combining the calculation methods of intermediate feature map output descriptors, difference descriptors, and multi-dimensional gradient descriptors. Next, we calculate four matching scores for the set descriptor and fuse these scores to form a set matching score. The detailed steps and formulas are as follows:

[0105] 1. Input descriptor:

[0106] MegaMatch feature matching takes descriptors extracted from two images as input. Here, MegaMatch feature matching is used to extract the corresponding descriptors D from the reference image and the current image. M ,D′ M .

[0107] 2. Matching score calculation:

[0108] For each descriptor in the reference image, we need to calculate a matching score with each descriptor in the current image. This score is based on the similarity between the descriptors.

[0109] a) Cosine similarity score:

[0110] For each descriptor in the first image, the neural network calculates a matching score with each descriptor in the second image. This score is based on the similarity between descriptors, measured using a cosine similarity metric.

[0111]

[0112] Where W is a learned weight matrix, and D M It is a descriptor from the reference image, D′ M It is a descriptor from the current image.

[0113] b) Point integral:

[0114] S dot (D M , D′ M ) = D M *D′ M

[0115] c) Euclidean distance fraction:

[0116]

[0117] Where n is the length of the descriptor.

[0118] d) Combining the three matching scores:

[0119] S total =FusionFunction(Scos S dot S euc )

[0120] The steps for FusionFunction are as follows:

[0121] First, each matching score is normalized to the range [0,1].

[0122] Secondly, to enhance the diversity of scores, we apply a non-linear function, namely the sigmoid function, to each score.

[0123] Next, similar to the VectorFusion method, we assign a weight to each score and calculate a weighted average score:

[0124] S avg =w cos ×S cos +w dot ×S dot +w euc ×S euc

[0125] Finally, to further enhance the matching score, we perform element-wise multiplication of the weighted average score with each of the original scores:

[0126] S fusion =S avg ⊙S cos ⊙S dot ⊙S euc

[0127] 3. Softmax normalization:

[0128] The matching scores are normalized using the softmax function, resulting in a probability that sums to one for each descriptor in the reference image.

[0129]

[0130] Here, k is an index used to iterate through all descriptors in the current image. Specifically, when we normalize the match scores using the softmax function, we consider the match scores between a specific descriptor in the reference image and all descriptors in the current image.

[0131] 4. Match filtering:

[0132] After obtaining the matching probabilities, the MegaMatch algorithm filters out unlikely matches and determines the best matching pair:

[0133] Thresholding: Matches with a probability below a certain threshold are discarded.

[0134] Nearest Neighbor Matching: Only matches that are best matches for both descriptors are preserved. This ensures that a match is most likely for both descriptors. In other words, if descriptor A has the highest probability of matching with descriptor B in the reference image, and descriptor B also has the highest probability of matching with descriptor A in the reference image, then descriptor A and B are considered a valid match pair. This method can further reduce the possibility of false matches.

[0135] 5. Output:

[0136] The network produces two main outputs: a heatmap of matching scores, representing the matching relationship between the reference image and the current image; and multiple best-match pairs for subsequent processing.

[0137] By following the steps described above, the MegaMatch method can obtain a more accurate and robust matching result.

[0138] C. Affine transformation:

[0139] An affine transformation is a linear transformation that can represent operations such as translation, rotation, scaling, and shearing. To estimate an affine transformation from a matching pair, we need at least three non-collinear matching point pairs.

[0140] From the matching pairs, select three pairs of non-collinear points with the highest matching probability. Using these three pairs of points, set up a system of linear equations to solve for the six parameters of the affine transformation. Using these six parameters, we can construct a 2x3 affine transformation matrix.

[0141] The 2x2 submatrix in the top left corner of the affine matrix represents rotation and scaling. We can obtain the rotation angle by calculating the singular value decomposition of this submatrix. The last column of the affine matrix represents translation. This gives us the translation and rotation amounts between the images.

[0142] 4) Offset Calculation

[0143] The translation dx obtained through affine transformation p dy p The estimated rotation θ values ​​need to be transformed for application to the displacement adjustment of the six-axis multi-joint robot. We convert translation to displacement of the robot's end effector and rotation to joint angle adjustment. These transformations involve the mapping between the workspace coordinate system and the six-axis multi-joint robot coordinate system, and the specific calculation method is as follows:

[0144] Based on the calibrated offset parameters, the offset obtained from image matching (in pixels) is converted into an actual distance offset (in cm):

[0145] dx=k a ·dx p

[0146] dy = k b ·dy p

[0147] 1. Translation transformation: Convert the offset (dx, dy, dz) of translation in the workspace coordinate system into the displacement in the end-effector coordinate system of the six-axis multi-joint robot.

[0148] TCP adjusted [0] = tcp current [0]+dx

[0149] TCP adjusted [1] = tcpad current [1]+dy

[0150] TCP adjusted [2] = tcp curren t[2]+dz

[0151] Among them, TCP current These are the coordinates of the end effector of the six-axis multi-joint robotic arm at its current position.

[0152] 2. Rotation Transformation: This involves shifting the rotation by an angle (dθ) in the workspace coordinate system. x ,dθ y ,dθ z This can be converted into an angle adjustment function for each joint of a six-axis multi-joint robotic arm.

[0153] joint adjusted [0] = joint current [0]+dθ x

[0154] joint adjusted [1] = joint current [1]+dθ y

[0155] joint adjusted [2] = joint current [2]+dθ z

[0156] Among them, joint current This is the current joint angle of the six-axis multi-joint robotic arm.

[0157] 5) Six-axis multi-joint robotic arm control:

[0158] The aforementioned offset of the actual distance is converted into a displacement command for the six-axis multi-joint robotic arm through the inverse kinematic equation, and the initial position of the six-axis multi-joint robotic arm is adjusted to achieve three-dimensional automatic placement error correction.

[0159] 6) Repeatedly detect

[0160] To ensure accurate workpiece placement, we don't just perform a single offset detection and correction. Instead, we perform multiple iterative checks, fine-tuning each time based on the previous result. In each iteration, the translational and rotational offsets are calculated. The correction process only ends when these offsets are all below a predetermined threshold. This ensures that the workpiece's placement and orientation are very close to the expected reference position and orientation.

[0161] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for correcting the three-dimensional placement error of a robot based on feature matching, characterized in that, The method includes the following steps: S1 robotic arm carries a camera and moves to the initial position to capture an image of the workpiece as a reference image; The S2 robotic arm moves and captures images of the workpiece during actual work to obtain actual images; S3 compares the actual image with the reference image, inputs the two images into the neural network to obtain an intermediate feature map, selects the detection key points corresponding to each image in the intermediate feature map, and calculates the descriptor of each detection key point in each image respectively. S4 Calculate the matching score between each detection keypoint in the reference image and the actual image, and obtain the two most matching detection keypoints in the reference image and the actual image based on the matching score; S5 Calculates the positional deviation between the current position of the robotic arm and the initial position based on the relative positions between the most matched detection key points, and moves the robotic arm according to the positional deviation; S6 Repeat steps S1 to S5 a preset number of times to achieve the position correction of the robotic arm; In step S3, the descriptor calculation for the detected key points is performed according to the following formula: in, It is a complex vector fusion method. It is the intermediate feature map output descriptor. It is a multidimensional gradient descriptor. It is a difference descriptor. It is the descriptor corresponding to each key detection point; The intermediate feature map output descriptor Obtain it in the following ways: in, It is the output value of the intermediate feature map. It is a function The parameters, It is the hyperbolic tangent function. It is a process that processes intermediate feature map output to compute the first descriptor. Functions of each component; The multidimensional gradient descriptor The following steps are taken: Select a 16x16 neighborhood around the keypoint and divide it into 4x4 sub-blocks; for each sub-block, calculate the gradient histogram in 8 directions to obtain an 8-dimensional vector for each sub-block; concatenate the vectors of all sub-blocks to obtain the required multi-dimensional gradient descriptor. The difference descriptor Obtain it in the following ways: in, It detects the binary string of the j-th sub-block surrounding the keypoint. It is a concatenation operation, where N is the number of sub-blocks.

2. The method for correcting the three-dimensional placement error of a robot based on feature matching as described in claim 1, characterized in that, In step S3, the selection of the detection key points is calculated according to the following probability formula: in, It is the sigmoid activation function. It is a pixel The output of the last convolutional layer, It is the probability of being a key point.

3. The method for correcting the three-dimensional placement error of a robot based on feature matching as described in claim 1, characterized in that, In step S4, the matching score is calculated according to the following formula: in, It is the cosine similarity score. It is a point integral. It is a Euclidean distance fraction. It's the matching score.

4. The method for correcting the three-dimensional placement error of a robot based on feature matching as described in claim 3, characterized in that, The cosine similarity, point integral, and Euclidean distance score distributions are calculated according to the following formulas: in, It is a learned weight matrix. It is a descriptor from the reference image. It is a descriptor from the current image. k It is the length of the descriptor.

5. A method for correcting the three-dimensional placement error of a robot based on feature matching as described in claim 3 or 4, characterized in that, After obtaining the matching score, the matching score needs to be normalized and converted into a matching probability. The matching probability is then compared with a preset threshold, and the two detection key points that meet the preset conditions are the most matching detection key points.

6. The method for correcting the three-dimensional placement error of a robot based on feature matching as described in claim 1, characterized in that, In step S5, after calculating the position deviation, the position deviation needs to be transformed to the coordinate system of the robotic arm.

Images