Method and system for correcting accurate position of mechanical arm in four-dimensional space

Abstract

The invention discloses a mechanical arm four-dimensional space accurate position correction method which is characterized by comprising the following steps: S1, respectively adjusting a mechanical arm to a standard working position and a quasi working position for shooting to correspondingly obtain a standard correction pattern and a to-be-corrected pattern, and performing image processing on the standard correction pattern and the to-be-corrected pattern; s2, obtaining three-dimensional coordinates and perimeters of the standard correction pattern and the to-be-corrected pattern, wherein the three-dimensional coordinates specifically comprise two-dimensional plane coordinates of a pattern center point and rotation angle coordinates of the pattern; and S3, performing three-dimensional coordinate correction and depth position correction on the quasi working position by comparing the three-dimensional coordinates and perimeters of the standard correction pattern and the pattern to be corrected.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of robots, in particular to a mechanical arm four-dimensional space accurate position correction method and system. BACKGROUND

[0002] In the field of industrial automation, the basic principle of mechanical arm programming control is to use computer language to write programs, and to convert the designed operation steps into specific mechanical arm movement trajectories. The movement step is one of the key parameters that need to be set in programming control. Due to the control error of the mechanical arm, as well as the mechanical deviation of the installed and maintained parts themselves, in the working scene with high precision requirements, the mechanical arm needs to be further corrected and adjusted after completing the movement according to the programmed movement step. The traditional mechanical arm operation position correction method has problems such as high cost, complex operation, poor real-time performance and insufficient precision. In recent years, with the development of machine vision technology, using image processing technology for position correction has become a new solution, but the existing methods still have deficiencies in all-around and multi-dimensional correction.

[0003] In the field of robot technology, especially in the aspect of accurate position correction for mechanical arm installation work, various technical methods have been proposed and applied. Traditional mechanical arm position correction methods usually rely on high-precision sensors such as laser sensors and encoders. These methods correct by measuring the deviation between the actual position of the mechanical arm and the preset position. However, these methods have the following shortcomings:

[0004] High cost: High-precision sensors are usually expensive, increasing the overall cost of the system.

[0005] Complex operation: The installation, debugging and maintenance of sensors require professional technicians, increasing the complexity and time cost of operation.

[0006] Poor real-time performance: Some sensors have delays in data acquisition and processing, making it difficult to meet the real-time correction requirements.

[0007] In recent years, with the development of machine vision technology, some vision-based correction methods have been applied to the position correction of mechanical arms. However, the existing vision correction methods still have deficiencies in comprehensiveness and accuracy:

[0008] Dimensional limitation: Some vision correction methods can only achieve two-dimensional plane position (X, Y axis), limited or unable to achieve correction ability in the depth direction (Z axis) and the rotation angle (α) in the plane.

[0009] Insufficient accuracy: Due to the limitations of image processing algorithms and hardware devices, some vision correction methods are difficult to achieve high-precision installation requirements in terms of accuracy.

[0010] Complex algorithms: Some visual correction methods use complex image processing algorithms, which require high computing resources and increase the complexity and cost of the system.

[0011] There are also some hybrid correction methods that combine the advantages of sensor correction and visual correction to try to improve the accuracy and real-time performance of correction. However, these methods still have the following problems:

[0012] Complex system: Hybrid correction methods usually require the integration of multiple sensors and visual devices, which increases the complexity of the system and the difficulty of maintenance.

[0013] High cost: Integrating multiple sensors and visual devices undoubtedly increases the overall cost of the system.

[0014] Poor compatibility: Compatibility and data fusion issues between different sensors and visual devices are also challenges that hybrid correction methods need to face. Summary of the Invention

[0015] In order to solve these problems, the present invention proposes a method and system for precise four-dimensional spatial position correction of a robotic arm.

[0016] According to one aspect of the present invention, a method for accurately correcting the position of a robotic arm in four-dimensional space is proposed, comprising the following steps:

[0017] S1, adjusting the robotic arm to a standard working position and a quasi-working position to respectively take pictures, obtaining a standard calibration pattern and a pattern to be calibrated, and performing image processing on the standard calibration pattern and the pattern to be calibrated;

[0018] S2, obtaining the three-dimensional coordinates and perimeter of the standard calibration pattern and the pattern to be calibrated, wherein the three-dimensional coordinates specifically include the two-dimensional plane coordinates of the center point of the pattern and the rotation angle coordinates of the pattern;

[0019] S3, performing three-dimensional coordinate correction and depth position correction on the quasi-working position by comparing the three-dimensional coordinates and perimeters of the standard correction pattern and the pattern to be corrected.

[0020] Specifically, S1 specifically includes performing inverse binarization processing on the standard correction pattern and the pattern to be corrected to separate the standard correction pattern and the pattern to be corrected from the background, then performing image edge detection, extracting the edge information of the standard correction pattern and the pattern to be corrected, and performing contour detection based on the edge information of the standard correction pattern and the pattern to be corrected to obtain the contour point set of the standard correction pattern and the pattern to be corrected.

[0021] Specifically, S2 specifically comprises fitting a minimum rectangular frame that can include the contour point set of the standard correction pattern and the contour point set of the to-be-corrected pattern by least square method, so as to calculate the three-dimensional coordinates and the circumference of the standard correction pattern and the to-be-corrected pattern.

[0022] Specifically, it also comprises calculating the two-dimensional plane coordinates of the positioning points of the to-be-corrected pattern, judging whether a large directional deviation occurs in the quasi-working position by comparing the two-dimensional plane coordinates of the center point of the to-be-corrected pattern with the two-dimensional plane coordinates of the positioning points of the to-be-corrected pattern, if it is judged that a large directional deviation occurs, an error is reported, and if it is judged that a large directional deviation does not occur, S3 is continuously executed.

[0023] Specifically, S3 specifically comprises the following steps:

[0024] S301, rotation angle correction is performed, an angle error range threshold value is set, the quasi-working position is readjusted based on the rotation angle coordinate difference value of the standard correction pattern and the to-be-corrected pattern, and the readjustment is continued until the rotation angle coordinate difference value is less than the angle error range threshold value;

[0025] S302, two-dimensional plane coordinate correction of the center point is performed, a two-dimensional plane coordinate offset threshold value is set, the quasi-working position is readjusted based on the two-dimensional plane coordinate difference value of the standard correction pattern and the to-be-corrected pattern, and the readjustment is continued until the two-dimensional plane coordinate difference value is less than the two-dimensional plane coordinate offset threshold value;

[0026] S303, longitudinal position correction is performed, a longitudinal position offset threshold value is set, the quasi-working position is readjusted based on the circumference difference value of the standard correction pattern and the to-be-corrected pattern, and the readjustment is continued until the circumference difference value is less than the longitudinal position offset threshold value.

[0027] According to an aspect of the present application, a mechanical arm four-dimensional space accurate position correction system is provided, comprising the following modules:

[0028] An image processing module is configured to adjust the mechanical arm to a standard working position and a quasi-working position respectively for shooting, so as to obtain a standard correction pattern and a to-be-corrected pattern, and perform image processing on the standard correction pattern and the to-be-corrected pattern.

[0029] A parameter calculation module is configured to obtain the three-dimensional coordinates and the circumference of the standard correction pattern and the to-be-corrected pattern, and the three-dimensional coordinates specifically include the two-dimensional plane coordinates of the center point of the pattern and the rotation angle coordinates of the pattern.

[0030] A position correction module is configured to perform three-dimensional coordinate correction and longitudinal position correction on the quasi-working position by comparing the three-dimensional coordinates and the circumference of the standard correction pattern and the to-be-corrected pattern.

[0031] Specifically, the image processing module is specifically configured to perform inverse binarization processing on the standard correction pattern and the to-be-corrected pattern to separate the standard correction pattern and the to-be-corrected pattern from a background, perform image edge detection, extract edge information of the standard correction pattern and the to-be-corrected pattern, and perform contour detection based on the edge information of the standard correction pattern and the to-be-corrected pattern to obtain a contour point set of the standard correction pattern and the to-be-corrected pattern.

[0032] Specifically, the parameter calculation module is specifically configured to fit a minimum rectangular frame that can include the contour point set by using a least square method based on the contour point set of the standard correction pattern and the to-be-corrected pattern, so as to calculate three-dimensional coordinates and a perimeter of the standard correction pattern and the to-be-corrected pattern.

[0033] Specifically, the parameter calculation module further includes calculating two-dimensional plane coordinates of the positioning points of the to-be-corrected pattern, judging whether a large directional deviation occurs in the quasi-working position by comparing the two-dimensional plane coordinates of the center point of the to-be-corrected pattern with the two-dimensional plane coordinates of the positioning points of the to-be-corrected pattern, if it is judged that a large directional deviation occurs, performing error reporting, and if it is judged that a large directional deviation does not occur, continuing to execute the position correction module.

[0034] Specifically, the position correction module specifically includes the following modules:

[0035] The rotation angle correction module is configured to perform rotation angle correction, set an angle error range threshold value, readjust the quasi-working position based on a rotation angle coordinate difference value of the standard correction pattern and the to-be-corrected pattern until the rotation angle coordinate difference value is less than the angle error range threshold value.

[0036] The two-dimensional plane coordinate correction module is configured to perform two-dimensional plane coordinate correction of the center point, set a two-dimensional plane coordinate offset threshold value, readjust the quasi-working position based on a two-dimensional plane coordinate difference value of the standard correction pattern and the to-be-corrected pattern until the two-dimensional plane coordinate difference value is less than the two-dimensional plane coordinate offset threshold value.

[0037] The longitudinal position correction module is configured to perform longitudinal position correction, set a longitudinal position offset threshold value, and readjust the quasi-working position based on a perimeter difference value of the standard correction pattern and the to-be-corrected pattern until the perimeter difference value is less than the longitudinal position offset threshold value.

[0038] The present application has the advantages of:

[0039] (1) High real-time performance: real-time correction is realized through machine vision technology, and production efficiency is improved.

[0040] (2) High precision: the position deviation is accurately calculated by using image processing technology, and the correction precision is improved.

[0041] (3) Low cost: the algorithm has low requirements for edge computing power, reducing system cost.

[0042] (4) Comprehensive correction: realize comprehensive correction in X, Y, alpha and Z axis directions, meet complex production requirements.

[0043] (5) Good versatility: the correction algorithm is not affected by specific installation and maintenance tasks, and can be used in various tooling fixture operation scenarios. BRIEF DESCRIPTION OF DRAWINGS

[0044] The accompanying drawings are included to provide a further understanding of embodiments and are incorporated in and constitute a part of this specification. The drawings illustrate embodiments and serve to explain principles of the present application. Other embodiments and many of the intended advantages of the present application will be readily appreciated as the same becomes better understood by reference to the following detailed description. The elements of the drawings are not necessarily to scale relative to each other. Like reference numerals designate corresponding similar parts.

[0045] Figure 1 A flowchart of a mechanical arm four-dimensional space accurate position correction method according to the present application is shown;

[0046] Figure 2 A correction pattern, a ring light source, a mechanical arm and an industrial camera fixedly connected thereto for correction in the present application are shown;

[0047] Figure 3 A coordinate system diagram of a correction image in the present application is shown;

[0048] Figure 4 A schematic diagram of the rotation angle of the correction pattern in the present application is shown;

[0049] Figure 5 A flowchart of the rotation angle correction according to the method of the present application is shown;

[0050] Figure 6 A flowchart of the two-dimensional plane coordinate correction of the center point according to the method of the present application is shown;

[0051] Figure 7 A schematic diagram of the two-dimensional plane coordinate correction of the center point according to the method of the present application is shown;

[0052] Figure 8 A flowchart of the depth position correction according to the method of the present application is shown;

[0053] Figure 9 A structural diagram of a mechanical arm four-dimensional space accurate position correction system according to the present application is shown. DETAILED DESCRIPTION

[0054] The application will be further described below in detail with reference to the drawings and embodiments. It can be understood that the specific embodiments described herein are only used to explain the related application, and not to limit the application. In addition, it should be noted that only the parts related to the application are shown in the drawings for ease of description.

[0055] It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict. The present application will be described in detail below with reference to the drawings and embodiments.

[0056] Figure 1 A mechanical arm four-dimensional space accurate position correction method is shown, comprising the following steps:

[0057] S1, respectively adjusting the mechanical arm to a standard working position and a quasi-working position for shooting, corresponding to obtain a standard correction pattern and a to-be-corrected pattern, and performing image processing on the standard correction pattern and the to-be-corrected pattern;

[0058] S2, obtaining three-dimensional coordinates and a circumference of the standard correction pattern and the to-be-corrected pattern, the three-dimensional coordinates specifically including two-dimensional plane coordinates of a pattern center point and a rotation angle coordinate of the pattern;

[0059] S3, correcting the quasi-working position in three-dimensional coordinates and longitudinal position by comparing the three-dimensional coordinates and the circumference of the standard correction pattern and the to-be-corrected pattern.

[0060] Figure 2 (a) shows a correction pattern for correction, which is fixedly installed or attached on a workbench. On the pattern, a rectangular frame, a positioning point and a rectangular center point are designed. Figure 2 (b) shows a ring-shaped light source. The light source is used to ensure that the industrial camera can form a relatively clear and stable image. In the case that the installation space is not allowed, a bar-shaped light source can also be added. Since the present scheme will perform denoising and binarization on the image, the requirement for the light source is not very high. Figure 2 (c) shows a mechanical arm and an industrial camera solidly connected thereto. The basic motion control of the mechanical arm is completed through a control program to reach the quasi-working position. Then, the imaging and processing of the correction pattern by the industrial camera are realized to correct the working position more accurately, so that the mechanical arm further reaches the standard working position.

[0061] The coordinate system of the correction image is as follows Figure 3As shown, the upper left corner is the (0, 0) coordinate, and the lower right corner is (w, h), where w and h are the resolution of the camera. The coordinate positions mentioned below comply with the following coordinate system definition. The red dot is the positioning point of the correction pattern, and the black dot is the center point of the correction pattern.

[0062] When the robot arm moves to the standard working position and the quasi-working position through the control program, the image of the correction pattern at the current working position is collected by the industrial camera, and further image processing such as image pseudo-binarization, image edge detection, image contour detection and fitting is performed to realize the correction of the rotation angle α, the two-dimensional plane position (X, Y axis) correction and the longitudinal direction Z axis coordinate correction.

[0063] The rotation angle α is the included angle between the upper long side of the rectangle (the long side close to the positioning point) and the x axis, ranging from -90° to +90°. The specific positive and negative directions of the rotation angle are as follows:

[0064] ① If α = 0°, one side of the rectangle is parallel to the x axis (for example, Figure 4 (a)).

[0065] ② If α is negative (α = -15°), the rectangular frame has a counterclockwise rotation relative to the X axis (for example, Figure 4 (b)).

[0066] ③ If α is positive (α = +15°), the rectangular frame has a clockwise rotation relative to the X axis (for example, Figure 4 (c)).

[0067] ④ In order to reduce the installation difficulty of the correction pattern, the standard working position of the correction pattern is imaged, and the rotation angle (α0) is not required to be 0. It is only required that the positioning point of the correction pattern is at the upper left corner of the center point of the rectangular frame, that is, the two sides of the rectangular frame of the correction pattern do not require to be parallel to the X axis and Y axis of the imaging coordinate system, but can have a certain included angle.

[0068] Specifically, S1 specifically includes performing inverse binarization processing on the standard correction pattern and the correction pattern to be corrected to separate the standard correction pattern and the correction pattern to be corrected from the background, performing image edge detection, extracting edge information of the standard correction pattern and the correction pattern to be corrected, and performing contour detection based on the edge information of the standard correction pattern and the correction pattern to be corrected to obtain a contour point set of the standard correction pattern and the correction pattern to be corrected.

[0069] Wherein, the specific method of inverse binarization processing is as follows: let the gray value of the pixel point in the image be I(x, y), and the threshold value of inverse binarization be T. The calculation formula is:

[0070]

[0071] T = 100 is selected by analyzing the gray histogram of the image. After inverse binarization, the correction pattern and the background in the image are clearly separated.

[0072] In a specific embodiment, the Canny operator is used to detect the edge of the image after inverse binarization. The Canny operator contains two thresholds, high threshold Th and low threshold Tl. In this embodiment, according to the characteristics of the image and the requirements of edge detection, the high threshold Th = 100 and the low threshold Tl = 50 are selected. First, the Gaussian filter is used to smooth the image to reduce noise interference. The kernel size of the Gaussian filter is set to 3x3, and the standard deviation σ = 1.0. Then, the gradient amplitude and direction of each pixel point in the image are calculated. Next, the non-maximum suppression and double threshold detection are performed on the gradient amplitude according to the high threshold and the low threshold, so as to obtain the edge information of the image. After Canny operator edge detection, the edge of the correction pattern in the image is clearly extracted.

[0073] Then, the edge detected image is subjected to contour detection. The findContours function in the OpenCV library is used to realize the contour detection. This function can find all the closed contours in the image and return the point set of each contour. In this embodiment, the contour retrieval mode is set to RETR_EXTERNAL, that is, only the outermost contour is detected; the contour approximation method is CHAIN_APPROX_SIMPLE, that is, the contour is compressed and only the key points of the contour are saved. Through contour detection, the contour point set of the correction pattern can be obtained.

[0074] Specifically, S2 specifically comprises fitting a minimum rectangular frame that can include the contour point set based on the standard correction pattern and the contour point set of the to-be-corrected pattern by least square method, so as to calculate the three-dimensional coordinates and the circumference of the standard correction pattern and the to-be-corrected pattern.

[0075] In a specific embodiment, the minAreaRect function in the OpenCV library is used to fit the minimum rectangular frame for the detected contour. This function fits the minimum rectangular frame that can completely surround the contour by least square method.

[0076] Let the length of the minimum rectangular frame be L, the width be W, and the rotation angle be α (with the image X axis as the reference, counterclockwise direction is positive). By calling the minAreaRect() function, the return value rect = ((x c ,y c ), (w, h), θ) is obtained. In rect, (x c ,y c ) is the center of the minimum rectangular frame, (w, h) is the size of the minimum rectangular frame, and θ is the rotation angle of the minimum rectangular frame. c c c c) is the center coordinate of the standard correction pattern or the to-be-corrected pattern, and (w, h) is the width and height of the standard correction pattern or the to-be-corrected pattern, where w is not necessarily greater than h, and needs to be combined with the rotation angle θ of the rectangular frame, and the conversion relationship formula of θ and α is as follows:

[0077]

[0078] The rotation angle α0 of the standard correction pattern relative to the X axis and the rotation angle α1 of the to-be-corrected pattern relative to the X axis can be calculated.

[0079] At the same time, the arcLength() function in the OpenCV library can be used to calculate the perimeter len0 of the standard correction pattern and the perimeter len1 of the to-be-corrected pattern.

[0080] Specifically, the two-dimensional plane coordinates of the positioning point of the to-be-corrected pattern are calculated, and whether a large directional deviation occurs in the quasi-working position is determined by comparing the two-dimensional plane coordinates of the center point of the to-be-corrected pattern and the two-dimensional plane coordinates of the positioning point of the to-be-corrected pattern. If it is determined that a large directional deviation occurs, an error is reported, and if it is determined that a large directional deviation does not occur, S3 is continued.

[0081] The relative relationship between the rectangular center point coordinates (x1, y1) and the positioning point coordinates (x lp1 ,y lp1 ) of the correction pattern image needs to be compared. When the conditions (x1>x lp1 ) and (y1>y lp1 ) are met, it indicates that the positioning point of the to-be-corrected pattern in the quasi-working position is at the upper left corner of the rectangular frame center point, so as to avoid directional errors during the rotation angle α correction.

[0082] Specifically, S3 specifically includes the following steps:

[0083] S301, rotation angle correction is performed, an angle error range threshold is set, the quasi-working position is re-adjusted based on the rotation angle coordinate difference value of the standard correction pattern and the to-be-corrected pattern, and the rotation angle coordinate difference value is less than the angle error range threshold;

[0084] S302, two-dimensional plane coordinate correction of the center point is performed, a two-dimensional plane coordinate offset threshold is set, the quasi-working position is re-adjusted based on the two-dimensional plane coordinate difference value of the standard correction pattern and the to-be-corrected pattern, and the two-dimensional plane coordinate difference value is less than the two-dimensional plane coordinate offset threshold;

[0085] S303, carry out depth position correction, set depth position offset threshold, re-adjust the quasi-working position based on the perimeter difference value of the standard correction pattern and the to-be-corrected pattern until the perimeter difference value is less than the depth position offset threshold.

[0086] As shown in Figure 5 , when the rotation angle correction is carried out, the acceptable angle error range threshold μ is set, and the mechanical arm needs to adjust the angle in the plane by Δα:

[0087] Δα = α0- α1.

[0088] ① When Δα > 0, the mechanical arm rotates in the clockwise direction by adjusting Δα degrees;

[0089] ② When Δα < 0, the mechanical arm rotates in the counterclockwise direction by adjusting |Δα| degrees.

[0090] ② When Δα = 0, the mechanical arm does not need to adjust the rotation angle.

[0091] The mechanical arm repeatedly adjusts the to-be-corrected position until |Δα| < μ.

[0092] As shown in Figure 6 , when the two-dimensional plane coordinate correction of the center point is carried out, the position of the mechanical arm in the X and Y axis directions is adjusted, the mechanical arm is translated by 10 mm in the X axis direction, and the center point coordinates (x 10 , y0) at this time are calculated. As shown in the figure, the conversion ratio ρ0 between the moving displacement of the mechanical arm and the image pixel number is calculated:

[0093]

[0094] Then, the two-dimensional plane coordinate difference Δx between the standard correction pattern and the to-be-corrected pattern is calculated: Δx = x0-x1, Δy: Δy = y0-y1.

[0095] Let ∈x and ∈y be the maximum error threshold allowed in the X and Y directions respectively. The conversion ratio ρ1 of the to-be-corrected current position pixl-mm is calculated.

[0096] When Δx > 0, the mechanical arm moves away from the Y axis direction by Δx × ρ1 (mm), and when Δx < 0, the mechanical arm moves towards the Y axis direction by |Δx| × ρ1 (mm);

[0097] When Δy > 0, the mechanical arm moves away from the X axis direction by Δy × ρ1 (mm), and when Δy < 0, the mechanical arm moves towards the X axis direction by |Δy| × ρ1 (mm).

[0098] As shown in Figure 7As shown, the robotic arm repeatedly adjusts the position to be corrected until |Δx|<∈x,|Δy|<∈y.

[0099] After the robot arm completes the correction of the plane's X-axis and Y-axis directions, as well as the rotation angle α within the plane, it begins to perform depth correction in the depth direction (for example, when installing some parts, it is necessary to control the movement in the depth direction). Here, we define the displacement of the robot arm in the depth direction as the Z axis, and the standard working depth of the initial correction as z0. As the robot arm moves deeper, the synchronously moving camera gets closer to the correction pattern. When the camera moves toward the correction pattern, the degree to which the correction pattern becomes larger in the image (for example, the circumference len) is usually not linearly proportional to the distance d that the camera moves forward, but follows the geometric law of perspective projection. Therefore, it is necessary to capture images in real time for adjustment. The specific implementation steps are as follows:

[0100] like Figure 8 As shown, let ∈z be the maximum error threshold allowed in the Z direction. By correcting the perimeter of the rectangular box contour of the position len0 and the perimeter of the rectangular box contour of the working position len1, it is determined in which direction the robot arm should move along the Z axis: Δz = len0-len1.

[0101] When Δz>0, the robotic arm moves slowly in the direction of increasing along the Z axis, and takes images at regular intervals during the movement until |Δz|<∈z.

[0102] When Δz<0, the robotic arm moves slowly in the decreasing direction along the Z axis, and takes images at regular intervals during the movement until |Δz|<∈z.

[0103] According to one aspect of the present invention, a four-dimensional space precise position correction system for a robotic arm is proposed. Figure 9 As shown, it includes the following modules:

[0104] An image processing module 901 is configured to adjust the robotic arm to a standard working position and a quasi-working position for photographing, obtain a standard calibration pattern and a pattern to be calibrated, and perform image processing on the standard calibration pattern and the pattern to be calibrated;

[0105] A parameter calculation module 902 is configured to obtain the three-dimensional coordinates and perimeter of the standard calibration pattern and the pattern to be calibrated, wherein the three-dimensional coordinates specifically include the two-dimensional plane coordinates of the center point of the pattern and the rotation angle coordinates of the pattern;

[0106] The position correction module 903 is configured to perform three-dimensional coordinate correction and depth position correction on the quasi-working position by comparing the three-dimensional coordinates and perimeters of the standard correction pattern and the pattern to be corrected.

[0107] Figure 2(a) shows the correction pattern for correction, which is fixedly installed or attached on the workbench, and on the pattern, a rectangular frame, a positioning point and a rectangular center point are designed. Figure 2 (b) shows the ring-shaped light source, which is used to ensure that the industrial camera can image more clearly and stably, and the strip-shaped light source can be added in the case where the installation space is not allowed. Since the scheme will perform denoising and binarization on the image, the light source requirement is not very high. Figure 2 (c) shows the mechanical arm and the industrial camera solidly connected thereto: the basic motion control of the mechanical arm to the quasi-working position is completed through the control program, and then the imaging and processing of the correction pattern by the industrial camera are performed to realize more accurate correction of the working position, so that the mechanical arm further reaches the standard working position.

[0108] The coordinate system of the correction image is shown in Figure 3 , where the upper left corner is the (0, 0) coordinate, and the lower right corner is (w, h), where w and h are the resolution of the camera. The coordinate positions mentioned below comply with the following coordinate system definition, and the red dot is the positioning point of the correction pattern, and the black dot is the center point of the correction pattern.

[0109] After the mechanical arm moves to the standard working position and the quasi-working position through the control program, the image of the correction pattern at the current working position is collected by the industrial camera, and further image processing such as image binarization, image edge detection, image contour detection and fitting is performed to realize correction of the rotation angle α, two-dimensional plane position (X, Y axis) correction and longitudinal direction Z axis coordinate correction.

[0110] The rotation angle α is the included angle between the upper long side of the rectangle (the long side close to the positioning point) and the x axis, and the range is [-90° ~ +90°], and the specific positive and negative directions of the rotation angle are as follows:

[0111] ③ If α = 0°, then one side of the rectangle is parallel to the x axis (as shown in Figure 4 (a)).

[0112] ② If α is negative (α = -15°), the rectangular frame is rotated counterclockwise relative to the X axis (as shown in Figure 4 (b)).

[0113] ③ If α is negative (α = +15°), the rectangular frame is rotated clockwise relative to the X axis (as shown in Figure 4 (c)).

[0114] ④In order to reduce the installation difficulty of the correction pattern, the standard working position of the correction pattern is imaged, and the rotation angle (a0) does not necessarily have to be 0, and it is required that the positioning point of the correction pattern is at the upper left corner of the center point of the rectangular frame, that is, the two edges of the rectangular frame of the correction pattern do not necessarily have to be parallel to the X axis and the Y axis of the imaging coordinate system, and can have a certain included angle.

[0115] Specifically, the image processing module is specifically configured to perform inverse binarization processing on the standard correction pattern and the to-be-corrected pattern to separate the standard correction pattern and the to-be-corrected pattern from the background, perform image edge detection, extract edge information of the standard correction pattern and the to-be-corrected pattern, perform contour detection based on the edge information of the standard correction pattern and the to-be-corrected pattern, and obtain a contour point set of the standard correction pattern and the to-be-corrected pattern.

[0116] In the specific method of inverse binarization processing, the gray value of a pixel point in the image is I(x, y), and the threshold value of inverse binarization is T. The calculation formula is as follows:

[0117]

[0118] Through analysis of the image gray histogram, T = 100 is selected. After inverse binarization processing, the correction pattern and the background in the image are clearly separated.

[0119] In a specific embodiment, a Canny operator is used to perform edge detection on the image after inverse binarization processing. The Canny operator includes two threshold values, a high threshold value Th and a low threshold value Tl. In this embodiment, according to the characteristics of the image and the requirements of edge detection, the high threshold value Th = 100 and the low threshold value Tl = 50 are selected. First, a Gaussian filter is used to perform smoothing processing on the image to reduce noise interference. The kernel size of the Gaussian filter is set to 3x3, and the standard deviation σ = 1.0. Then, the gradient amplitude and direction of each pixel point in the image are calculated. Next, the gradient amplitude is subjected to non-maximum suppression and double-threshold detection according to the high threshold value and the low threshold value, so as to obtain the edge information of the image. After Canny operator edge detection, the edges of the correction pattern in the image are clearly extracted.

[0120] Next, the image after edge detection is subjected to contour detection. The findContours function in the OpenCV library is used to realize contour detection. This function can find all closed contours in the image and return the point set of each contour. In this embodiment, the contour retrieval mode is set to RETR_EXTERNAL, that is, only the outermost contour is detected; and the contour approximation method is CHAIN_APPROX_SIMPLE, that is, the contour is compressed and only the key points of the contour are saved. Through contour detection, the contour point set of the correction pattern can be obtained.

[0121] Specifically, the parameter calculation module is specifically configured to calculate the three-dimensional coordinates and perimeter of the standard correction pattern and the pattern to be corrected by fitting the minimum rectangular frame that can include the contour point set based on the contour point set of the standard correction pattern and the pattern to be corrected through the least squares method.

[0122] In a specific embodiment, the minAreaRect function in the OpenCV library is used to perform minimum rectangular frame fitting on the detected contour, and the minimum rectangular frame that can completely surround the contour is fitted using the least squares method.

[0123] Assume that the minimum rectangle has a length of L, a width of W, and a rotation angle of α (based on the image's X axis, with counterclockwise direction as positive). By calling the minAreaRect() function, the return value rect = ((x c ,y c ),(w,h),θ),rect(x c ,y c ) is the center coordinate of the standard calibration pattern or the pattern to be calibrated, (w, h) is the width and height of the standard calibration pattern or the pattern to be calibrated, where w is not necessarily greater than h and needs to be combined with the rotation angle θ of the rectangular frame. The conversion relationship between θ and α is as follows:

[0124]

[0125] The rotation angle α0 of the standard calibration pattern relative to the X-axis and the rotation angle α1 of the pattern to be calibrated relative to the X-axis can be calculated.

[0126] At the same time, the arcLength() function in the OpenCV library can be used to calculate the perimeter len0 of the standard calibration pattern and the perimeter len1 of the pattern to be calibrated.

[0127] Specifically, the parameter calculation module also includes calculating the two-dimensional plane coordinates of the positioning point of the pattern to be corrected, and judging whether a large directional deviation occurs in the quasi-working position by comparing the two-dimensional plane coordinates of the center point of the pattern to be corrected with the two-dimensional plane coordinates of the positioning point of the pattern to be corrected; if it is judged that a large directional deviation occurs, an error is reported; if it is judged that no large directional deviation occurs, the position correction module continues to be executed.

[0128] It is necessary to compare the coordinates of the center point of the rectangle (x1, y1) of the calibration pattern imaging with the coordinate position of the positioning point (x lp1 ,y lp1 ) relative relationship, when (x1>x lp1 ) and (y1>y lp1When the condition is met, it indicates that the positioning point of the to-be-corrected pattern in the current quasi-working position is at the upper left corner of the center point of the rectangular frame, and the direction is incorrect when the rotation angle α is corrected.

[0129] Specifically, the position correction module specifically includes the following modules:

[0130] The rotation angle correction module is configured to perform rotation angle correction, set an angle error range threshold, and readjust the quasi-working position based on a rotation angle coordinate difference value of the standard correction pattern and the to-be-corrected pattern until the rotation angle coordinate difference value is less than the angle error range threshold.

[0131] The two-dimensional plane coordinate correction module is configured to perform two-dimensional plane coordinate correction of the center point, set a two-dimensional plane coordinate offset threshold, and readjust the quasi-working position based on a two-dimensional plane coordinate difference value of the standard correction pattern and the to-be-corrected pattern until the two-dimensional plane coordinate difference value is less than the two-dimensional plane coordinate offset threshold.

[0132] The longitudinal position correction module is configured to perform longitudinal position correction, set a longitudinal position offset threshold, and readjust the quasi-working position based on a perimeter difference value of the standard correction pattern and the to-be-corrected pattern until the perimeter difference value is less than the longitudinal position offset threshold.

[0133] As shown in Figure 5 , when performing rotation angle correction, an acceptable angle error range threshold μ is set, and the mechanical arm needs to rotate an angle adjustment angle Δα in the plane:

[0134] Δα = α0- α1.

[0135] ① When Δα > 0, the mechanical arm rotates by Δα degrees in the clockwise direction;

[0136] ② When Δα < 0, the mechanical arm rotates by |Δα| degrees in the counterclockwise direction.

[0137] ④ When Δα = 0, the mechanical arm does not need to perform rotation angle adjustment.

[0138] The mechanical arm repeatedly adjusts the to-be-corrected position until |Δα| < μ.

[0139] As shown in Figure 6 , when performing two-dimensional plane coordinate correction of the center point, the position of the mechanical arm in the X and Y axis directions is adjusted, and the mechanical arm is translated by 10 mm in the X axis direction. The center point coordinates (x0, y0) at this time are calculated, and the conversion ratio ρ0 between the movement displacement of the mechanical arm and the number of image pixels is calculated as shown in the figure. 10

[0140] ​

[0141] Then calculate the two-dimensional plane coordinate difference Δx: Δx = x0-x1, Δy: Δy = y0-y1 of the standard correction pattern and the to-be-corrected pattern.

[0142] Let ∈x and ∈y be the maximum error threshold allowed in the X and Y directions respectively. Calculate the conversion ratio ρ1 of the current position pixl to be corrected (mm).

[0143] When Δx > 0, the robot moves Δx × ρ1 (mm) away from the Y axis direction, and when Δx < 0, the robot moves |Δx| × ρ1 (mm) towards the Y axis direction.

[0144] When Δy > 0, the robot moves Δy × ρ1 (mm) away from the X axis direction, and when Δy < 0, the robot moves |Δy| × ρ1 (mm) towards the X axis direction.

[0145] As shown in Figure 7 , the robot repeatedly adjusts the to-be-corrected position until |Δx| < ∈x, |Δy| < ∈y.

[0146] After the robot completes the correction of the X and Y axes in the plane and the rotation angle α in the plane, it begins to correct the working depth in the longitudinal direction (for example, partial part installation, which requires control of the movement amount in the depth direction). Here, we define the displacement of the robot in the longitudinal direction as the Z axis, and the initial correction standard working depth as z0. As the robot moves deeper, the camera that moves synchronously moves closer to the correction pattern. When the camera moves towards the correction pattern, the degree of enlargement of the correction pattern in the image (for example, the perimeter len) is not usually in a linear proportional relationship with the distance d of the camera moving forward, but follows the geometric law of perspective projection. Therefore, it is necessary to adjust in real time by capturing images. The specific implementation steps are as follows:

[0147] As shown in Figure 8 , let ∈z be the maximum error threshold allowed in the Z direction. By comparing the perimeter len0 of the correction position rectangular frame outline with the perimeter len1 of the working position rectangular frame outline, determine which direction the robot should move along the Z axis: Δz = len0 - len1.

[0148] When Δz > 0, the robot slowly moves along the Z axis in the direction of increasing Z axis. During the movement, images are taken at regular intervals until |Δz| < ∈z.

[0149] When Δz < 0, the robot slowly moves along the Z axis in the direction of decreasing Z axis. During the movement, images are taken at regular intervals until |Δz| < ∈z.

[0150] The above description is only the preferred embodiment of the present application and the explanation of the technical principles. It should be understood by those skilled in the art that the scope of the protection of the present application is not limited to the technical solutions formed by the specific combinations of the above technical features. It should also cover other technical solutions formed by the combinations of the above technical features or their equivalent features without departing from the concept of the present application. For example, the technical solutions formed by the mutual replacement of the above features and the technical features with similar functions disclosed (but not limited to) in the present application.

Claims

1. A method for accurate four-dimensional position correction of a robotic arm, characterized in that: The following steps are involved: S1, adjusting the robotic arm to a standard working position and a quasi-working position to respectively take pictures, obtaining a standard calibration pattern and a pattern to be calibrated, and performing image processing on the standard calibration pattern and the pattern to be calibrated; S2, obtaining the three-dimensional coordinates and perimeter of the standard calibration pattern and the pattern to be calibrated, wherein the three-dimensional coordinates specifically include the two-dimensional plane coordinates of the center point of the pattern and the rotation angle coordinates of the pattern; S3, performing three-dimensional coordinate correction and depth position correction on the quasi-working position by comparing the three-dimensional coordinates and perimeters of the standard correction pattern and the pattern to be corrected.

2. A method for calibrating the precise position of a robotic arm in four-dimensional space according to claim 1, characterized in that: S1 specifically includes performing inverse binarization processing on the standard correction pattern and the pattern to be corrected to separate the standard correction pattern and the pattern to be corrected from the background, then performing image edge detection to extract edge information of the standard correction pattern and the pattern to be corrected, and performing contour detection based on the edge information of the standard correction pattern and the pattern to be corrected to obtain the contour point set of the standard correction pattern and the pattern to be corrected.

3. The method for calibrating the precise position of a robotic arm in four-dimensional space according to claim 2, characterized in that: S2 specifically includes fitting a minimum rectangular frame that can include the contour point set based on the contour point set of the standard correction pattern and the pattern to be corrected by the least square method, thereby calculating the three-dimensional coordinates and perimeter of the standard correction pattern and the pattern to be corrected.

4. The method for calibrating the precise position of a robotic arm in four-dimensional space according to claim 3, characterized in that: It also includes calculating the two-dimensional plane coordinates of the positioning point of the pattern to be corrected, and judging whether a large direction deviation occurs in the quasi-working position by comparing the two-dimensional plane coordinates of the center point of the pattern to be corrected with the two-dimensional plane coordinates of the positioning point of the pattern to be corrected. If it is judged that a large direction deviation occurs, an error is reported; if it is judged that no large direction deviation occurs, S3 is continued to be executed.

5. The method for calibrating the precise position of a robotic arm in four-dimensional space according to claim 1, characterized in that: S3 specifically includes the following steps: S301, performing rotation angle correction, setting an angle error range threshold, and readjusting the quasi-working position based on the rotation angle coordinate difference between the standard correction pattern and the pattern to be corrected, until the rotation angle coordinate difference is less than the angle error range threshold; S302, performing two-dimensional plane coordinate correction of the center point, setting a two-dimensional plane coordinate offset threshold, and readjusting the quasi-working position based on the two-dimensional plane coordinate difference between the standard correction pattern and the pattern to be corrected until the two-dimensional plane coordinate difference is less than the two-dimensional plane coordinate offset threshold; S303 , performing depth position correction, setting a depth position offset threshold, and readjusting the quasi-working position based on the perimeter difference between the standard correction pattern and the pattern to be corrected, until the perimeter difference is less than the depth position offset threshold.

6. A four-dimensional space precise position correction system for a robotic arm, characterized in that: Includes the following modules: an image processing module configured to adjust the robotic arm to a standard working position and a quasi-working position for photographing, obtain a standard calibration pattern and a pattern to be calibrated, and perform image processing on the standard calibration pattern and the pattern to be calibrated; a parameter calculation module configured to obtain the three-dimensional coordinates and perimeter of the standard calibration pattern and the pattern to be calibrated, wherein the three-dimensional coordinates specifically include the two-dimensional plane coordinates of the center point of the pattern and the rotation angle coordinates of the pattern; The position correction module is configured to perform three-dimensional coordinate correction and depth position correction on the quasi-working position by comparing the three-dimensional coordinates and perimeters of the standard correction pattern and the pattern to be corrected.

7. The four-dimensional space precise position correction system for a robotic arm according to claim 6, characterized in that: The image processing module is specifically configured to perform inverse binarization processing on the standard correction pattern and the pattern to be corrected to separate the standard correction pattern and the pattern to be corrected from the background, then perform image edge detection, extract edge information of the standard correction pattern and the pattern to be corrected, perform contour detection based on the edge information of the standard correction pattern and the pattern to be corrected, and obtain the contour point set of the standard correction pattern and the pattern to be corrected.

8. The four-dimensional space precise position correction system for a robotic arm according to claim 7, characterized in that: The parameter calculation module is specifically configured to calculate the three-dimensional coordinates and perimeter of the standard correction pattern and the pattern to be corrected by fitting the minimum rectangular frame that can include the contour point set based on the contour point set of the standard correction pattern and the pattern to be corrected through the least squares method.

9. The four-dimensional space precise position correction system for a robotic arm according to claim 8, characterized in that: The parameter calculation module also includes calculating the two-dimensional plane coordinates of the positioning point of the pattern to be corrected, and judging whether a large directional deviation occurs in the quasi-working position by comparing the two-dimensional plane coordinates of the center point of the pattern to be corrected with the two-dimensional plane coordinates of the positioning point of the pattern to be corrected. If it is judged that a large directional deviation occurs, an error is reported; if it is judged that no large directional deviation occurs, the position correction module continues to be executed.

10. The four-dimensional space precise position correction system for a robotic arm according to claim 6, characterized in that: The position correction module specifically includes the following modules: a rotation angle correction module configured to perform rotation angle correction, set an angle error range threshold, and readjust the quasi-working position based on a rotation angle coordinate difference between the standard correction pattern and the pattern to be corrected until the rotation angle coordinate difference is less than the angle error range threshold; a two-dimensional plane coordinate correction module configured to perform two-dimensional plane coordinate correction of a center point, set a two-dimensional plane coordinate offset threshold, and readjust the quasi-working position based on a two-dimensional plane coordinate difference between the standard correction pattern and the pattern to be corrected until the two-dimensional plane coordinate difference is less than the two-dimensional plane coordinate offset threshold; The depth position correction module is configured to perform depth position correction, set a depth position offset threshold, and readjust the quasi-working position based on the perimeter difference between the standard correction pattern and the pattern to be corrected until the perimeter difference is less than the depth position offset threshold.

Images