Method, device and electronic equipment for correcting position of a robotic arm

Abstract

The present application provides a method, device and electronic device for correcting the position of a robotic arm; the method includes: determining a first distance and a first angle between the center of the end of the robotic arm and the optical axis of the second industrial camera based on a first industrial camera, a second industrial camera and a calibration plate; collecting a first standard image of a workpiece based on the first industrial camera, and determining a coordinate transformation matrix based on the first standard image; after the center of the end of the robotic arm moves to a designated target point, collecting a first partial image of the workpiece based on the second industrial camera; determining a position error of the center of the end of the robotic arm based on the first distance, the first angle, the coordinate transformation matrix, the designated target point, the first standard image and the first partial image; and performing position correction on the center of the end of the robotic arm based on the position error.

Description

Technical Field

[0001] The present application relates to the field of visual detection technology, and in particular to a method, device, and electronic equipment for correcting the position of a robotic arm. Background Art

[0002] With the continuous advancement of industrialization, the integration of equipment in the consumer electronics and printed circuit board manufacturing industries has become increasingly prominent. High precision and high integration are key development directions. A prerequisite for achieving high integration is the ability to achieve high-precision positioning. Currently, the primary reference indicator for robotic arm positioning accuracy is repeatability. This refers to the consistency of the actual position after repeating the same commanded position n times from the same direction. Domestic robotic arms typically achieve a positioning accuracy of around 0.03mm. The absolute positioning accuracy of a robotic arm is the deviation between its actual position and its ideal position. This is affected by factors such as machining errors, installation errors, and component wear, making it difficult to guarantee absolute positioning accuracy.

[0003] Robotic arms have a wide range of applications. In reciprocating motion, they often perform well due to their high repeatability. However, in non-reciprocating motion, due to their low absolute positioning accuracy, robotic arms are less effective when high-precision positioning is required. Summary of the Invention

[0004] The embodiments of the present application provide a method, device, and electronic device for correcting the position of a robotic arm, which can ensure the absolute positioning accuracy of the robotic arm during reciprocating motion and ensure the application effect of the robotic arm.

[0005] The technical solution of the embodiment of the present application is implemented as follows:

[0006] In a first aspect, an embodiment of the present application provides a method for position correction of a robotic arm, comprising:

[0007] Determine a first distance and a first angle between a center of a robotic arm end and an optical axis of the second industrial camera according to the first industrial camera, the second industrial camera, and the calibration plate;

[0008] Acquire a first standard image of the workpiece according to the first industrial camera, and determine a coordinate transformation matrix based on the first standard image;

[0009] After the center of the end of the robotic arm moves to a designated target point, collecting a first partial image of the workpiece according to the second industrial camera;

[0010] determining a position error of the center of the end of the robotic arm according to the first distance, the first angle, the coordinate transformation matrix, the designated target point, the first standard image, and the first partial image;

[0011] According to the position error, the position of the center of the end of the robot arm is corrected.

[0012] In the above solution, determining the first distance and the first angle between the center of the end of the robot arm and the optical axis of the second industrial camera based on the first industrial camera, the second industrial camera, and the calibration plate includes:

[0013] Acquiring a complete image of the calibration plate by the first industrial camera, and determining a first center coordinate of a center point on the complete image of the calibration plate;

[0014] Moving the end center of the robotic arm to the first center coordinate;

[0015] capturing a second partial image of the calibration plate according to the second industrial camera, and matching the complete image of the calibration plate with the second partial image to obtain a matched first area;

[0016] The first distance and the first angle are determined according to the first area and the first center coordinates.

[0017] In the above solution, determining the first distance and the first angle according to the first area and the first center coordinates includes:

[0018] Obtaining second center coordinates of a center point of the first area, and determining the second center coordinates as optical axis coordinates of the second industrial camera;

[0019] The first distance and the first angle are determined according to the first center coordinates and the optical axis coordinates.

[0020] In the above solution, determining a coordinate transformation matrix based on the first standard image includes:

[0021] Preprocessing the first standard image to determine the coordinates of a first reference point of a positioning reference point on the first standard image;

[0022] Determining the coordinates of a second reference point of the positioning reference point on the workpiece;

[0023] The coordinate transformation matrix is ​​determined according to the first reference point coordinates and the second reference point coordinates.

[0024] In the above solution, determining the position error of the center of the end of the robotic arm according to the first distance, the first angle, the coordinate transformation matrix, the designated target point, the first standard image, and the first partial image includes:

[0025] Determining first ideal coordinates corresponding to the optical axis of the second industrial camera on the first standard image according to the first distance, the first angle, the coordinate conversion matrix, and the designated target point;

[0026] Determining, based on the first standard image and the first partial image, a first actual coordinate corresponding to the optical axis of the second industrial camera on the first standard image;

[0027] A position error of the center of the end of the robotic arm is determined according to the first ideal coordinate and the first actual coordinate.

[0028] In the above solution, obtaining the first ideal coordinates corresponding to the optical axis of the second industrial camera on the first standard image based on the first distance, the first angle, the coordinate conversion matrix, and the designated target point includes:

[0029] Acquire a second ideal coordinate of the optical axis of the second industrial camera on the workpiece according to the first distance, the first angle, and the designated target point;

[0030] Determine, according to the coordinate conversion matrix, first ideal coordinates corresponding to the second ideal coordinates on the first standard image.

[0031] In the above solution, obtaining the second ideal coordinate of the optical axis of the second industrial camera on the workpiece based on the first distance, the first angle, and the designated target point includes:

[0032] The value of the second ideal coordinate in the first coordinate direction is the sum of the product of the first distance and the cosine value of the first angle and the value of the designated target point in the first coordinate direction;

[0033] The value of the second ideal coordinate in the second coordinate direction is the sum of the product of the first distance and the sine value of the first angle and the value of the designated target point in the second coordinate direction.

[0034] In the above solution, determining the first actual coordinates corresponding to the optical axis of the second industrial camera on the first standard image based on the first standard image and the first partial image includes:

[0035] Performing image matching on the first standard image and the first partial image to obtain a matched second region;

[0036] The coordinates of the center point of the second area are determined as the first actual coordinates.

[0037] In the above solution, determining the position error of the center of the end of the robotic arm according to the first ideal coordinate and the first actual coordinate includes:

[0038] Determine the distance between the first ideal coordinate and the first actual coordinate as the distance error of the center of the end of the robotic arm;

[0039] The angle between the line connecting the first ideal coordinate and the first actual coordinate and the horizontal direction is determined as the angular error of the center of the end of the robotic arm.

[0040] In a second aspect, an embodiment of the present application provides a position correction device for a robotic arm, the device comprising:

[0041] a calibration module, configured to determine a first distance and a first angle between a center of an end of a robotic arm and an optical axis of the second industrial camera based on the first industrial camera, the second industrial camera, and a calibration plate;

[0042] an image acquisition module, configured to acquire a first standard image of the workpiece using the first industrial camera and determine a coordinate transformation matrix based on the first standard image; and to acquire a first partial image of the workpiece using the second industrial camera after the center of the end of the robotic arm moves to a designated target point;

[0043] an error calculation module, configured to determine a position error of the center of the end of the robotic arm based on the first distance, the first angle, the coordinate transformation matrix, the designated target point, the first standard image, and the first partial image;

[0044] The correction module corrects the position of the center of the end of the robotic arm according to the position error.

[0045] In a third aspect, an embodiment of the present application provides an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the position correction method of the robotic arm provided in an embodiment of the present application.

[0046] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which includes a set of computer-executable instructions, which, when executed, are used to execute the position correction method of the robotic arm provided in the embodiment of the present application.

[0047] The embodiment of the present application provides a method for correcting the position of a robotic arm. The method determines a first distance and a first angle between the center of the robotic arm end and the optical axis of the second industrial camera based on a first industrial camera, a second industrial camera, and a calibration plate; acquires a first standard image of a workpiece based on the first industrial camera, and determines a coordinate transformation matrix based on the first standard image; after the center of the robotic arm end moves to a specified target point, acquires a first partial image of the workpiece based on the second industrial camera; determines a position error of the center of the robotic arm end based on the first distance, the first angle, the coordinate transformation matrix, the first standard image, and the first partial image; and corrects the position of the center of the robotic arm end based on the position error. The method for correcting the position of the robotic arm of the present application determines a first distance and a first angle between the center of the robotic arm end and the optical axis of the second industrial camera, and acquires a first standard image, a coordinate transformation matrix, and a first partial image of the workpiece. The method corrects the position of the center of the robotic arm end based on the position error, thereby ensuring the absolute positioning accuracy of the robotic arm in reciprocating motion and the accuracy of the robotic arm during application. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The accompanying drawings are provided to facilitate a better understanding of the present invention and do not constitute a limitation of the present application.

[0049] Figure 1 This is a schematic diagram of an optional processing flow of the position correction method of the robotic arm provided in an embodiment of the present application;

[0050] Figure 2 This is a calibration diagram provided in an embodiment of the present application;

[0051] Figure 3 is a schematic diagram of a calibration plate provided in an embodiment of the present application;

[0052] Figure 4 This is a schematic diagram of an optional structure of the position correction device of the robotic arm provided in an embodiment of the present application;

[0053] Figure 5 This is a schematic block diagram of an optional electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solutions and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limiting this application. All other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0055] In the following description, reference is made to “some embodiments”, which describes a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0056] In the following description, the terms "first\second" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It is understandable that "first\second" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0057] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.

[0058] The following describes a method for correcting the position of a robotic arm provided by an embodiment of the present application. Figure 1 , Figure 1 This is an optional processing flow diagram of the position correction method of the robot arm provided in the embodiment of the present application. Figure 1 Steps S101-S105 are shown for explanation.

[0059] Step S101: determining a first distance and a first angle between a center of a robot arm end and an optical axis of the second industrial camera according to a first industrial camera, a second industrial camera, and a calibration plate.

[0060] In some embodiments, a first industrial camera can be installed on the equipment rack. The pixel of the first industrial camera can be 2000W, and the field of view can be 315×210mm. It is facing the workpiece to capture the complete image of the workpiece, that is, the standard image; a second industrial camera is installed at the end of the robotic arm. The pixel of the second industrial camera can be 200W, and the field of view can be 20×215mm, which is used to capture a partial image of the workpiece.

[0061] In some embodiments, a calibration plate can be installed on the equipment rack to calibrate the center of the end of the manipulator and the optical axis of the second industrial camera, and determine the first distance and first angle between the center of the end of the manipulator and the optical axis of the second industrial camera. The distance between the coordinates of the center of the end of the manipulator and the coordinates of the optical axis of the second industrial camera is the first distance, and the angle between the coordinates of the center of the end of the manipulator and the coordinates of the optical axis of the second industrial camera in the horizontal direction is the first angle. Figure 2 The following steps 201 to 204 illustrate the calibration process.

[0062] Step 201: Capture a complete image of the calibration plate and determine the first center coordinates of the complete image. Use a first industrial camera on the equipment rack to capture the complete image of the calibration plate, and determine the coordinates of the center point on the complete image of the calibration plate, i.e., the first center coordinates.

[0063] Step 202: Capture a partial image of the calibration plate. Move the center 210 of the robotic arm end to the first center coordinate, and use the second industrial camera 100 of the robotic arm end 200 to capture a partial image of the calibration plate.

[0064] Step 203: Obtain the coordinates of the optical axis 110 of the second industrial camera. Perform image similarity matching on the complete image of the calibration plate and the partial image of the calibration plate. Determine the matched similar area as the first area, and determine the coordinates of the center point of the first area as the second center coordinates. The second center coordinates are also the coordinates of the optical axis 110 of the second industrial camera.

[0065] Step 204: Determine a first distance and a first angle between the center of the manipulator end and the optical axis of the second industrial camera. Based on the coordinates of the first center and the optical axis of the second industrial camera, and using a distance formula, such as a two-point distance formula, the distance between the manipulator end center 210 and the optical axis 110 of the second industrial camera, i.e., the first distance, can be determined. The angle between the manipulator end center 210 and the optical axis 110 of the second industrial camera, i.e., the first angle, can be determined using an angle formula, such as the tangent value between the line connecting the two coordinates and the horizontal direction, and then applying an inverse trigonometric function. The distance formula and angle formula can be determined based on actual needs.

[0066] In some embodiments, the calibration plate can be Figure 3 As shown. Figure 3 In the figure, the calibration plate consists of 9 concentric circles of different radii. Counting from the inside to the outside, there is a cross mark at the center of the first circle; the area between the second and third circles is divided into 6 parts; the area between the third, fourth and fifth circles is divided into 12 parts respectively; the area between the fifth to ninth circles is divided into 24 parts respectively; to ensure sufficient pixel information during image matching, the grids are filled with black and white patterns.

[0067] This application designs a calibration plate and uses it to calibrate the center of the manipulator end and the optical axis of the second industrial camera, thereby determining a first distance and a first angle between the center of the manipulator end and the optical axis of the second industrial camera. Therefore, the first distance and the first angle can be used to conveniently and efficiently calculate the position of the optical axis of the second industrial camera from the position of the center point of the manipulator end, and the position of the center point of the manipulator end can be determined from the position of the optical axis of the second industrial camera.

[0068] Step S102: Capture a first standard image of the workpiece using the first industrial camera, and determine a coordinate transformation matrix based on the first standard image.

[0069] In some embodiments, after calibration is completed, the calibration plate can be replaced with the workpiece. The complete image of the workpiece is captured based on the first industrial camera, and the complete image of the workpiece can be called a first standard image.

[0070] In some embodiments, based on the first standard image, the coordinates of the positioning reference point on the first standard image, i.e., the first reference point coordinates, can be identified, and a coordinate transformation matrix between the positioning reference point on the first standard image and the corresponding reference point coordinates on the workpiece can be determined. The process of determining the coordinate transformation matrix can be as follows: Steps 301 to 303.

[0071] Step 301: Preprocess a first standard image to determine the coordinates of first reference points of positioning reference points on the first standard image. Preprocessing the first standard image may include filtering and grayscale processing the first standard image, identifying positioning reference points on the first standard image using a Hough circle transform algorithm, template matching, or curve fitting, and determining the coordinates of the first reference points of the positioning reference points. Typically, the positioning reference points are three circular positioning reference points.

[0072] Step 302: Determine the real coordinates corresponding to the positioning reference point on the first standard image on the workpiece, that is, the coordinates of the second reference point.

[0073] Step 303: Determine the coordinate transformation matrix based on the coordinates of the first reference point and the coordinates of the second reference point. If the coordinates of the first reference point are expressed as (X1, Y1), (X2, Y2), (X3, Y3), and the coordinates of the second reference point are expressed as (x1, y1), (x2, y2), (x3, y3), the coordinate transformation matrix is ​​expressed as The relationship between the first reference point coordinates, the second reference point coordinates and the coordinate conversion matrix can be shown as formula (1). Among them, the values ​​of a to f in the coordinate conversion matrix can be obtained based on formula (1), the first reference point coordinates and the second reference point coordinates.

[0074]

[0075] In this application, by calculating the coordinate transformation matrix between the first standard image and the first partial image, the position of the center of the manipulator end on the workpiece can be obtained based on the position of the center of the manipulator end on the workpiece, and the position of the optical axis of the second industrial camera on the workpiece can be obtained based on the position of the optical axis of the second industrial camera on the workpiece. This facilitates the subsequent calculation of position errors.

[0076] Step S103 : After the center of the end of the robotic arm moves to the designated target point, a first partial image of the workpiece is captured by the second industrial camera.

[0077] In some embodiments, the designated target point is the coordinate point on the workpiece to which the center of the end of the manipulator is desired to move, and the position of the designated target point can be configured in advance. The center of the end of the manipulator will move based on the pre-set configuration information for the designated target point. However, under normal circumstances, the position to which the center of the end of the manipulator ultimately moves will have an error relative to the designated target point. Therefore, after the center of the end of the manipulator has completed its movement based on the designated target point, a partial image of the workpiece, i.e., a first partial image, can be captured using a second industrial camera, and the position error of the center of the end of the manipulator can then be determined based on subsequent steps.

[0078] Step S104 : determining a position error of the center of the end of the robotic arm according to the first distance, the first angle, the coordinate transformation matrix, the first standard image of the designated target point, and the first partial image.

[0079] In some embodiments, the process of determining the position error of the center of the end of the robot arm may include the following steps 401 to 404.

[0080] Step 401: Determine the second ideal coordinates corresponding to the optical axis of the second industrial camera on the workpiece. If the first distance is represented by X, the first angle is represented by θ, the coordinates corresponding to the designated target point on the workpiece are P(px,py), and the second ideal coordinates are (cx,cy). Based on the first distance, the first angle, and the designated target point, the second ideal coordinates are calculated as shown in the following formula (2). The value of the second ideal coordinate in the first coordinate direction is the sum of the product value between the first distance and the cosine value of the first angle and the value of the designated target point in the first coordinate direction; the value of the second ideal coordinate in the second coordinate direction is the sum of the product value between the first distance and the sine value of the first angle and the value of the designated target point in the second coordinate direction.

[0081] cx=X*cosθ+px, cy=X*sinθ+py (2)

[0082] Step 402: Determine the first ideal coordinates corresponding to the second ideal coordinates on the first standard image. Based on the coordinate transformation matrix and the above formula (1), the coordinates corresponding to the second ideal coordinates obtained in step 402 on the first standard image are calculated, i.e., the first ideal coordinates.

[0083] Step 403: Determine the first actual coordinates corresponding to the optical axis of the second industrial camera on the first standard image. Scale the first partial image to ensure that its pixel size is similar to that of the first standard image. Then, perform image similarity matching on the first standard image and the first partial image based on a sliding window. The resulting similar region is determined as the second region, and the coordinates of the center point of the second region are determined as the first actual coordinates.

[0084] Step 404: Determine the position error of the center of the end of the manipulator based on the first ideal coordinate and the first actual coordinate. The position error includes a distance error and an angle error. The distance between the first ideal coordinate and the first actual coordinate is determined as the distance error of the center of the end of the manipulator, and the angle between the line connecting the first ideal coordinate and the first actual coordinate and the horizontal direction is determined as the angle error of the center of the end of the manipulator. Since the optical axis of the second industrial camera is installed at the end of the manipulator, the position error of the optical axis of the second industrial camera is consistent with the position error of the center of the end of the manipulator. Therefore, the position error of the center of the end of the manipulator, which includes the distance error and the angle error, can be determined by simply obtaining the distance between the first ideal coordinate and the first actual coordinate of the optical axis of the second industrial camera, and the angle between the line connecting the first ideal coordinate and the first actual coordinate of the optical axis of the second industrial camera and the horizontal direction.

[0085] Step S105: Correct the position of the center of the end of the robotic arm according to the position error.

[0086] In some embodiments, the position of the center of the manipulator end can be corrected based on the distance error and angle error in the position error. The center of the manipulator end can be moved in a direction with an angle error relative to the horizontal by the value of the distance error to accurately calibrate the position of the center of the manipulator end to ensure the absolute position accuracy of the center of the manipulator end.

[0087] In the embodiments of the present application, errors may occur during the reciprocating motion of the robotic arm, and the target point actually reached by the robotic arm is not the designated target point. Consequently, an error exists between the target point actually reached by the robotic arm and the designated target point. Because the position error of the center of the robotic arm end cannot be directly determined by comparing the designated target point with the position actually reached by the center of the robotic arm end, the position error of the robotic arm end is indirectly determined by obtaining the position error of the optical axis of a second industrial camera at the robotic arm end.

[0088] Using a calibration plate, the first distance and first angle between the center of the manipulator end and the optical axis of the second industrial camera are determined. Based on this first distance and angle and a specified target point, the second ideal coordinates that the optical axis of the second industrial camera is intended to reach are calculated. By determining the coordinate transformation matrix, the first ideal coordinates corresponding to the second ideal coordinates on the first standard image are obtained. By matching the first standard image with the first partial image, the first actual coordinates, the actual position coordinates of the optical axis of the second industrial camera on the first standard image, are obtained. The position error between the first actual coordinates and the first ideal coordinates is used to determine the position error of the manipulator end center. Based on this position error, the manipulator end center is adjusted to ensure the accuracy of the manipulator.

[0089] It should be understood that in the embodiments of the present application, the position error is calculated based on the first actual position actually reached by the optical axis of the second industrial camera on the first standard image and the first ideal position corresponding to the optical axis of the second industrial camera on the first standard image. However, based on the ideas in the embodiments of the present application, in some embodiments, the position error can also be determined based on the position actually reached by the optical axis of the second industrial camera on the workpiece and the second ideal position corresponding to the optical axis of the second industrial camera on the workpiece. The position actually reached by the second industrial camera on the workpiece can be obtained by the first actual coordinates and the coordinate transformation matrix. This should also be within the scope of protection of the present application. Figure 4 This is a schematic diagram of an optional device structure of a position correction device for a robotic arm provided in an embodiment of the present application. The position correction device 400 for a robotic arm includes a calibration module 401, an image acquisition module 402, an error calculation module 403, and a correction module 404.

[0090] A calibration module 401 is configured to determine a first distance and a first angle between a center of a robot arm end and an optical axis of the second industrial camera based on the first industrial camera, the second industrial camera, and a calibration plate;

[0091] An image acquisition module 402 is configured to acquire a first standard image of the workpiece using the first industrial camera and determine a coordinate transformation matrix based on the first standard image; and after the center of the end of the robotic arm moves to a designated target point, acquire a first partial image of the workpiece using the second industrial camera;

[0092] an error calculation module 403, configured to determine a position error of the center of the end of the robotic arm based on the first distance, the first angle, the coordinate transformation matrix, the designated target point, the first standard image, and the first partial image;

[0093] The correction module 404 is used to correct the position of the center of the end of the robot arm according to the position error.

[0094] In some embodiments, the calibration module 401 is also used to: determine the first center coordinates of the center point on the complete image of the calibration plate according to the first industrial camera; move the center of the end of the robotic arm to the first center coordinates; capture the second partial image of the calibration plate according to the second industrial camera, match the complete image of the calibration plate and the second partial image to obtain a matched first area; determine the first distance and the first angle according to the first area and the first center coordinates.

[0095] In some embodiments, the calibration module 401 is also used to: obtain the second center coordinates of the center point of the first area, and determine the second center coordinates as the optical axis coordinates of the second industrial camera; determine the first distance and the first angle based on the first center coordinates and the optical axis coordinates.

[0096] In some embodiments, the image acquisition module 402 is also used to: pre-process the first standard image to determine the first reference point coordinates of the positioning reference point on the first standard image; determine the second reference point coordinates of the positioning reference point on the workpiece; and determine the coordinate transformation matrix based on the first reference point coordinates and the second reference point coordinates.

[0097] In some embodiments, the error calculation module 403 is also used to: determine the first ideal coordinates corresponding to the optical axis of the second industrial camera on the first standard image based on the first distance, the first angle, the coordinate transformation matrix and the designated target point; determine the first actual coordinates corresponding to the optical axis of the second industrial camera on the first standard image based on the first standard image and the first local image; determine the position error of the center of the end of the robotic arm based on the first ideal coordinates and the first actual coordinates.

[0098] In some embodiments, the error calculation module 403 is also used to: obtain the second ideal coordinates of the optical axis of the second industrial camera on the workpiece based on the first distance, the first angle and the designated target point; and determine the first ideal coordinates corresponding to the second ideal coordinates on the first standard image based on the coordinate transformation matrix.

[0099] In some embodiments, the error calculation module 403 is also used to: the value of the second ideal coordinate in the first coordinate direction is the sum of the product value between the cosine value of the first distance and the first angle and the value of the designated target point in the first coordinate direction; the value of the second ideal coordinate in the second coordinate direction is the sum of the product value between the sine value of the first distance and the first angle and the value of the designated target point in the second coordinate direction.

[0100] In some embodiments, the error calculation module 403 is further configured to: perform image matching on the first standard image and the first partial image to obtain a matched second region; and determine the coordinates of the center point of the second region as the first actual coordinates.

[0101] In some embodiments, the error calculation module 403 is also used to: determine the distance between the first ideal coordinate and the first actual coordinate as the distance error of the center of the end of the robotic arm; determine the angle between the line connecting the first ideal coordinate and the first actual coordinate and the horizontal direction as the angle error of the center of the end of the robotic arm.

[0102] It should be noted that the position correction device of the robot arm in the embodiment of the present application is similar to the description of the position correction method of the robot arm in the embodiment of the present application, and has similar beneficial effects as the method embodiment, so it will not be described in detail. Figures 1 to 3 The present invention shall be understood by reference to the description of any of the accompanying drawings.

[0103] Figure 5 A schematic block diagram of an example electronic device 500 that can be used to implement an embodiment of the present disclosure is shown. The electronic device 500 is used to implement the position correction method of the robotic arm of an embodiment of the present disclosure. In some optional embodiments, the electronic device 500 can implement the position correction method of the robotic arm provided in the embodiment of the present application by running a computer program. For example, the computer program can be a software module in the operating system; it can be a native APP (Application), that is, a program that needs to be installed in the operating system to run; it can also be a small program, that is, a program that only needs to be downloaded to a browser environment to run; it can also be a small program that can be embedded in any APP. In short, the above-mentioned computer program can be an application, module or plug-in in any form.

[0104] In actual applications, the electronic device 500 can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. Cloud technology refers to a hosting technology that unifies a series of resources such as hardware, software, and networks within a wide area network or local area network to achieve data computing, storage, processing, and sharing. The electronic device 500 can be a smartphone, tablet computer, laptop computer, desktop computer, smart speaker, smart TV, smart watch, etc., but is not limited to these.

[0105] Electronic device is intended to refer to various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Electronic device can also refer to various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices, vehicle-mounted terminals, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or claimed herein.

[0106] like Figure 5 As shown, the electronic device 500 includes a computing unit 501, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 502 or a computer program loaded from a storage unit 508 into a random access memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the electronic device 500 can also be stored. The computing unit 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0107] Multiple components in the electronic device 500 are connected to the I / O interface 505, including: an input unit 506, such as a keyboard, a mouse, etc.; an output unit 507, such as various types of displays, speakers, etc.; a storage unit 508, such as a magnetic disk, an optical disk, etc.; and a communication unit 509, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 509 allows the electronic device 500 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0108] The computing unit 501 can be various general-purpose and / or specialized processing components with processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The computing unit 501 performs the various methods and processes described above, such as the position correction method for the robotic arm. For example, in some optional embodiments, the position correction method for the robotic arm can be implemented as a computer software program that is tangibly contained in a machine-readable medium, such as the storage unit 508. In some optional embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 500 via the ROM 502 and / or the communication unit 509. When the computer program is loaded into the RAM 503 and executed by the computing unit 501, one or more steps of the position correction method for the robotic arm described above can be performed. Alternatively, in other embodiments, the computing unit 501 can be configured to perform the position correction method for the robotic arm by any other appropriate means (e.g., by means of firmware).

[0109] An embodiment of the present application provides a computer-readable storage medium storing executable instructions, wherein the executable instructions are stored. When the executable instructions are executed by a processor, the processor will execute the position correction method of the robotic arm provided in the embodiment of the present application.

[0110] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface storage, optical disk, or CD-ROM; or various devices including one or any combination of the above memories.

[0111] In some embodiments, executable instructions may be in the form of a program, software, software module, script, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0112] By way of example, executable instructions may be deployed to be executed on one computing device, or on multiple computing devices at one site, or on multiple computing devices distributed across multiple sites and interconnected by a communication network.

[0113] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0114] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0115] It should be understood that in the various embodiments of the present application, the size of the serial number of each implementation process does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0116] The above are merely examples of the present application and are not intended to limit the scope of protection of the present application. Any modifications, equivalent replacements, and improvements made within the spirit and scope of the present application are included in the scope of protection of the present application.

Claims

1. A method for correcting the position of a robotic arm, characterized in that: The method comprises: Determining a first distance and a first angle between a center of an end of a robotic arm and an optical axis of the second industrial camera based on a first industrial camera, a second industrial camera, and a calibration plate, wherein the first industrial camera is mounted on an equipment rack and faces a workpiece, and the second industrial camera is mounted on the end of the robotic arm; Acquire a first standard image of the workpiece according to the first industrial camera, and determine a coordinate transformation matrix based on the first standard image; After the center of the end of the robotic arm moves to a designated target point, collecting a first partial image of the workpiece according to the second industrial camera; determining a position error of the center of the end of the robotic arm according to the first distance, the first angle, the coordinate transformation matrix, the designated target point, the first standard image, and the first partial image; Correcting the position of the center of the end of the robotic arm according to the position error; The determining, based on the first industrial camera, the second industrial camera, and the calibration plate, a first distance and a first angle between the center of the end of the robotic arm and the optical axis of the second industrial camera includes: Acquiring a complete image of the calibration plate by the first industrial camera, and determining a first center coordinate of a center point on the complete image of the calibration plate; Moving the end center of the robotic arm to the first center coordinate; capturing a second partial image of the calibration plate according to the second industrial camera, and matching the complete image of the calibration plate with the second partial image to obtain a matched first area; Obtaining second center coordinates of a center point of the first area, and determining the second center coordinates as optical axis coordinates of the second industrial camera; Determine the first distance and the first angle based on the first center coordinates and the optical axis coordinates, wherein the first distance is the distance between the center of the end of the robotic arm and the optical axis of the second industrial camera, and the first angle is the angle between the center of the end of the robotic arm and the optical axis of the second industrial camera in the horizontal direction.

2. The method according to claim 1, characterized in that The determining of a coordinate transformation matrix based on the first standard image includes: Preprocessing the first standard image to determine the coordinates of a first reference point of a positioning reference point on the first standard image; Determining the coordinates of a second reference point of the positioning reference point on the workpiece; The coordinate transformation matrix is ​​determined according to the first reference point coordinates and the second reference point coordinates.

3. The method according to claim 1, characterized in that The determining the position error of the center of the end of the robotic arm according to the first distance, the first angle, the coordinate transformation matrix, the designated target point, the first standard image, and the first partial image includes: Determining first ideal coordinates corresponding to the optical axis of the second industrial camera on the first standard image according to the first distance, the first angle, the coordinate conversion matrix, and the designated target point; Determining, based on the first standard image and the first partial image, a first actual coordinate corresponding to the optical axis of the second industrial camera on the first standard image; A position error of the center of the end of the robotic arm is determined according to the first ideal coordinate and the first actual coordinate.

4. The method according to claim 3, characterized in that The acquiring, according to the first distance, the first angle, the coordinate conversion matrix, and the designated target point, a first ideal coordinate corresponding to the optical axis of the second industrial camera on the first standard image includes: Acquire a second ideal coordinate of the optical axis of the second industrial camera on the workpiece according to the first distance, the first angle, and the designated target point; Determine, according to the coordinate conversion matrix, first ideal coordinates corresponding to the second ideal coordinates on the first standard image.

5. The method according to claim 4, characterized in that The step of acquiring a second ideal coordinate of the optical axis of the second industrial camera on the workpiece according to the first distance, the first angle, and the designated target point includes: The value of the second ideal coordinate in the first coordinate direction is the sum of the product of the first distance and the cosine value of the first angle and the value of the designated target point in the first coordinate direction; The value of the second ideal coordinate in the second coordinate direction is the sum of the product of the first distance and the sine value of the first angle and the value of the designated target point in the second coordinate direction.

6. The method according to claim 3, characterized in that The determining, based on the first standard image and the first partial image, a first actual coordinate corresponding to the optical axis of the second industrial camera on the first standard image includes: Performing image matching on the first standard image and the first partial image to obtain a matched second region; The coordinates of the center point of the second area are determined as the first actual coordinates.

7. The method according to claim 3, characterized in that Determining the position error of the center of the end of the robotic arm according to the first ideal coordinate and the first actual coordinate includes: Determine the distance between the first ideal coordinate and the first actual coordinate as the distance error of the center of the end of the robotic arm; The angle between the line connecting the first ideal coordinate and the first actual coordinate and the horizontal direction is determined as the angular error of the center of the end of the robotic arm.

8. A position correction device for a robotic arm, characterized in that: The device comprises: a calibration module, configured to determine a first distance and a first angle between a center of a robotic arm end and an optical axis of the second industrial camera based on a first industrial camera, a second industrial camera, and a calibration plate, wherein the first industrial camera is mounted on an equipment rack and faces a workpiece, and the second industrial camera is mounted on the robotic arm end; an image acquisition module, configured to acquire a first standard image of the workpiece using the first industrial camera and determine a coordinate transformation matrix based on the first standard image; and to acquire a first partial image of the workpiece using the second industrial camera after the center of the end of the robotic arm moves to a designated target point; an error calculation module, configured to determine a position error of the center of the end of the robotic arm based on the first distance, the first angle, the coordinate transformation matrix, the designated target point, the first standard image, and the first partial image; A correction module, configured to correct the position of the end center of the robotic arm according to the position error; The determining, based on the first industrial camera, the second industrial camera, and the calibration plate, a first distance and a first angle between the center of the end of the robotic arm and the optical axis of the second industrial camera includes: Acquiring a complete image of the calibration plate by the first industrial camera, and determining a first center coordinate of a center point on the complete image of the calibration plate; Moving the end center of the robotic arm to the first center coordinate; capturing a second partial image of the calibration plate according to the second industrial camera, and matching the complete image of the calibration plate with the second partial image to obtain a matched first area; Obtaining second center coordinates of a center point of the first area, and determining the second center coordinates as optical axis coordinates of the second industrial camera; Determine the first distance and the first angle based on the first center coordinates and the optical axis coordinates, wherein the first distance is the distance between the center of the end of the robotic arm and the optical axis of the second industrial camera, and the first angle is the angle between the center of the end of the robotic arm and the optical axis of the second industrial camera in the horizontal direction.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively coupled to the at least one processor; The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The storage medium includes a set of computer-executable instructions, which, when executed, are used to execute the position correction method of the robotic arm according to any one of claims 1 to 7.

Images