An error assessment method, apparatus, and electronic device

Abstract

This application provides an error assessment method, apparatus, and electronic device, relating to the field of machine vision technology. It involves acquiring a first set of images and a second set of images of a target object from an image acquisition device; determining the camera's overall error using the first maximum and minimum coordinates of feature points of the target object on a first coordinate axis and a second maximum and minimum coordinate on a second coordinate axis in the first set of images; determining the system translation error of the motion mechanism using the camera's overall error, the third maximum and third minimum coordinates of feature points on the first coordinate axis in the second set of images, and the fourth maximum and fourth minimum coordinates on the second coordinate axis; and determining the system's overall error relating to the image acquisition device and the motion mechanism based on the camera's overall error and the system translation error. Applying the solution provided in this application, the system's overall error can be determined.

Description

Technical Field

[0001] This application relates to the field of machine vision technology, and in particular to an error assessment method, apparatus and electronic device. Background Technology

[0002] With the continuous development of machine vision technology, machine vision systems are widely used in the production processes of various high-precision products, such as in the production process of electronic products, where machine vision systems are used to grasp the components used.

[0003] The aforementioned machine vision system typically includes an image acquisition device for image acquisition and a motion mechanism for moving objects or the image acquisition device. For example, a machine vision system may include a lathe and an image acquisition device; a machine vision system may include a camera, a robotic arm base, and a robotic arm, etc.

[0004] Typically, errors in a machine vision system will affect its operational accuracy. Therefore, how to assess the overall system error of a machine vision system and use the determined overall system error for error compensation is a technical problem that urgently needs to be solved. Summary of the Invention

[0005] The purpose of this application is to provide an error assessment method, apparatus, and electronic device to determine the overall system error of a machine vision system relating to an image acquisition device and a motion mechanism. The specific technical solution is as follows:

[0006] In a first aspect, embodiments of this application provide an error assessment method, the method comprising:

[0007] Acquire a first set of images and a second set of images of a target object from an image acquisition device; wherein, the first set of images is acquired by the image acquisition device at a first position and the target object at a second position; the second set of images is acquired by the image acquisition device of the target object each time the motion mechanism rotates from an initial position to a target position so that the image acquisition device and the target object are in a specified relative position;

[0008] Using the first maximum and first minimum coordinates of the feature points of the target object on the first coordinate axis of the image coordinate system corresponding to the image acquisition device in the first set of images, and the second maximum and second minimum coordinates of the feature points on the second axis of the image coordinate system, the camera comprehensive error of the image acquisition device is determined;

[0009] The system translation error of the motion mechanism is determined by using the camera integrated error, the third maximum and third minimum coordinates of the feature points in the second set of images on the first coordinate axis, and the fourth maximum and fourth minimum coordinates of the feature points in the second set of images on the second coordinate axis.

[0010] Based on the camera's overall error and the system's translation error, the overall system error for the image acquisition device and the motion mechanism is determined.

[0011] Optionally, in one specific implementation, the motion mechanism includes a rotation axis; before determining the system comprehensive error regarding the image acquisition device and the motion mechanism based on the camera comprehensive error and the system translation error, the method further includes:

[0012] Acquire a third set of images of the target object acquired by the image acquisition device; wherein, the third set of images is acquired by the image acquisition device when the rotation axis rotates a specified angle each time the rotation axis rotates, during the process of the rotation axis rotating multiple times continuously along the first direction so that the target object rotates multiple times continuously relative to the image acquisition device along the second direction.

[0013] The system rotation error of the motion mechanism is determined by using the rotation axis length of the rotation axis and the maximum value of the difference between the specified angle and the angle change of the feature point in the third set of images;

[0014] The determination of the system comprehensive error regarding the image acquisition device and the motion mechanism based on the camera comprehensive error and the system translation error includes:

[0015] Based on the camera's overall error, the system's translation error, and the system's rotation error, the overall system error for the image acquisition device and the motion mechanism is determined.

[0016] Optionally, in one specific implementation, determining the camera overall error of the image acquisition device using the first maximum and first minimum coordinates of the feature points of the target object on the first coordinate axis of the image coordinate system corresponding to the image acquisition device, and the second maximum and second minimum coordinates of the feature points on the second axis of the image coordinate system, includes:

[0017] Calculate the difference between the first maximum coordinate and the first minimum coordinate of the feature points of the target object on the first coordinate axis of the image coordinate system corresponding to the image acquisition device in the first group of images, and obtain the first difference;

[0018] Calculate the difference between the second maximum coordinate and the second minimum coordinate of the feature point on the second coordinate axis of the image coordinate system in the first group of images to obtain the second difference;

[0019] The camera overall error of the image acquisition device is determined using the first difference, the second difference, and a preset relationship; wherein the preset relationship includes: the hand-eye calibration matrix of the image acquisition device and the motion mechanism, or the single-pixel accuracy of the image acquisition device.

[0020] Optionally, in one specific implementation, determining the system translation error of the motion mechanism using the camera's overall error, the third maximum and third minimum coordinates of the feature points in the second set of images on the first coordinate axis, and the fourth maximum and fourth minimum coordinates of the feature points in the second set of images on the second coordinate axis includes:

[0021] Calculate the difference between the third maximum coordinate and the third minimum coordinate of the feature point on the first coordinate axis in the second set of images to obtain the third difference;

[0022] Calculate the difference between the fourth maximum coordinate and the fourth minimum coordinate of the feature point on the second coordinate axis in the second set of images to obtain the fourth difference;

[0023] Using the camera's overall error, the third difference, the fourth difference, and a preset relationship, the repeatability error of the motion mechanism is determined, and based on the repeatability error, the system translation error of the motion mechanism is determined; wherein, the preset relationship includes: the hand-eye calibration matrix of the image acquisition device and the motion mechanism, or the single-pixel accuracy of the image acquisition device.

[0024] Optionally, in one specific implementation, the preset relationship includes: the hand-eye calibration matrix; determining the system translation error of the motion mechanism based on the repetitive positioning error includes:

[0025] The absolute positioning error of the motion mechanism is determined by using the camera integrated error, the hand-eye calibration matrix, and multiple sets of calibration coordinates used to determine the hand-eye calibration matrix.

[0026] Based on the repeatability error and the absolute positioning error, the system translation error of the motion mechanism is determined.

[0027] Optionally, in one specific implementation, the system rotation error of the motion mechanism is affected by the angle extraction accuracy error and the rotation accuracy error of the rotation axis; the method for determining the angle extraction accuracy error includes:

[0028] Calculate the difference between the maximum and minimum angles of the feature points in the first set of images to obtain the angle difference;

[0029] Using the angle difference and the rotation axis length, the angle extraction accuracy error for the feature points in the first set of images is determined.

[0030] The method for determining the rotational accuracy error includes:

[0031] The rotational accuracy error of the rotating mechanism is determined using the angle difference, the maximum value, and the rotation axis length.

[0032] Optionally, in one specific implementation, before determining the system rotation error of the motion mechanism using the maximum value of the difference between the rotation axis length of the rotation axis and the difference between the specified angle and the angular change of the feature point in the third set of images, the method further includes:

[0033] Using the hand-eye calibration matrix of the image acquisition device and the motion mechanism, the rotation axis calibration error of the motion mechanism is determined;

[0034] The determination of the system rotation error of the motion mechanism by utilizing the rotation axis length of the rotation axis and the maximum value of the difference between the specified angle and the angular change of the feature point in the third set of images includes:

[0035] The rotation error of the motion mechanism is determined by using the rotation axis length of the rotation axis and the maximum value of the difference between the specified angle and the angle change of the feature point in the third set of images.

[0036] The system rotation error of the motion mechanism is determined by using the rotation error of the rotating shaft and the calibration error of the rotating shaft.

[0037] Optionally, in one specific implementation, before determining the system comprehensive error regarding the image acquisition device and the motion mechanism based on the camera comprehensive error and the system translation error, the method further includes:

[0038] Determine the teaching error of the motion mechanism;

[0039] The determination of the system comprehensive error regarding the image acquisition device and the motion mechanism based on the camera comprehensive error and the system translation error includes:

[0040] Based on the camera's overall error, the system translation error, and the teaching error, the overall system error concerning the image acquisition device and the motion mechanism is determined.

[0041] Optionally, in one specific implementation, determining the camera overall error of the image acquisition device using the first difference, the second difference, and a preset relationship includes:

[0042] The overall camera error of the image acquisition device is determined using the first formula.

[0043] The first formula is:

[0044]

[0045] Wherein, CIE is the camera integrated error of the image acquisition device; M is the hand-eye calibration matrix of the image acquisition device and the motion mechanism; XPixRange_S is the first difference; YPixRange_S is the second difference;

[0046] or,

[0047] The overall camera error of the image acquisition device is determined using the second formula.

[0048] The second formula is:

[0049]

[0050] Wherein, PixAcc represents the single-pixel accuracy of the image acquisition device.

[0051] Optionally, in one specific implementation, determining the repeatability error of the motion mechanism using the camera's overall error, the third difference, the fourth difference, and a preset relationship includes:

[0052] The repeatability error of the motion mechanism is determined using the third formula.

[0053] The third formula is as follows:

[0054]

[0055] Wherein, MRTE is the repeatability error of the motion mechanism; XPixRange_D is the third difference; YPixRange_D is the fourth difference; M is the hand-eye calibration matrix for the image acquisition device and the motion mechanism; CIE is the camera integrated error;

[0056] or,

[0057] The repeatability error of the motion mechanism is determined using the fourth formula.

[0058] The fourth formula is:

[0059]

[0060] Wherein, PixAcc represents the single-pixel accuracy of the image acquisition device.

[0061] Optionally, in one specific implementation, determining the absolute positioning error of the motion mechanism using the camera's overall error, the hand-eye calibration matrix, and multiple sets of calibration coordinates used to determine the hand-eye calibration matrix includes:

[0062] The absolute positioning error of the motion mechanism is determined using the fifth formula.

[0063] The fifth formula is as follows:

[0064]

[0065] Where MATE is the absolute positioning error of the motion mechanism; X_Wld i X_Pix is ​​the X-coordinate of the calibration point in the i-th set of calibration coordinates in the world coordinate system. i The X coordinate of the calibration point in the i-th set of calibration coordinates in the image coordinate system; Y_Wld i Y_Pix is ​​the Y-coordinate of the calibration point in the i-th set of calibration coordinates in the world coordinate system. i The Y-coordinate of the calibration point in the i-th group of calibration coordinates in the image coordinate system; N is the number of calibration coordinate groups for the calibration point;

[0066] The determination of the system translation error of the motion mechanism based on the repeatability error and the absolute positioning error includes:

[0067] Using the sixth formula, the system translation error of the motion mechanism is determined:

[0068] The sixth formula is as follows:

[0069]

[0070] Wherein, STLE is the system translation error of the motion mechanism; MRTE is the repeatability error of the motion mechanism; and MATE is the absolute positioning error of the motion mechanism.

[0071] Optionally, in one specific implementation, determining the system rotation error of the motion mechanism using the rotation error of the rotation axis and the calibration error of the rotation axis includes:

[0072] The system rotational error of the motion mechanism is determined using the seventh formula.

[0073] The seventh formula is as follows:

[0074]

[0075] Wherein, SRE is the system rotation error of the motion mechanism; MRAE is the rotation error of the rotation axis; MRCE is the calibration error of the rotation axis; and deltaR is the maximum value among the differences between the specified angle and the angle change of the feature point in the third set of images.

[0076] Secondly, embodiments of this application provide an error assessment apparatus, the apparatus comprising:

[0077] An image acquisition module is used to acquire a first set of images and a second set of images of a target object acquired by an image acquisition device; wherein, the first set of images is acquired by the image acquisition device at a first position and the target object at a second position; the second set of images is acquired by the image acquisition device of the target object each time the motion mechanism rotates from an initial position to a target position so that the image acquisition device and the target object are in a specified relative position;

[0078] The first determining module is used to determine the camera comprehensive error of the image acquisition device by using the first maximum coordinates and the first minimum coordinates of the feature points of the target object on the first coordinate axis of the image coordinate system corresponding to the image acquisition device in the first group of images, and the second maximum coordinates and the second minimum coordinates of the feature points on the second axis of the image coordinate system.

[0079] The second determining module is used to determine the system translation error of the motion mechanism by using the camera integrated error, the third maximum coordinate and the third minimum coordinate of the feature point in the second set of images on the first coordinate axis, and the fourth maximum coordinate and the fourth minimum coordinate of the feature point in the second set of images on the second coordinate axis.

[0080] The comprehensive error determination module is used to determine the comprehensive system error of the image acquisition device and the motion mechanism based on the comprehensive camera error and the system translation error.

[0081] Optionally, in one specific implementation, the motion mechanism includes a rotating shaft; the device further includes:

[0082] The third determining module is used to acquire a third set of images of the target object acquired by the image acquisition device before determining the system comprehensive error of the image acquisition device and the motion mechanism based on the camera comprehensive error and the system translation error; wherein, the third set of images is acquired by the image acquisition device when the rotation axis rotates the target object multiple times in a first direction so that the target object rotates multiple times in a second direction relative to the image acquisition device;

[0083] The system rotation error determination module is used to determine the system rotation error of the motion mechanism by using the rotation axis length of the rotation axis and the maximum value of the difference between the specified angle and the angle change of the feature point in the third set of images;

[0084] The comprehensive error determination module is specifically used for:

[0085] Based on the camera's overall error, the system's translation error, and the system's rotation error, the overall system error for the image acquisition device and the motion mechanism is determined.

[0086] Optionally, in one specific implementation, the first determining module includes:

[0087] The first calculation submodule is used to calculate the difference between the first maximum coordinate and the first minimum coordinate of the feature points of the target object on the first coordinate axis of the image coordinate system corresponding to the image acquisition device in the first group of images, and obtain the first difference.

[0088] The second calculation submodule is used to calculate the difference between the second maximum coordinate and the second minimum coordinate of the feature point on the second coordinate axis of the image coordinate system in the first group of images, and obtain the second difference.

[0089] The first determining submodule is used to determine the camera comprehensive error of the image acquisition device using the first difference, the second difference, and a preset relationship; wherein the preset relationship includes: the hand-eye calibration matrix of the image acquisition device and the motion mechanism, or the single-pixel accuracy of the image acquisition device.

[0090] Optionally, in one specific implementation, the second determining module includes:

[0091] The third calculation submodule is used to calculate the difference between the third maximum coordinate and the third minimum coordinate of the feature point on the first coordinate axis in the second group of images, and obtain the third difference.

[0092] The fourth calculation submodule is used to calculate the difference between the fourth maximum coordinate and the fourth minimum coordinate of the feature point on the second coordinate axis in the second set of images, and obtain the fourth difference.

[0093] The second determining submodule is used to determine the repeatability error of the motion mechanism by using the camera comprehensive error, the third difference, the fourth difference, and a preset relationship, and to determine the system translation error of the motion mechanism based on the repeatability error; wherein, the preset relationship includes: the hand-eye calibration matrix of the image acquisition device and the motion mechanism, or the single-pixel accuracy of the image acquisition device.

[0094] Optionally, in one specific implementation, the preset relationship includes: the hand-eye calibration matrix; the first determining submodule includes:

[0095] The third determining submodule is used to determine the absolute positioning error of the motion mechanism by using the camera integrated error, the hand-eye calibration matrix, and multiple sets of calibration coordinates used to determine the hand-eye calibration matrix;

[0096] The fourth determining submodule is used to determine the system translation error of the motion mechanism based on the repeatability error and the absolute positioning error.

[0097] Optionally, in one specific implementation, the system rotation error of the motion mechanism is affected by the angle extraction accuracy error and the rotation accuracy error of the rotation axis; the device further includes a fourth determining module and a fifth determining module;

[0098] The fourth determining module is specifically used to calculate the difference between the maximum and minimum angles of the feature points in the first group of images to obtain the angle difference; and to determine the angle extraction accuracy error of the feature points in the first group of images using the angle difference and the rotation axis length.

[0099] The fifth determining module is specifically used to determine the rotation accuracy error of the rotating mechanism using the angle difference, the maximum value, and the rotation axis length.

[0100] Optionally, in one specific implementation, the apparatus further includes:

[0101] The sixth determining module is used to determine the rotation axis calibration error of the motion mechanism by using the hand-eye calibration matrix of the image acquisition device and the motion mechanism before determining the system rotation error of the motion mechanism by using the maximum value of the difference between the rotation axis length of the rotation axis and the angle change of the specified angle and the feature point in the third set of images;

[0102] The system rotation error determination module includes:

[0103] The fifth determining submodule is used to determine the rotation error of the rotating shaft of the motion mechanism by using the rotation axis length of the rotating shaft and the maximum value of the difference between the specified angle and the angle change of the feature point in the third set of images;

[0104] The sixth determining submodule is used to determine the system rotation error of the motion mechanism by using the rotation error of the rotating shaft and the calibration error of the rotating shaft.

[0105] Optionally, in one specific implementation, the apparatus further includes:

[0106] The seventh determining module is used to determine the teaching error of the motion mechanism before determining the system comprehensive error of the image acquisition device and the motion mechanism based on the camera comprehensive error and the system translation error;

[0107] The comprehensive error determination module is specifically used for:

[0108] Based on the camera's overall error, the system's translation error, the system's rotation error, and the teaching error, the overall system error for the image acquisition device and the motion mechanism is determined.

[0109] Optionally, in one specific implementation, the first determining submodule is specifically used for:

[0110] The overall camera error of the image acquisition device is determined using the first formula.

[0111] The first formula is:

[0112]

[0113] Wherein, CIE is the camera integrated error of the image acquisition device; M is the hand-eye calibration matrix of the image acquisition device and the motion mechanism; XPixRange_S is the first difference; YPixRange_S is the second difference;

[0114] or,

[0115] The overall camera error of the image acquisition device is determined using the second formula.

[0116] The second formula is:

[0117]

[0118] Wherein, PixAcc represents the single-pixel accuracy of the image acquisition device.

[0119] Optionally, in one specific implementation, the second determining submodule is specifically used for:

[0120] The repeatability error of the motion mechanism is determined using the third formula.

[0121] The third formula is as follows:

[0122]

[0123] Wherein, MRTE is the repeatability error of the motion mechanism; XPixRange_D is the third difference; YPixRange_D is the fourth difference; M is the hand-eye calibration matrix for the image acquisition device and the motion mechanism; CIE is the camera integrated error;

[0124] or,

[0125] The repeatability error of the motion mechanism is determined using the fourth formula.

[0126] The fourth formula is:

[0127]

[0128] Wherein, PixAcc represents the single-pixel accuracy of the image acquisition device.

[0129] Optionally, in one specific implementation, the third determining submodule is specifically used for:

[0130] The absolute positioning error of the motion mechanism is determined using the fifth formula.

[0131] The fifth formula is as follows:

[0132]

[0133] Where MATE is the absolute positioning error of the motion mechanism; X_Wld i X_Pix is ​​the X-coordinate of the calibration point in the i-th set of calibration coordinates in the world coordinate system. i The X coordinate of the calibration point in the i-th set of calibration coordinates in the image coordinate system; Y_Wld i Y_Pix is ​​the Y-coordinate of the calibration point in the i-th set of calibration coordinates in the world coordinate system. i The Y-coordinate of the calibration point in the i-th group of calibration coordinates in the image coordinate system; N is the number of calibration coordinate groups for the calibration point;

[0134] The fourth determining submodule is specifically used for:

[0135] Using the sixth formula, the system translation error of the motion mechanism is determined:

[0136] The sixth formula is as follows:

[0137]

[0138] Wherein, STLE is the system translation error of the motion mechanism; MRTE is the repeatability error of the motion mechanism; and MATE is the absolute positioning error of the motion mechanism.

[0139] Optionally, in one specific implementation, the sixth determining submodule is specifically used for:

[0140] The system rotational error of the motion mechanism is determined using the seventh formula.

[0141] The seventh formula is as follows:

[0142]

[0143] Wherein, SRE is the system rotation error of the motion mechanism; MRAE is the system rotation error of the rotation axis; MRCE is the preset rotation axis calibration error of the motion mechanism; and deltaR is the maximum value among the differences between the specified angle and the angle change of the feature point in the third set of images.

[0144] Thirdly, embodiments of this application provide an electronic device, including:

[0145] Memory, used to store computer programs;

[0146] When a processor executes a program stored in memory, it implements the steps of any of the above method embodiments.

[0147] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of any of the above method embodiments.

[0148] Fifthly, embodiments of this application also provide a computer program product containing instructions that, when run on a computer, cause the computer to perform the steps of any of the above method embodiments.

[0149] Beneficial effects of the embodiments in this application:

[0150] As can be seen from the above, by applying the solution provided in the embodiments of this application, the positional relationship and related motions among the image acquisition device, the target object, and the motion mechanism can be utilized to acquire a first set of images and a second set of images of the target object acquired by the image acquisition device. Then, using these two sets of images, the camera overall error and system translation error included in the system concerning the image acquisition device and the motion mechanism can be determined respectively. Therefore, based on the aforementioned camera overall error and system translation error, the system overall error can be determined. Furthermore, since the errors of each component in the system are comprehensively considered, the accuracy of the determined system overall error is improved.

[0151] Of course, implementing any product or method of this application does not necessarily require achieving all of the advantages described above at the same time. Attached Figure Description

[0152] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other embodiments can be obtained based on these drawings.

[0153] Figure 1 A flowchart illustrating an error assessment method provided in an embodiment of this application;

[0154] Figure 2(a)-Figure 2(b) These are schematic diagrams showing the positional relationship between the image acquisition device and the motion mechanism provided in the embodiments of this application;

[0155] Figure 3 A flowchart illustrating another error assessment method provided in an embodiment of this application;

[0156] Figure 4 A schematic diagram illustrating the system rotation error provided in an embodiment of this application;

[0157] Figure 5 A flowchart illustrating a specific example of the error assessment method provided in this application embodiment;

[0158] Figure 6 This is a schematic diagram of the structure of an error assessment device provided in an embodiment of this application;

[0159] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0160] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art based on this application are within the scope of protection of this application.

[0161] Typically, errors in a machine vision system will affect its operational accuracy. Therefore, how to assess the overall system error of a machine vision system and use the determined overall system error for error compensation is a technical problem that urgently needs to be solved.

[0162] To address the aforementioned technical problems, this application provides an error assessment method.

[0163] This method can be applied to various application scenarios that require evaluation of the system integration error of a machine vision system with respect to image acquisition equipment and motion mechanism, such as determining the system integration error of a camera and a robotic arm mounted on a fixed base, or determining the system integration error of a system consisting of a punch and a camera.

[0164] Furthermore, this method can be applied to various image acquisition devices with image data capabilities, such as cameras and camcorders. After acquiring various images of the target object, this method can be executed to determine the overall system error of the image acquisition device and the motion mechanism. It can also be applied to various electronic devices that can communicate with the image acquisition device, such as servers, mobile phones, and computers. The electronic device can acquire various images of the target object acquired by the image acquisition device and then execute this method to determine the overall system error of the image acquisition device and the motion mechanism. In addition, when the subject of this method is an electronic device, the electronic device can be a single electronic device or a cluster of multiple electronic devices. For clarity, it will be referred to as an electronic device below.

[0165] Therefore, the embodiments of this application do not specifically limit the application scenarios and execution entities of the method.

[0166] An error assessment method provided in this application embodiment may include the following steps:

[0167] Acquire a first set of images and a second set of images of a target object from an image acquisition device; wherein, the first set of images is acquired by the image acquisition device at a first position and the target object at a second position; the second set of images is acquired by the image acquisition device of the target object each time the motion mechanism rotates from an initial position to a target position so that the image acquisition device and the target object are in a specified relative position;

[0168] Using the first maximum and first minimum coordinates of the feature points of the target object on the first coordinate axis of the image coordinate system corresponding to the image acquisition device in the first set of images, and the second maximum and second minimum coordinates of the feature points on the second axis of the image coordinate system, the camera comprehensive error of the image acquisition device is determined;

[0169] The system translation error of the motion mechanism is determined by using the camera integrated error, the third maximum and third minimum coordinates of the feature points in the second set of images on the first coordinate axis, and the fourth maximum and fourth minimum coordinates of the feature points in the second set of images on the second coordinate axis.

[0170] Based on the camera's overall error and the system's translation error, the overall system error for the image acquisition device and the motion mechanism is determined.

[0171] As can be seen from the above, by applying the solution provided in the embodiments of this application, the positional relationship and related motions among the image acquisition device, the target object, and the motion mechanism can be utilized to acquire a first set of images and a second set of images of the target object acquired by the image acquisition device. Then, using these two sets of images, the camera overall error and system translation error included in the system concerning the image acquisition device and the motion mechanism can be determined respectively. Therefore, based on the aforementioned camera overall error and system translation error, the system overall error can be determined. Furthermore, since the errors of each component in the system are comprehensively considered, the accuracy of the determined system overall error is improved.

[0172] Typically, to improve operational accuracy, machine vision systems, which combine image acquisition equipment and motion mechanisms, are applied to the production processes of various high-precision products, with the machine vision system performing the production tasks. For example, to improve the precision of grasping electronic components, a machine vision system combining image acquisition equipment and a robotic arm can be used to perform component grasping tasks. During production, the image acquisition equipment acquires an image of the product, determines its position in the image, and then uses the homography transformation between the image coordinate system and the world coordinate system to determine the product's position in the world coordinate system. The motion mechanism can then operate on the product based on the position determined by the image acquisition equipment.

[0173] In a machine vision system, the positional relationship between the image acquisition device and the motion mechanism can be either eye-in-hand or eye-to-hand; where "eye" refers to the image acquisition device and "hand" refers to the motion mechanism.

[0174] The term "eye on hand" refers to an image acquisition device mounted on a motion mechanism that can move with the motion mechanism. For example, in the case of a robotic arm, "eye on hand" means that an image acquisition device is mounted on the robotic arm and moves with the robotic arm.

[0175] The term "eye outside the hand" refers to an image acquisition device that is installed outside the motion mechanism and does not move with the motion mechanism. For example, in the case of a robotic arm, "eye outside the hand" means that the image acquisition device is installed outside the robotic arm and does not move with the robotic arm.

[0176] To facilitate the determination of the overall system error of a machine vision system, embodiments of this application provide various errors involved in a machine vision system.

[0177] The motion mechanisms in machine vision systems can be divided into: motion mechanisms that do not include a rotation axis and motion mechanisms that include a rotation axis.

[0178] The aforementioned motion mechanism excluding the rotary axis may include a milling machine, a slide module, or other motion mechanisms. Furthermore, the motion mechanism excluding the rotary axis may perform translational motion. For example, the XY-axis slide module may perform translational motion along the X-axis direction or the Y-axis direction in the world coordinate system corresponding to the module.

[0179] The aforementioned motion mechanism including a rotating axis may include a robotic arm, a motion module with a rotating axis, and other motion mechanisms. Furthermore, the motion mechanism including the rotating axis can perform translational and rotational motions. For example, the robotic arm can rotate around the Z-axis perpendicular to the XY plane.

[0180] Based on this, during operation, the motion mechanism in the machine vision system can perform translational motion. However, due to the influence of the manufacturing process and motion parameters, the motion mechanism will be affected by the system translation error (STLE) when performing translational motion. Therefore, the error of any motion mechanism can include the system translation error.

[0181] Furthermore, image acquisition devices in machine vision systems suffer from camera integrated error (CIE) due to various factors such as lens optical deviation, temperature drift, image edge sharpness or edge transition zone, image feature point coordinate extraction accuracy, and device shake.

[0182] Therefore, the System Total Error (STE) of a machine vision system can include the aforementioned camera total error and system translation error.

[0183] Furthermore, since the motion mechanism in some machine vision systems may include a rotational axis capable of rotational motion, the motion mechanism will also be affected by system rotational error (SRE) during rotational motion due to the influence of the mechanism's manufacturing process and motion parameters. Therefore, for a motion mechanism including a rotational axis, the error of the motion mechanism may include system translation error and system rotational error. Consequently, the overall system error of the machine vision system may include the aforementioned camera overall error, system translation error, and system rotational error.

[0184] The following sections explain the camera error, system translation error, and system rotation error involved in the overall system error of a machine vision system.

[0185] The so-called camera comprehensive error of an image acquisition device refers to the error generated by the image acquisition device under the influence of various factors such as optical error, camera temperature drift, image edge sharpness or edge transition zone, image feature point coordinate extraction accuracy, and mechanism jitter.

[0186] The so-called system translation error refers to the positional error between the actual position reached by the motion mechanism and the designated position each time it is moved to a designated position due to the influence of manufacturing process and motion parameters, when the motion mechanism repeatedly carries the image acquisition device or the target object to a designated position.

[0187] Furthermore, the translation error of this system may include machine relative translation error (MRTE) and machine absolute translation error (MATE).

[0188] Among them, the aforementioned repetitive positioning error refers to the relative positioning error of the motion mechanism. That is, when the motion mechanism makes multiple repetitive movements, due to the aforementioned repetitive positioning error, there are fluctuations between the positioning positions reached each time.

[0189] The aforementioned absolute positioning error refers to the fact that the motion mechanism cannot move to the target position with strict accuracy each time it moves, and there is a fixed positional error between the position reached and the target position.

[0190] The so-called system rotation error refers to the angular error between the rotating angle and the target angle, which is caused by the influence of the manufacturing process and motion parameters of the mechanism.

[0191] In the case where the motion mechanism includes a rotating shaft, the rotational motion of the motion mechanism is accomplished by the rotating shaft. Therefore, the system rotational error of the motion mechanism may include the rotational error of the rotating shaft of the motion mechanism.

[0192] For a motion mechanism including a rotating shaft, when the rotation axis center of the rotating shaft of the motion mechanism is not coaxial with the centroid of the end effector, the end effector rotates with the rotating shaft, and the distance between the rotation axis center of the rotating shaft and the end effector is the length of the rotating shaft.

[0193] The aforementioned end effector may include a suction nozzle, gripper, dispensing head, etc.

[0194] The aforementioned rotation axis length can be calibrated during the hand-eye calibration of the image acquisition device and motion mechanism. Since errors may exist during the calibration process, the aforementioned system rotation error also includes the Machine Rotation Axis Calibration Error (MRCE). This MRCE refers to the error present during rotation axis calibration, specifically the deviation between the calibrated rotation axis length and the actual length of the rotation axis.

[0195] In addition, the aforementioned rotational axis rotation error is affected by the rotational accuracy error (Machine Rotation Axis Dynamic Error, MRDE) and the angle extraction accuracy error (Machine Rotation Axis Static Error, MRSE).

[0196] The aforementioned rotational accuracy error is the deviation between the actual angle rotated by the rotating shaft of the motion mechanism after rotating by a fixed angle and the aforementioned fixed angle.

[0197] The aforementioned angle extraction accuracy error refers to the deviation between the true angle of the feature point of the target object and the image-extracted angle of that feature point, affected by the accuracy of the image feature point angle extraction algorithm. It can also refer to the deviation between the actual coordinates and the ideal coordinates reached by the rotation axis of the motion mechanism after rotating according to the change in the image angle of that feature point.

[0198] Furthermore, both the rotational accuracy error and the angle extraction accuracy error mentioned above are related to the length of the rotation axis of the motion mechanism.

[0199] For clarity, the calculation methods for the camera composite error, system translation error, and system rotation error described above will be explained below.

[0200] Based on the errors involved in the machine vision system described in the embodiments of this application, the embodiments of this application propose an error assessment method for determining the overall system error of a machine vision system.

[0201] The following, with reference to the accompanying drawings, provides a detailed description of an error assessment method provided in an embodiment of this application.

[0202] Figure 1 This is a flowchart illustrating an error assessment method provided in an embodiment of this application, as shown below. Figure 1 As shown, the method may include the following steps S101-S104:

[0203] S101: Acquire the first set of images and the second set of images about the target object acquired by the image acquisition device;

[0204] The first set of images is acquired by the image acquisition device located at the first position and the target object located at the second position; the second set of images is acquired by the image acquisition device and the target object each time the motion mechanism rotates from the initial position to the target position so that the image acquisition device and the target object are in a specified relative position.

[0205] As mentioned earlier, the overall system error of the image acquisition device and the motion mechanism includes both camera overall error and system translation error. Therefore, in order to determine the overall system error, the camera overall error and the system translation error can be calculated separately.

[0206] Due to the influence of camera overall error, when an image acquisition device repeatedly captures images of the same object from a stationary position, the pixel coordinates of feature points on that object in the image coordinate system corresponding to the image acquisition device may differ. Furthermore, the range of coordinates of this feature point along each coordinate axis of the image coordinate system can reflect the camera overall error of the image acquisition device.

[0207] The so-called range of coordinates of a feature point along a coordinate axis refers to the difference between the maximum and minimum coordinates of the feature point on that coordinate axis.

[0208] Based on this, in order to calculate the overall camera error of the image acquisition device, the first set of images obtained by the image acquisition device taking multiple still shots of the target object can be acquired. That is, the image acquisition device can be placed at a first position, and the target object can be placed at a second position within the acquisition range of the image acquisition device. Then, the image acquisition device can be controlled to acquire multiple images of the target object, thereby obtaining the first set of images of the target object.

[0209] To calculate the system translation error of the motion mechanism, the motion mechanism in the initial position can be controlled to translate from the initial position to the target position each time. In this way, when the motion mechanism translates to the target position, the image acquisition device and the target object can be in a specified relative position. When the target position is reached, the image acquisition device is controlled to acquire images of the target object, thereby obtaining a second set of images of the target object.

[0210] If the positional relationship between the motion mechanism and the image acquisition device is that the eye is on the hand, the target object can be placed in a fixed position. Then, the motion mechanism can be controlled to carry the image acquisition device to repeatedly move from the initial position to the target position where the image of the target object can be acquired. Each time the target position is reached, the image acquisition device is controlled to acquire the image of the target object, thereby obtaining a second set of images.

[0211] If the positional relationship between the motion mechanism and the image acquisition device is that the eye is outside the hand, the image acquisition device can be installed in a fixed position. Then, the motion mechanism can be controlled to carry the target object repeatedly from the initial position to the target position within the acquisition area of ​​the image acquisition device. Each time the target position is reached, the image acquisition device is controlled to acquire an image of the target object, thereby obtaining a second set of images.

[0212] Thus, after acquiring the first set of images and the second set of images of the target object by the image acquisition device, the camera overall error and the system translation error can be calculated using the first set of images and the second set of images, respectively.

[0213] S102: Using the first maximum and first minimum coordinates of the feature points of the target object on the first coordinate axis of the image coordinate system corresponding to the image acquisition device in the first set of images, and the second maximum and second minimum coordinates of the feature points on the second axis of the image coordinate system, determine the camera comprehensive error of the image acquisition device;

[0214] Each image in the first set of images can be used to determine multiple image coordinates of the feature points of the target object in the image coordinate system. Then, using these multiple image coordinates, the first maximum and first minimum coordinates of the feature points of the target object on the first coordinate axis of the image coordinate system, as well as the second maximum and second minimum coordinates of the feature points on the second coordinate axis of the image coordinate system, can be determined. Finally, using these first maximum, first minimum, second maximum, and second minimum coordinates, the overall camera error of the image acquisition device can be determined.

[0215] Optionally, in one specific implementation, step S102 above may include the following steps 11-13:

[0216] Step 11: Calculate the difference between the first maximum coordinate and the first minimum coordinate of the feature points of the target object on the first coordinate axis of the image coordinate system corresponding to the image acquisition device in the first set of images, and obtain the first difference;

[0217] Step 12: Calculate the difference between the second maximum coordinate and the second minimum coordinate of the feature points on the second coordinate axis of the image coordinate system in the first set of images, and obtain the second difference;

[0218] Step 13: Determine the overall camera error of the image acquisition device using the first difference, the second difference, and the preset relationship;

[0219] The preset relationships include: the hand-eye calibration matrix of the image acquisition device and the motion mechanism, or the single-pixel accuracy of the image acquisition device.

[0220] In this specific implementation, after acquiring the first set of images of the target object acquired by the image acquisition device, the difference between the first maximum coordinate and the first minimum coordinate of the feature points of the target object on the first coordinate axis of the image coordinate system corresponding to the image acquisition device in the first set of images can be calculated to obtain the first difference; and the difference between the second maximum coordinate and the second minimum coordinate of the feature points on the second coordinate axis of the image coordinate system in the first set of images can be calculated to obtain the second difference.

[0221] Since the first and second differences mentioned above are coordinate differences in the image coordinate system, while the camera overall error to be calculated is the error in the world coordinate system, based on this, when calculating the camera overall error using the first and second differences mentioned above, the hand-eye calibration matrix of the image acquisition device and the motion mechanism, or the single-pixel accuracy of the image acquisition device, can be used to achieve the purpose of calculating the camera overall error of the image acquisition device in the world coordinate system using the image coordinates in the image coordinate system.

[0222] Among them, the aforementioned hand-eye calibration matrix for image acquisition equipment and motion mechanism is pre-calibrated using calibration objects, and is used to characterize the transformation relationship between the image coordinates of the calibration object in the image coordinate system and the world coordinates of the calibration object in the world coordinate system.

[0223] The world coordinate system described above is a three-dimensional rectangular coordinate system that reflects the position of objects in the real world. The origin of the world coordinate system can be determined based on the actual situation.

[0224] The image coordinate system described above is a two-dimensional rectangular coordinate system that reflects the arrangement of pixels in the image acquisition device. The origin of the image coordinate system is located at the upper left corner of the image acquired by the image acquisition device, and the X-axis and Y-axis are parallel to the two sides of the image plane, respectively. The unit of the coordinate axes in the image coordinate system is pixels (integers).

[0225] Furthermore, as shown in Figure 2(a), if the relationship between the image acquisition device and the motion mechanism is that the eye is outside the hand, the hand-eye calibration matrix can be calibrated using the eye-outside-hand calibration method; while as shown in Figure 2(b), if the relationship between the image acquisition device and the motion mechanism is that the eye is on the hand, the hand-eye calibration matrix can be calibrated using the eye-on-hand calibration method. Since the above-mentioned eye-outside-hand calibration method and eye-on-hand calibration method can be set according to actual needs, they can be Zhang Youzheng calibration method or other calibration methods, which are all reasonable and are not specifically limited in the embodiments of this application.

[0226] The single-pixel precision of the image acquisition device mentioned above is used to characterize the proportional relationship between the coordinate dimensions in the image coordinate system and the coordinate dimensions in the world coordinate system. The single-pixel precision is the difference between the unidirectional field of view of the image acquisition device and the unidirectional resolution of the image acquisition device, and the unit is mm / pixel.

[0227] Furthermore, the aforementioned hand-eye calibration matrix for the image acquisition device and motion mechanism, as well as the single-pixel accuracy of the image acquisition device, can be referred to as the homography transformation relationship between the image coordinate system and the world coordinate system.

[0228] Typically, when a machine vision system has not yet been calibrated or entered the debugging stage, single-pixel accuracy can be used to evaluate the overall system error. However, when the machine vision system has been calibrated or entered the debugging stage, a hand-eye calibration matrix can be used to evaluate the overall system error, thus obtaining a more comprehensive and accurate overall system error evaluation result.

[0229] After determining the first difference and the second difference, the camera overall error of the image acquisition device can be determined using the first difference, the second difference, and the preset relationship.

[0230] The first and second differences mentioned above can be expressed as follows:

[0231] XPixRange_S=XPixMax_S-XPixMin_S

[0232] YPixRange_S=YPixMax_S-YPixMin_S

[0233] Wherein, XPixRange_S is the first difference, XPixMax_S is the first maximum coordinate of the feature point of the target object on the first coordinate axis of the image coordinate system; XPixMin_S is the first minimum coordinate of the feature point of the target object on the first coordinate axis of the image coordinate system; YPixRange_S is the second difference, YPixMax_S is the second maximum coordinate of the feature point of the target object on the second coordinate axis of the image coordinate system; YPixMin_S is the second minimum coordinate of the feature point of the target object on the second coordinate axis of the image coordinate system.

[0234] Optionally, in one specific implementation, when the aforementioned preset relationship is a hand-eye calibration matrix relating to the image acquisition device and the motion mechanism, step 13, which uses the first difference, the second difference, and the preset relationship to determine the camera comprehensive error of the image acquisition device, may include the following step 131:

[0235] Step 131: Use the first formula to determine the overall camera error of the image acquisition device;

[0236] The first formula is:

[0237]

[0238] Where CIE is the camera integrated error of the image acquisition device; M is the hand-eye calibration matrix of the image acquisition device and the motion mechanism; XPixRange_S is the first difference; and YPixRange_S is the second difference.

[0239] Optionally, in one specific implementation, when the aforementioned preset relationship relates to the single-pixel accuracy of the image acquisition device, step 13, which uses the first difference, the second difference, and the preset relationship to determine the overall camera error of the image acquisition device, may include the following step 132:

[0240] Step 132: Use the second formula to determine the overall camera error of the image acquisition device;

[0241] The second formula is:

[0242]

[0243] Where PixAcc represents the single-pixel accuracy of the image acquisition device.

[0244] S103: Using the camera's overall error, the third maximum and third minimum coordinates of the feature points in the second set of images on the first coordinate axis, and the fourth maximum and fourth minimum coordinates of the feature points in the second set of images on the second coordinate axis, determine the system translation error of the motion mechanism;

[0245] Each image in the second set of images can be used to determine multiple image coordinates of the feature points of the target object in the image coordinate system. Then, the third maximum and third minimum coordinates of the feature points of the target object on the first coordinate axis of the image coordinate system, as well as the fourth maximum and fourth minimum coordinates of the feature points on the second coordinate axis of the image coordinate system, are determined. Using these third maximum, third minimum, fourth maximum, and fourth minimum coordinates, the system translation error of the motion mechanism is determined.

[0246] In determining the system translation error of the motion mechanism using the second set of images, the camera error needs to be removed because each image acquired by the image acquisition device includes the camera's overall error.

[0247] Furthermore, in the second set of images, the range of coordinates of the feature points of the target object in each coordinate axis of the image coordinate system can reflect the repetitive positioning error; while the absolute positioning error of the motion mechanism can be determined using multiple sets of calibration coordinates used for hand-eye calibration matrix.

[0248] Based on this, in one optional implementation, step S103 may include the following steps 21-23:

[0249] Step 21: Calculate the difference between the third maximum coordinate and the third minimum coordinate of the feature point on the first coordinate axis in the second set of images, and obtain the third difference;

[0250] Step 22: Calculate the difference between the fourth maximum coordinate and the fourth minimum coordinate of the feature points on the second coordinate axis in the second set of images, and obtain the fourth difference;

[0251] Step 23: Using the camera's overall error, the third difference, the fourth difference, and the preset relationship, determine the repeatability error of the motion mechanism, and based on the repeatability error, determine the system translation error of the motion mechanism;

[0252] The preset relationships include: the hand-eye calibration matrix of the image acquisition device and the motion mechanism, or the single-pixel accuracy of the image acquisition device.

[0253] In this specific implementation, after acquiring the second set of images of the target object acquired by the image acquisition device, the difference between the third maximum coordinate and the third minimum coordinate of the feature points of the target object on the first coordinate axis of the image coordinate system corresponding to the image acquisition device in the second set of images can be calculated to obtain the third difference; and the difference between the fourth maximum coordinate and the fourth minimum coordinate of the feature points on the second coordinate axis of the image coordinate system in the second set of images can be calculated to obtain the fourth difference.

[0254] The first and second differences mentioned above can be expressed as follows:

[0255] XPixRange_D=XPixMax_D-XPixMin_D

[0256] YPixRange_D=YPixMax_D-YPixMin_D

[0257] Wherein, XPixRange_D is the third difference, XPixMax_D is the third maximum coordinate of the feature point of the target object on the first coordinate axis of the image coordinate system; XPixMin_D is the third minimum coordinate of the feature point of the target object on the first coordinate axis of the image coordinate system; YPixRange_D is the fourth difference, YPixMax_D is the fourth maximum coordinate of the feature point of the target object on the second coordinate axis of the image coordinate system; YPixMin_D is the fourth minimum coordinate of the feature point of the target object on the second coordinate axis of the image coordinate system.

[0258] Since the third and fourth differences mentioned above are coordinate differences in the image coordinate system, while the camera overall error to be calculated is an error in the world coordinate system, based on this, when using the third and fourth differences to calculate the system translation error, the hand-eye calibration matrix of the image acquisition device and the motion mechanism, or the single-pixel accuracy of the image acquisition device, can be used to achieve the purpose of calculating the system translation error of the motion mechanism using the image coordinates in the image coordinate system.

[0259] Furthermore, when determining the system translation error of the motion mechanism using images acquired by the image acquisition device, the calculation result includes the camera comprehensive error. Therefore, the repeatability error of the motion mechanism can be determined using the camera comprehensive error, the aforementioned third difference, the aforementioned fourth difference, and the preset relationship.

[0260] Furthermore, after calculating the aforementioned repeatability error, the system translation error of the motion mechanism can be determined based on the aforementioned repeatability error.

[0261] Since the absolute positioning error of the motion mechanism can only be determined using the hand-eye calibration matrix, it is impossible to calculate the absolute positioning error when the machine vision system has not yet completed calibration or entered the debugging stage. In this case, when the above-mentioned preset relationship only includes the single-pixel accuracy of the image acquisition device, the repeatability error can be calculated using the above-mentioned single-pixel accuracy, and the repeatability error can be used as the system translation error of the motion mechanism.

[0262] Optionally, in one specific implementation, step 23 above, which uses the camera's overall error, the third difference, the fourth difference, and the preset relationship to determine the repeatability error of the motion mechanism, may include the following step 231:

[0263] Step 231: Use the fourth formula to determine the repeatability error of the motion mechanism;

[0264] The fourth formula is:

[0265]

[0266] Wherein, MRTE represents the repeatability error of the motion mechanism.

[0267] When the machine vision system has completed calibration or entered the debugging stage, that is, when the above-mentioned preset relationship includes the hand-eye calibration matrix of the image acquisition device and the motion mechanism, the above-mentioned hand-eye calibration matrix can be used to calculate the repeatability error and the absolute positioning error, and based on the above-mentioned repeatability error and absolute positioning error, the system translation error of the motion mechanism can be calculated.

[0268] Based on this, the preset relationship may include: hand-eye calibration matrix;

[0269] Optionally, in one specific implementation, step 23 above, which uses the camera's overall error, the third difference, the fourth difference, and the preset relationship to determine the repeatability error of the motion mechanism, may include the following step 232:

[0270] The repeatability error of the motion mechanism is determined using the third formula.

[0271] The third formula is:

[0272]

[0273] Correspondingly, optionally, in one specific implementation, step 23 above, determining the system translation error of the motion mechanism based on the repetitive positioning error, may include the following steps 233-234:

[0274] Step 233: Determine the absolute positioning error of the motion mechanism using the camera integrated error, the hand-eye calibration matrix, and multiple sets of calibration coordinates used to determine the hand-eye calibration matrix;

[0275] Step 234: Determine the system translation error of the motion mechanism based on the repeatability error and the absolute positioning error.

[0276] In this specific implementation, when the preset relationship includes the hand-eye calibration matrix, the absolute positioning error of the motion mechanism can be determined by using the camera comprehensive error, the hand-eye calibration matrix, and multiple sets of calibration coordinates used to determine the hand-eye calibration matrix.

[0277] Each set of calibration coordinates used to determine the hand-eye calibration matrix may include the world coordinates of the calibration object in the world coordinate system and the image coordinates of the calibration point in the image coordinate system.

[0278] After calculating the repeatability error and the absolute positioning error, the sum of the squares of the repeatability error and the absolute positioning error can be calculated to obtain the system translation error of the motion mechanism.

[0279] Optionally, in one specific implementation, step 233 above may include the following step 233A:

[0280] Step 233A: Use the fifth formula to determine the absolute positioning error of the motion mechanism;

[0281] The fifth formula is:

[0282]

[0283] Where MATE represents the absolute positioning error of the motion mechanism; X_Wld i X_Pix is ​​the X-coordinate of the calibration point in the i-th set of calibration coordinates in the world coordinate system. i Y_Wld is the X-coordinate of the calibration point in the i-th set of calibration coordinates in the image coordinate system; i Y_Pix is ​​the Y-coordinate of the calibration point in the i-th set of calibration coordinates in the world coordinate system. i The y-coordinate of the calibration point in the i-th calibration coordinate group in the image coordinate system; N is the number of calibration coordinate groups for the calibration point.

[0284] Accordingly, step 234 above may include the following step 234A:

[0285] Step 234A: Using the sixth formula, determine the system translation error of the motion mechanism:

[0286] The sixth formula is:

[0287]

[0288] Where STLE represents the system translation error of the motion mechanism.

[0289] S104: Based on the camera's overall error and the system's translation error, determine the overall system error of the image acquisition device and the motion mechanism.

[0290] After determining the aforementioned camera integrated error and system translation error, the system integrated error of the image acquisition device and motion mechanism can be determined based on these errors.

[0291] Optionally, after determining the above-mentioned camera integrated error and system translation error, the sum of the above-mentioned camera integrated error and system translation error can be calculated as the system integrated error regarding the image acquisition device and motion mechanism.

[0292] The overall system error can be expressed as:

[0293] STE = CIE + STLE

[0294] STE refers to the system-wide error relating to the image acquisition device and the motion mechanism.

[0295] Optionally, after determining the above-mentioned camera integrated error and system translation error, the weights of the above-mentioned camera integrated error and system translation error can be determined. Then, based on the above-mentioned camera integrated error, the weights of the camera integrated error, the system translation error, and the weights of the system translation error, the system integrated error of the image acquisition device and the motion mechanism can be calculated.

[0296] The overall system error can be expressed as:

[0297] STE=αCIE+βSTLE

[0298] Where α is the weight of the camera's overall error; β is the weight of the system's translation error.

[0299] As can be seen from the above, by applying the solution provided in the embodiments of this application, the positional relationship and related motions among the image acquisition device, the target object, and the motion mechanism can be utilized to acquire a first set of images and a second set of images of the target object acquired by the image acquisition device. Then, using these two sets of images, the camera overall error and system translation error included in the system concerning the image acquisition device and the motion mechanism can be determined respectively. Therefore, based on the aforementioned camera overall error and system translation error, the system overall error can be determined. Furthermore, since the errors of each component in the system are comprehensively considered, the accuracy of the determined system overall error is improved.

[0300] Furthermore, when the aforementioned motion mechanism includes a rotating shaft, the system error may also include the system rotation error of the motion mechanism.

[0301] Based on this, such as Figure 3 As shown, the error assessment method provided in this application embodiment may further include the following steps S105-S106 before step S104:

[0302] S105: Acquire a third set of images of the target object acquired by the image acquisition device;

[0303] The third set of images is formed when the rotating axis rotates multiple times along the first direction, causing the target object to rotate multiple times along the second direction relative to the image acquisition device. The image acquisition device captures the target object every time the rotating axis rotates by a specified angle.

[0304] S106: Determine the system rotation error of the motion mechanism by using the rotation axis length of the rotation axis and the maximum value of the difference between the specified angle and the angle change of the feature points in the third set of images;

[0305] Accordingly, step S104 above may include the following step S1041:

[0306] S1041: Determine the system comprehensive error of the image acquisition device and motion mechanism based on the camera comprehensive error, system translation error, and system rotation error.

[0307] In this specific implementation, when the motion mechanism includes a rotation axis, the motion mechanism can perform rotational motion. Therefore, in order to calculate the system rotation error of the motion mechanism, the motion mechanism can be controlled to rotate continuously multiple times along the first direction, so that the target object rotates continuously multiple times relative to the image acquisition device along the second direction. Furthermore, during the rotation of the motion mechanism, when the motion mechanism rotates by a specified angle, the image acquisition device can be controlled to acquire images of the target object, thereby obtaining a third set of images of the target object.

[0308] The first direction and the second direction mentioned above can be the same or different.

[0309] Specifically, if the positional relationship between the motion mechanism and the image acquisition device is such that the eye is on the hand, the motion mechanism can be controlled to carry the image acquisition device to rotate continuously multiple times along the first direction, thereby causing the target object to rotate continuously multiple times relative to the image acquisition device along the second direction; in this case, the first direction and the second direction are the same direction.

[0310] If the positional relationship between the motion mechanism and the image acquisition device is such that the eye is outside the hand, the motion mechanism can be controlled to carry the target object and rotate it multiple times in a first direction, thereby causing the target object to rotate multiple times in a second direction relative to the image acquisition device; in this case, the first direction and the second direction are different directions.

[0311] Therefore, during the process of controlling the rotation of the motion mechanism, the image acquisition device can be controlled to acquire a third-type image of the target object every time the motion mechanism rotates by a specified angle. The motion mechanism can be controlled to stop rotating when the number of third-type images acquired by the image acquisition device reaches a preset number, or when the motion mechanism rotates by a specified number of specified angles.

[0312] The specified angle, preset quantity, or specified quantity can be set according to actual needs. For example, the specified angle can be 10 degrees, 15 degrees, etc., and the preset quantity can be 5, 10, etc., or the specified quantity can be 5, 10, etc. All of these are reasonable and are not specifically limited in this application embodiment.

[0313] After acquiring the third set of images, for each image in the third set, the angular change of the feature points of the target object in that image can be determined. Furthermore, the maximum value among the differences between the specified angle and the angular change of the feature points of the target object in the third set of images can be determined. Additionally, the rotation axis length of the motion mechanism can be determined. Then, using the maximum value and the rotation axis length of the motion mechanism, the system rotation error of the motion mechanism can be determined.

[0314] The system rotation error can include the rotation error of the motion mechanism's rotation axis and the rotation axis calibration error. Since the rotation axis calibration error of the motion mechanism can only be determined using the hand-eye calibration matrix, it cannot be calculated when the machine vision system has not yet completed calibration or entered the debugging stage. In this case, the rotation axis rotation error can be determined using the above-mentioned maximum value and the rotation axis length of the motion mechanism, and the above-mentioned rotation axis rotation error can be used as the system rotation error.

[0315] The rotation error of the rotating shaft of the above-mentioned motion mechanism can be expressed as:

[0316]

[0317] Where L is the length of the rotation axis; deltaR is the maximum value among the differences between the specified angle and the angle change of the feature points of the target object in the third set of images.

[0318] Optionally, when the rotation axis center of the aforementioned motion mechanism is coaxial with the centroid of the end effector, the rotation error of the rotation axis can be determined using the aforementioned maximum value and the rotation axis length of the motion mechanism, and the aforementioned rotation error of the rotation axis can be used as the system rotation error.

[0319] When the machine vision system has been calibrated or has entered the debugging stage, and when the rotation axis center of the aforementioned motion mechanism is not coaxial with the centroid of the end effector, the rotation axis calibration error can be determined using the hand-eye calibration matrix. Thus, the system rotation error of the motion mechanism can be determined based on the rotation axis rotation error and the rotation axis calibration error.

[0320] Based on this, optionally, in one specific implementation, before S105 above, the error evaluation method provided in this application embodiment may further include the following step 31:

[0321] Step 31: Using the hand-eye calibration matrix of the image acquisition device and the motion mechanism, determine the calibration error of the rotation axis of the motion mechanism;

[0322] Accordingly, step S106 above may include the following steps 32-33:

[0323] Step 32: Determine the rotation error of the motion mechanism's rotating axis by using the rotation axis length and the maximum value of the difference between the specified angle and the angle change of the feature points in the third set of images;

[0324] Step 33: Determine the system rotation error of the motion mechanism using the rotation error and calibration error of the rotating shaft.

[0325] In this specific implementation, the rotation axis calibration error of the motion mechanism can be determined by using the hand-eye calibration matrix of the image acquisition device and the motion mechanism.

[0326] Optionally, the aforementioned rotation axis length can be determined by the hand-eye calibration matrix of the image acquisition device and the motion mechanism, as well as the physical coordinates of the rotation center of the motion mechanism, obtained when performing hand-eye calibration of the image acquisition device and the motion mechanism using the 12-point calibration method.

[0327] Specifically, hand-eye calibration can be performed using designated calibration points to obtain a rotational fitting circle. Thus, the fitting error of the rotational fitting circle of the motion mechanism is the rotational axis calibration error.

[0328] The specified calibration points can be multiple calibration points among the 12 calibration points, such as the last three points, the last two points, or all 12 calibration points. All of these are reasonable and are not specifically limited in this implementation.

[0329] After determining the calibration error of the rotating shaft of the motion mechanism, the system rotation error of the motion mechanism can be determined based on the rotation error and the calibration error of the rotating shaft.

[0330] Optionally, in one specific implementation, step 33 above may include the following step 331:

[0331] Step 331: Use the seventh formula to determine the system rotation error of the motion mechanism;

[0332] The seventh formula is:

[0333]

[0334] To facilitate understanding of the system rotational error of the aforementioned motion mechanism, we can utilize... Figure 4The system rotational error of this motion mechanism is explained.

[0335] like Figure 4 As shown, point P is the true position of the feature point of the target object, and point P′ is the actual position reached by the motion mechanism after rotating by a specified angle. The length of the line connecting the two points represents the system rotation error of the motion mechanism.

[0336] Let the center of rotation be point O, the actual length of the rotation axis be OP, the calibrated length of the rotation axis obtained by calibration be line segment OP′, the intersection of line OP′ and circle OP be point C, and the maximum value of the difference between the specified angle and the angle change of the feature points in the third set of images be deltaR.

[0337] In the figure, the length of line segment CP is the rotation error MRAE of the rotating shaft of the motion mechanism, the length of line segment CP′ is the calibration error MRCE of the rotating shaft of the motion mechanism, and the length of line segment P′P is the system rotation error SRE of the motion mechanism.

[0338] From the knowledge of trigonometric functions, we know that:

[0339] ∠P′CP=180-(180-deltaR) / 2

[0340]

[0341] Therefore, SRE can be expressed as:

[0342]

[0343] After determining the above-mentioned system rotation error, the overall system error of the image acquisition device and motion mechanism can be determined based on the above-mentioned camera comprehensive error, system translation error, and system rotation error.

[0344] Optionally, after determining the above-mentioned camera integrated error, system translation error, and system rotation error, the sum of the above-mentioned camera integrated error, system translation error, and system rotation error can be calculated as the system integrated error regarding the image acquisition device and the motion mechanism.

[0345] The overall system error can be expressed as:

[0346] STE = CIE + STLE + SRE

[0347] Optionally, after determining the aforementioned camera integrated error, system translation error, and system rotation error, the weights of the aforementioned camera integrated error, system translation error, and system rotation error can be determined. Then, based on the aforementioned camera integrated error, the weights of the camera integrated error, the system translation error, the system rotation error, and the system rotation error, the system integrated error for the image acquisition device and the motion mechanism can be calculated.

[0348] The overall system error can be expressed as:

[0349] STE=αCIE+βSTLE+γSRE

[0350] Where γ is the weight of the system rotation error.

[0351] Since the rotation error of the above system is caused by insufficient accuracy of the image extraction algorithm and insufficient accuracy of the mechanism rotation, the rotation error of the system is affected by the angle extraction accuracy error and the rotation accuracy error of the rotation axis.

[0352] In some cases, users may need to analyze the factors affecting the rotation error of the system. For example, when the rotation error of the system is large, the error ratios of the angle extraction accuracy error and the rotation accuracy error of the rotation axis can be analyzed. Thus, the angle extraction accuracy error and the rotation accuracy error of the rotation axis can be calculated separately, so that error analysis and parameter adjustment can be performed based on the calculation results.

[0353] Optionally, in one specific implementation, the method for determining the angle extraction accuracy error may include the following steps 41-42:

[0354] Step 41: Calculate the difference between the maximum and minimum angles of the feature points in the first set of images to obtain the angle difference;

[0355] Step 42: Using the angle difference and rotation axis length, determine the angle extraction accuracy error for feature points in the first set of images;

[0356] Accordingly, the method for determining the angle extraction accuracy error may include the following step 43:

[0357] Step 43: Use the angle difference, maximum value, and rotation axis length to determine the rotational accuracy error of the motion mechanism.

[0358] In this specific implementation, when extracting angles from an image, the extracted angles are affected by the angle extraction accuracy error due to factors such as the accuracy of the image feature point extraction algorithm, the quality of the image edge, and lens distortion, resulting in a certain deviation.

[0359] Based on this, in order to determine the aforementioned angle extraction accuracy error, after acquiring the first set of images, the maximum and minimum angles of the feature points of the target object can be determined using each image in the first set of images.

[0360] In the first set of images, for each point on the target object, the maximum angle of each point is the same, and the minimum angle of each point is also the same.

[0361] After determining the maximum and minimum angles of the feature points of the target object in the first set of images, the difference between the maximum and minimum angles can be calculated to obtain the angle difference.

[0362] The aforementioned angle difference can be expressed as:

[0363] RPixRange_S=RPixMax_S-RPixMin_S

[0364] Wherein, RPixRange_S is the angle difference; RPixMax_S is the maximum angle of the feature points of the target object in the first set of images; and RPixMin_S is the minimum angle of the feature points of the target object in the first set of images.

[0365] Then, the angle extraction accuracy error of the feature points in the first set of images can be determined by using the aforementioned angle difference and the rotation axis length of the motion mechanism.

[0366] The angle extraction accuracy error of the feature points in the first set of images can be expressed as follows:

[0367]

[0368] Furthermore, the rotation angle of this motion mechanism is affected by the rotation accuracy error of the motion mechanism, and there is a certain deviation.

[0369] Based on this, in order to determine the aforementioned rotational accuracy error, the maximum value among the aforementioned angle difference, the difference between the specified angle and the angle change of the feature points in the third set of images, and the length of the rotation axis can be used to determine the rotational accuracy error of the motion mechanism.

[0370] When calculating the aforementioned rotational accuracy error using images, the influence of angle extraction accuracy error must also be considered. Therefore, the aforementioned rotational accuracy error of the motion mechanism can be expressed as:

[0371]

[0372] After calculating the angle extraction accuracy error and rotation accuracy error, the angle extraction accuracy error and rotation accuracy error can be output so that users can analyze the rotation error of the system based on the angle extraction accuracy error and rotation accuracy error and adjust the various parameters of the machine vision system.

[0373] In addition, the overall system error of a machine vision system may also include the system teaching error (STHE) of the motion mechanism.

[0374] Teaching error refers to the error caused by factors such as human eye resolution and human operation error when setting the reference motion posture of a motion mechanism. The aforementioned reference motion posture can include setting a grasping posture, a placement posture, or a contact posture.

[0375] Typically, this teaching error can be determined by measurement using measuring tools or by re-verification using an image sensor. For example, after the operator controls the motion mechanism to place the material into the reference position in the tray, while keeping the material and tray stationary, an image of the center of the material relative to the center of the tray is captured using a camera. Then, the coordinate deviation between the center of the material and the center of the tray is calculated using the captured image, thus obtaining the teaching error.

[0376] Optionally, after obtaining the teaching error, the teaching error can be compensated.

[0377] In this embodiment of the application, in order to improve the accuracy of the determined system integration error, the influence of the teaching error on the system integration error can be considered when calculating the system integration error.

[0378] Based on this, in one optional implementation, before determining the system comprehensive error of the image acquisition device and the motion mechanism based on the camera comprehensive error and the system translation error in step S104 above, the error assessment method provided in this application embodiment may further include the following step 51:

[0379] Step 51: Determine the teaching error of the motion mechanism;

[0380] Accordingly, step S104 above, which determines the system comprehensive error of the image acquisition device and the motion mechanism based on the camera comprehensive error and the system translation error, may include step 52:

[0381] Step 52: Based on the camera integrated error, system translation error, and teaching error, determine the system integrated error of the image acquisition device and motion mechanism.

[0382] In this specific implementation, in order to improve the accuracy of the determined system comprehensive error, the teaching error of the motion mechanism can also be determined.

[0383] Optionally, the device parameters used to indicate the teaching error can be read from the device parameters of the machine vision system.

[0384] Optionally, the teaching error of the motion mechanism can be determined using a teaching error calibration model.

[0385] Optionally, after the target object is moved to the reference position by the manually controlled motion mechanism, the target object is kept stationary. Then, the coordinate deviation is measured using the image of the target object acquired by the image acquisition device, thereby obtaining the teaching error.

[0386] After determining the teaching error mentioned above, the system comprehensive error of the image acquisition device and motion mechanism can be determined based on the camera comprehensive error, system translation error, and teaching error.

[0387] The overall system error can be expressed as:

[0388] STE = CIE + STLE + STHE

[0389] Optionally, when the aforementioned motion mechanism includes a rotation axis, the system comprehensive error regarding the image acquisition device and the motion mechanism can be determined based on the camera comprehensive error, system translation error, system rotation error, and teaching error.

[0390] The overall system error can be expressed as:

[0391] STE = CIE + STLE + SRE + STHE

[0392] To facilitate understanding of the error assessment method in the embodiments of this application, the following will be combined with... Figure 5 The specific implementation process and effects of the error assessment method are explained.

[0393] For machine vision systems involving cameras and motion mechanisms, error evaluation procedures can be performed as follows: Figure 5 The steps shown are used to calculate the overall system error.

[0394] like Figure 5 As shown, the error assessment process may include the following steps S501-S504:

[0395] S501: Camera static test, obtain coordinate range, calculate camera overall error;

[0396] S502: Dynamic testing of the mechanism to obtain the coordinate range and calculate the system translation error;

[0397] S503: Mechanism rotation test, obtain the angle range and rotation axis length, and calculate the system rotation error;

[0398] S504: Mechanism teaching test, obtain teaching error, and calculate system teaching error.

[0399] During static camera testing, the target object can be placed at a fixed position within the camera's image acquisition range, while maintaining the camera's position. Then, the camera can be controlled to continuously acquire images of the target object several times (generally no less than 100 times), obtaining a first set of images of the target object. Next, using the first maximum and first minimum coordinates of the target object's feature points on the first coordinate axis of the image coordinate system corresponding to the image acquisition device, and the second maximum and second minimum coordinates of the feature points on the second coordinate axis of the image coordinate system, the coordinate range of the feature points on each coordinate axis of the image coordinate system is obtained. This coordinate range is then used to calculate the overall camera error.

[0400] When conducting dynamic testing of a mechanism, if a camera is mounted on the motion mechanism, a target object can be placed at a fixed position within the camera's image acquisition range. Then, the motion mechanism is controlled to move from the initial position to the target position multiple times, so that the image acquisition device and the target object are in a specified relative position. Each time the camera moves to the target position, it is controlled to acquire images of the target object, thus obtaining a second set of images of the target object.

[0401] If the camera is mounted outside the motion mechanism and its position relative to the base of the motion mechanism is fixed, the motion mechanism can be controlled to move from the initial position to the target position multiple times, so that the image acquisition device and the target object are in a specified relative position. Each time the camera moves to the target position, it is controlled to acquire an image of the target object, thus obtaining a second set of images of the target object.

[0402] After obtaining the second set of images, the third maximum and third minimum coordinates of the feature points on the first coordinate axis, and the fourth maximum and fourth minimum coordinates of the feature points on the second coordinate axis can be used to obtain the coordinate range of the feature points on each coordinate axis of the image coordinate system. The coordinate range and the camera comprehensive error can then be used to determine the system translation error.

[0403] When conducting a mechanism rotation test, if the camera is mounted on the motion mechanism, the target object is placed at a fixed position within the camera's image acquisition range. Then, the motion mechanism is controlled to rotate continuously from the initial position in a clockwise or counterclockwise direction by a fixed angle several times (generally no less than three times). Each time the motion mechanism rotates by a fixed angle, the camera is controlled to acquire images of the target object, thus obtaining a third set of images of the target object.

[0404] If the camera is mounted outside the motion mechanism and its position relative to the base of the motion mechanism is fixed, the motion mechanism can be controlled to rotate continuously from its initial position to a fixed angle several times (generally no less than three times). Each time the motion mechanism rotates to a fixed angle, the camera is controlled to acquire images of the target object, thus obtaining a third set of images of the target object.

[0405] After obtaining the third set of images, the rotation axis length and angular range of the motion mechanism can be obtained, and the system rotation error of the motion mechanism can be determined.

[0406] Then, the teaching error of the system can be obtained through institutional teaching tests. Subsequently, using the aforementioned camera comprehensive error, system translation error, system rotation error, and teaching error, the system comprehensive error of the machine vision system can be calculated.

[0407] Furthermore, by using the first set of images, the second set of images, and the third set of images, the camera overall error, system translation error, system rotation error, and system overall error are calculated. After calculating the camera overall error, system translation error, system rotation error, system teaching error, and system overall error, these errors can be output in the preset display interface to help users understand the system overall error of the system composed of the camera and the motion mechanism.

[0408] Based on the same inventive concept, and corresponding to the embodiments provided in this application above... Figure 1 The present application provides an error assessment device in addition to the error assessment method shown in the embodiment.

[0409] Figure 6 This is a schematic diagram of the structure of an error assessment device provided in an embodiment of this application, as shown below. Figure 6 As shown, the device may include the following modules:

[0410] The image acquisition module 610 is used to acquire a first set of images and a second set of images of a target object acquired by an image acquisition device; wherein, the first set of images is acquired by the image acquisition device at a first position and the target object at a second position; the second set of images is acquired by the image acquisition device of the target object each time the motion mechanism rotates from an initial position to a target position so that the image acquisition device and the target object are in a specified relative position;

[0411] The first determining module 620 is used to determine the camera comprehensive error of the image acquisition device by using the first maximum coordinates and the first minimum coordinates of the feature points of the target object on the first coordinate axis of the image coordinate system corresponding to the image acquisition device in the first group of images, and the second maximum coordinates and the second minimum coordinates of the feature points on the second axis of the image coordinate system.

[0412] The second determining module 630 is used to determine the system translation error of the motion mechanism by using the camera comprehensive error, the third maximum coordinate and the third minimum coordinate of the feature point in the second set of images on the first coordinate axis, and the fourth maximum coordinate and the fourth minimum coordinate of the feature point in the second set of images on the second coordinate axis.

[0413] The comprehensive error determination module 640 is used to determine the comprehensive system error of the image acquisition device and the motion mechanism based on the comprehensive camera error and the system translation error.

[0414] As can be seen from the above, by applying the solution provided in the embodiments of this application, the positional relationship and related motions among the image acquisition device, the target object, and the motion mechanism can be utilized to acquire a first set of images and a second set of images of the target object acquired by the image acquisition device. Then, using these two sets of images, the camera overall error and system translation error included in the system concerning the image acquisition device and the motion mechanism can be determined respectively. Therefore, based on the aforementioned camera overall error and system translation error, the system overall error can be determined. Furthermore, since the errors of each component in the system are comprehensively considered, the accuracy of the determined system overall error is improved.

[0415] Optionally, in one specific implementation, the apparatus further includes:

[0416] The third determining module is used to acquire a third set of images of the target object acquired by the image acquisition device before determining the system comprehensive error of the image acquisition device and the motion mechanism based on the camera comprehensive error and the system translation error; wherein, the third set of images is acquired by the image acquisition device when the rotation axis rotates the target object multiple times in a first direction so that the target object rotates multiple times in a second direction relative to the image acquisition device;

[0417] The system rotation error determination module is used to determine the system rotation error of the motion mechanism by using the rotation axis length of the rotation axis and the maximum value of the difference between the specified angle and the angle change of the feature point in the third set of images;

[0418] The comprehensive error determination module 640 is specifically used for:

[0419] Based on the camera's overall error, the system's translation error, and the system's rotation error, the overall system error for the image acquisition device and the motion mechanism is determined.

[0420] Optionally, in one specific implementation, the first determining module 620 includes:

[0421] The first calculation submodule is used to calculate the difference between the first maximum coordinate and the first minimum coordinate of the feature points of the target object on the first coordinate axis of the image coordinate system corresponding to the image acquisition device in the first group of images, and obtain the first difference.

[0422] The second calculation submodule is used to calculate the difference between the second maximum coordinate and the second minimum coordinate of the feature point on the second coordinate axis of the image coordinate system in the first group of images, and obtain the second difference.

[0423] The first determining submodule is used to determine the camera comprehensive error of the image acquisition device using the first difference, the second difference, and a preset relationship; wherein the preset relationship includes: the hand-eye calibration matrix of the image acquisition device and the motion mechanism, or the single-pixel accuracy of the image acquisition device.

[0424] Optionally, in one specific implementation, the second determining module 630 includes:

[0425] The third calculation submodule is used to calculate the difference between the third maximum coordinate and the third minimum coordinate of the feature point on the first coordinate axis in the second group of images, and obtain the third difference.

[0426] The fourth calculation submodule is used to calculate the difference between the fourth maximum coordinate and the fourth minimum coordinate of the feature point on the second coordinate axis in the second set of images, and obtain the fourth difference.

[0427] The second determining submodule is used to determine the repeatability error of the motion mechanism by using the camera comprehensive error, the third difference, the fourth difference, and a preset relationship, and to determine the system translation error of the motion mechanism based on the repeatability error; wherein, the preset relationship includes: the hand-eye calibration matrix of the image acquisition device and the motion mechanism, or the single-pixel accuracy of the image acquisition device.

[0428] Optionally, in one specific implementation, the preset relationship includes: the hand-eye calibration matrix; the first determining submodule includes:

[0429] The third determining submodule is used to determine the absolute positioning error of the motion mechanism by using the camera integrated error, the hand-eye calibration matrix, and multiple sets of calibration coordinates used to determine the hand-eye calibration matrix;

[0430] The fourth determining submodule is used to determine the system translation error of the motion mechanism based on the repeatability error and the absolute positioning error.

[0431] Optionally, in one specific implementation, the system rotation error of the motion mechanism is affected by the angle extraction accuracy error and the rotation accuracy error of the rotation axis; the device further includes a fourth determining module and a fifth determining module;

[0432] The fourth determining module is specifically used to calculate the difference between the maximum and minimum angles of the feature points in the first group of images to obtain the angle difference; and to determine the angle extraction accuracy error of the feature points in the first group of images using the angle difference and the rotation axis length.

[0433] The fifth determining module is specifically used to determine the rotation accuracy error of the rotating mechanism using the angle difference, the maximum value, and the rotation axis length.

[0434] Optionally, in one specific implementation, the apparatus further includes:

[0435] The sixth determining module is used to determine the rotation axis calibration error of the motion mechanism by using the hand-eye calibration matrix of the image acquisition device and the motion mechanism before determining the system rotation error of the motion mechanism by using the maximum value of the difference between the rotation axis length of the rotation axis and the angle change of the specified angle and the feature point in the third set of images;

[0436] The system rotation error determination module includes:

[0437] The fifth determining submodule is used to determine the rotation error of the rotating shaft of the motion mechanism by using the rotation axis length of the rotating shaft and the maximum value of the difference between the specified angle and the angle change of the feature point in the third set of images;

[0438] The sixth determining submodule is used to determine the system rotation error of the motion mechanism by using the rotation error of the rotating shaft and the calibration error of the rotating shaft.

[0439] Optionally, in one specific implementation, the apparatus further includes:

[0440] The seventh determining module is used to determine the teaching error of the motion mechanism before determining the system comprehensive error of the image acquisition device and the motion mechanism based on the camera comprehensive error and the system translation error;

[0441] The comprehensive error determination module 640 is specifically used for:

[0442] Based on the camera's overall error, the system's translation error, the system's rotation error, and the teaching error, the overall system error for the image acquisition device and the motion mechanism is determined.

[0443] Optionally, in one specific implementation, the first determining submodule is specifically used for:

[0444] The overall camera error of the image acquisition device is determined using the first formula.

[0445] The first formula is:

[0446]

[0447] Wherein, CIE is the camera integrated error of the image acquisition device; M is the hand-eye calibration matrix of the image acquisition device and the motion mechanism; XPixRange_S is the first difference; YPixRange_S is the second difference;

[0448] or,

[0449] The overall camera error of the image acquisition device is determined using the second formula.

[0450] The second formula is:

[0451]

[0452] Wherein, PixAcc represents the single-pixel accuracy of the image acquisition device.

[0453] Optionally, in one specific implementation, the second determining submodule is specifically used for:

[0454] The repeatability error of the motion mechanism is determined using the third formula.

[0455] The third formula is as follows:

[0456]

[0457] Wherein, MRTE is the repeatability error of the motion mechanism; XPixRange_D is the third difference; YPixRange_D is the fourth difference; M is the hand-eye calibration matrix for the image acquisition device and the motion mechanism; and CIE is the camera's overall error.

[0458] Alternatively, the fourth formula can be used to determine the repeatability error of the motion mechanism;

[0459] The fourth formula is:

[0460]

[0461] Wherein, PixAcc represents the single-pixel accuracy of the image acquisition device.

[0462] Optionally, in one specific implementation, the third determining submodule is specifically used for:

[0463] The absolute positioning error of the motion mechanism is determined using the fifth formula.

[0464] The fifth formula is as follows:

[0465]

[0466] Where MATE is the absolute positioning error of the motion mechanism; X_Wld i X_Pix is ​​the X-coordinate of the calibration point in the i-th set of calibration coordinates in the world coordinate system. i The X coordinate of the calibration point in the i-th set of calibration coordinates in the image coordinate system; Y_Wld i Y_Pix is ​​the Y-coordinate of the calibration point in the i-th set of calibration coordinates in the world coordinate system. i The Y-coordinate of the calibration point in the i-th group of calibration coordinates in the image coordinate system; N is the number of calibration coordinate groups for the calibration point;

[0467] The fourth determining submodule is specifically used for:

[0468] Using the sixth formula, the system translation error of the motion mechanism is determined:

[0469] The sixth formula is as follows:

[0470]

[0471] Wherein, STLE is the system translation error of the motion mechanism; MRTE is the repeatability error of the motion mechanism; and MATE is the absolute positioning error of the motion mechanism.

[0472] Optionally, in one specific implementation, the sixth determining submodule is specifically used for:

[0473] The system rotational error of the motion mechanism is determined using the seventh formula.

[0474] The seventh formula is as follows:

[0475]

[0476] Wherein, SRE is the system rotation error of the motion mechanism; MRAE is the system rotation error of the rotating shaft; and MRCE is the preset calibration error of the rotating shaft of the motion mechanism.

[0477] This application also provides an electronic device, such as... Figure 7 As shown, it includes:

[0478] Memory 701 is used to store computer programs;

[0479] When the processor 702 executes the program stored in the memory 701, it implements the steps of any error evaluation method provided in the embodiments of this application.

[0480] Furthermore, the aforementioned electronic device may also include a communication bus and / or a communication interface, with the processor 702, the communication interface, and the memory 701 communicating with each other via the communication bus.

[0481] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0482] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0483] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0484] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0485] In another embodiment provided in this application, a computer-readable storage medium is also provided, which stores a computer program that, when executed by a processor, implements the steps of any of the above error evaluation methods.

[0486] In another embodiment provided in this application, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to perform any of the error evaluation methods described above.

[0487] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a solid-state drive (SSD), etc.

[0488] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0489] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the device embodiments, electronic device embodiments, and storable medium embodiments are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0490] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application are included within the scope of protection of this application.

Claims

1. An error assessment method, characterized in that, The method includes: Acquire a first set of images and a second set of images of a target object from an image acquisition device; wherein, the first set of images is acquired by the image acquisition device at a first position and the target object at a second position; the second set of images is acquired by the image acquisition device of the target object each time the motion mechanism rotates from an initial position to a target position so that the image acquisition device and the target object are in a specified relative position; Using the difference between the first maximum coordinate and the first minimum coordinate of the feature points of the target object on the first coordinate axis of the image coordinate system corresponding to the image acquisition device in the first set of images, and the difference between the second maximum coordinate and the second minimum coordinate of the feature points on the second axis of the image coordinate system, the camera comprehensive error of the image acquisition device is determined. The system translation error of the motion mechanism is determined by using the camera's overall error, the difference between the third maximum coordinate and the third minimum coordinate of the feature point in the second set of images on the first coordinate axis, and the difference between the fourth maximum coordinate and the fourth minimum coordinate of the feature point in the second set of images on the second coordinate axis. Based on the camera's overall error and the system's translation error, the overall system error for the image acquisition device and the motion mechanism is determined.

2. The method according to claim 1, characterized in that, The motion mechanism includes a rotation axis; before determining the system comprehensive error regarding the image acquisition device and the motion mechanism based on the camera comprehensive error and the system translation error, the method further includes: Acquire a third set of images of the target object acquired by the image acquisition device; wherein, the third set of images is acquired by the image acquisition device when the rotation axis rotates a specified angle each time the rotation axis rotates, during the process of the rotation axis rotating multiple times continuously along the first direction so that the target object rotates multiple times continuously relative to the image acquisition device along the second direction. The system rotation error of the motion mechanism is determined by using the rotation axis length of the rotation axis and the maximum value of the difference between the specified angle and the angle change of the feature point in the third set of images; The determination of the system comprehensive error regarding the image acquisition device and the motion mechanism based on the camera comprehensive error and the system translation error includes: Based on the camera's overall error, the system's translation error, and the system's rotation error, the overall system error for the image acquisition device and the motion mechanism is determined.

3. The method according to claim 1, characterized in that, The step of determining the camera overall error of the image acquisition device by using the difference between the first maximum coordinate and the first minimum coordinate of the feature points of the target object on the first coordinate axis of the image coordinate system corresponding to the image acquisition device in the first set of images, and the difference between the second maximum coordinate and the second minimum coordinate of the feature points on the second coordinate axis of the image coordinate system, includes: Calculate the difference between the first maximum coordinate and the first minimum coordinate of the feature points of the target object on the first coordinate axis of the image coordinate system corresponding to the image acquisition device in the first group of images, and obtain the first difference; Calculate the difference between the second maximum coordinate and the second minimum coordinate of the feature point on the second coordinate axis of the image coordinate system in the first group of images to obtain the second difference; The camera overall error of the image acquisition device is determined using the first difference, the second difference, and a preset relationship; wherein the preset relationship includes: the hand-eye calibration matrix of the image acquisition device and the motion mechanism, or the single-pixel accuracy of the image acquisition device.

4. The method according to claim 1, characterized in that, The method of determining the system translation error of the motion mechanism by utilizing the camera's overall error, the difference between the third maximum and third minimum coordinates of the feature points in the second set of images on the first coordinate axis, and the difference between the fourth maximum and fourth minimum coordinates of the feature points in the second set of images on the second coordinate axis includes: Calculate the difference between the third maximum coordinate and the third minimum coordinate of the feature point on the first coordinate axis in the second set of images to obtain the third difference; Calculate the difference between the fourth maximum coordinate and the fourth minimum coordinate of the feature point on the second coordinate axis in the second set of images to obtain the fourth difference; Using the camera's overall error, the third difference, the fourth difference, and a preset relationship, the repeatability error of the motion mechanism is determined, and based on the repeatability error, the system translation error of the motion mechanism is determined; wherein, the preset relationship includes: the hand-eye calibration matrix of the image acquisition device and the motion mechanism, or the single-pixel accuracy of the image acquisition device.

5. The method according to claim 4, characterized in that, The preset relationship includes: the hand-eye calibration matrix; the determination of the system translation error of the motion mechanism based on the repetitive positioning error includes: The absolute positioning error of the motion mechanism is determined by using the camera integrated error, the hand-eye calibration matrix, and multiple sets of calibration coordinates used to determine the hand-eye calibration matrix. Based on the repeatability error and the absolute positioning error, the system translation error of the motion mechanism is determined.

6. The method according to claim 2, characterized in that, The system rotational error of the motion mechanism is affected by the angle extraction accuracy error and the rotation accuracy error of the rotation axis; the determination method of the angle extraction accuracy error includes: Calculate the difference between the maximum and minimum angles of the feature points in the first set of images to obtain the angle difference; Using the angle difference and the rotation axis length, the angle extraction accuracy error for the feature points in the first set of images is determined. The method for determining the rotational accuracy error includes: The rotational accuracy error with respect to the rotational axis is determined using the angle difference, the maximum value, and the length of the rotational axis.

7. The method according to claim 6, characterized in that, Before determining the system rotation error of the motion mechanism using the maximum value of the difference between the rotation axis length of the rotation axis and the angular change of the specified angle and the feature point in the third set of images, the method further includes: Using the hand-eye calibration matrix of the image acquisition device and the motion mechanism, the rotation axis calibration error of the motion mechanism is determined; The determination of the system rotation error of the motion mechanism by utilizing the rotation axis length of the rotation axis and the maximum value of the difference between the specified angle and the angular change of the feature point in the third set of images includes: The rotation error of the motion mechanism is determined by using the rotation axis length of the rotation axis and the maximum value of the difference between the specified angle and the angle change of the feature point in the third set of images. The system rotation error of the motion mechanism is determined by using the rotation error of the rotating shaft and the calibration error of the rotating shaft.

8. The method according to any one of claims 1-7, characterized in that, Before determining the system-wide integrated error of the image acquisition device and the motion mechanism based on the camera integrated error and the system translation error, the method further includes: Determine the teaching error of the motion mechanism; The determination of the system comprehensive error regarding the image acquisition device and the motion mechanism based on the camera comprehensive error and the system translation error includes: Based on the camera's overall error, the system translation error, and the teaching error, the overall system error concerning the image acquisition device and the motion mechanism is determined.

9. The method according to claim 3, characterized in that, The step of determining the camera overall error of the image acquisition device using the first difference, the second difference, and a preset relationship includes: The overall camera error of the image acquisition device is determined using the first formula. The first formula is: ; in, The overall camera error of the image acquisition device; This is a hand-eye calibration matrix relating to the image acquisition device and the motion mechanism; This is the first difference; This is the second difference; or, The overall camera error of the image acquisition device is determined using the second formula. The second formula is: ; in, This refers to the single-pixel accuracy of the image acquisition device.

10. The method according to claim 4, characterized in that, The method of determining the repeatability error of the motion mechanism by utilizing the camera's overall error, the third difference, the fourth difference, and a preset relationship includes: The repeatability error of the motion mechanism is determined using the third formula. The third formula is as follows: ; in, This refers to the repeatability error of the motion mechanism; The third difference, The fourth difference is M; M is the hand-eye calibration matrix for the image acquisition device and the motion mechanism. This refers to the overall error of the camera; or, The repeatability error of the motion mechanism is determined using the fourth formula. The fourth formula is: ; in, This refers to the single-pixel accuracy of the image acquisition device.

11. The method according to claim 5, characterized in that, The method of determining the absolute positioning error of the motion mechanism using the camera's overall error, the hand-eye calibration matrix, and multiple sets of calibration coordinates used to determine the hand-eye calibration matrix includes: The absolute positioning error of the motion mechanism is determined using the fifth formula. The fifth formula is as follows: ; in, The absolute positioning error of the motion mechanism; Let X be the X-coordinate of the calibration point in the i-th set of calibration coordinates in the world coordinate system; The X-coordinate of the calibration point in the i-th set of calibration coordinates in the image coordinate system; The Y-coordinate of the calibration point in the i-th set of calibration coordinates in the world coordinate system; The Y-coordinate of the calibration point in the i-th group of calibration coordinates in the image coordinate system; N is the number of calibration coordinate groups for the calibration point; This is a hand-eye calibration matrix relating to the image acquisition device and the motion mechanism; The determination of the system translation error of the motion mechanism based on the repeatability error and the absolute positioning error includes: Using the sixth formula, the system translation error of the motion mechanism is determined: The sixth formula is as follows: ; in, The system translation error of the motion mechanism; This refers to the repeatability error of the motion mechanism; The absolute positioning error of the motion mechanism is given.

12. The method according to claim 7, characterized in that, The determination of the system rotation error of the motion mechanism using the rotation error of the rotating shaft and the calibration error of the rotating shaft includes: The system rotational error of the motion mechanism is determined using the seventh formula. The seventh formula is as follows: ; in, The system rotational error of the motion mechanism; The rotational error of the rotating axis; The calibration error of the rotating axis; The maximum value among the differences between the specified angle and the angle change of the feature point in the third set of images.

13. An error assessment device, characterized in that, The device includes: An image acquisition module is used to acquire a first set of images and a second set of images of a target object acquired by an image acquisition device; wherein, the first set of images is acquired by the image acquisition device at a first position and the target object at a second position; the second set of images is acquired by the image acquisition device of the target object each time the motion mechanism rotates from an initial position to a target position so that the image acquisition device and the target object are in a specified relative position; The first determining module is used to determine the camera overall error of the image acquisition device by using the difference between the first maximum coordinate and the first minimum coordinate of the feature points of the target object on the first coordinate axis of the image coordinate system corresponding to the image acquisition device in the first group of images, and the difference between the second maximum coordinate and the second minimum coordinate of the feature points on the second axis of the image coordinate system. The second determining module is used to determine the system translation error of the motion mechanism by using the camera comprehensive error, the difference between the third maximum coordinate and the third minimum coordinate of the feature point in the second set of images on the first coordinate axis, and the difference between the fourth maximum coordinate and the fourth minimum coordinate of the feature point in the second set of images on the second coordinate axis. The comprehensive error determination module is used to determine the comprehensive system error of the image acquisition device and the motion mechanism based on the comprehensive camera error and the system translation error.

14. The apparatus according to claim 13, characterized in that, The motion mechanism includes a rotating shaft; the device further includes: The third determining module is used to acquire a third set of images of the target object acquired by the image acquisition device before determining the system comprehensive error of the image acquisition device and the motion mechanism based on the camera comprehensive error and the system translation error; wherein, the third set of images is acquired by the image acquisition device when the rotation axis rotates the target object multiple times in a first direction so that the target object rotates multiple times in a second direction relative to the image acquisition device; The system rotation error determination module is used to determine the system rotation error of the motion mechanism by using the rotation axis length of the rotation axis and the maximum value of the difference between the specified angle and the angle change of the feature point in the third set of images; The comprehensive error determination module is specifically used for: Based on the camera's overall error, the system's translation error, and the system's rotation error, determine the overall system error relating to the image acquisition device and the motion mechanism; And / or, The first determining module includes: The first calculation submodule is used to calculate the difference between the first maximum coordinate and the first minimum coordinate of the feature points of the target object on the first coordinate axis of the image coordinate system corresponding to the image acquisition device in the first group of images, and obtain the first difference. The second calculation submodule is used to calculate the difference between the second maximum coordinate and the second minimum coordinate of the feature point on the second coordinate axis of the image coordinate system in the first group of images, and obtain the second difference. The first determining submodule is used to determine the camera comprehensive error of the image acquisition device using the first difference, the second difference, and a preset relationship; wherein the preset relationship includes: a hand-eye calibration matrix of the image acquisition device and the motion mechanism, or a single pixel accuracy of the image acquisition device; And / or, The second determining module includes: The third calculation submodule is used to calculate the difference between the third maximum coordinate and the third minimum coordinate of the feature point on the first coordinate axis in the second group of images, and obtain the third difference. The fourth calculation submodule is used to calculate the difference between the fourth maximum coordinate and the fourth minimum coordinate of the feature point on the second coordinate axis in the second set of images, and obtain the fourth difference. The second determining submodule is used to determine the repeatability error of the motion mechanism by using the camera comprehensive error, the third difference, the fourth difference, and a preset relationship, and to determine the system translation error of the motion mechanism based on the repeatability error; wherein, the preset relationship includes: the hand-eye calibration matrix of the image acquisition device and the motion mechanism, or the single-pixel accuracy of the image acquisition device; And / or, The preset relationship includes: the hand-eye calibration matrix; the first determining submodule includes: The third determining submodule is used to determine the absolute positioning error of the motion mechanism by using the camera integrated error, the hand-eye calibration matrix, and multiple sets of calibration coordinates used to determine the hand-eye calibration matrix; The fourth determining submodule is used to determine the system translation error of the motion mechanism based on the repeatability error and the absolute positioning error. And / or, The system rotation error of the motion mechanism is affected by the angle extraction accuracy error and the rotation accuracy error of the rotating shaft; the device also includes a fourth determination module and a fifth determination module; The fourth determining module is specifically used to calculate the difference between the maximum and minimum angles of the feature points in the first group of images to obtain the angle difference; and to determine the angle extraction accuracy error of the feature points in the first group of images using the angle difference and the rotation axis length. The fifth determining module is specifically used to determine the rotation accuracy error about the rotation axis using the angle difference, the maximum value, and the rotation axis length. And / or, The device further includes: The sixth determining module is used to determine the rotation axis calibration error of the motion mechanism by using the hand-eye calibration matrix of the image acquisition device and the motion mechanism before determining the system rotation error of the motion mechanism by using the maximum value of the difference between the rotation axis length of the rotation axis and the angle change of the specified angle and the feature point in the third set of images; The system rotation error determination module includes: The fifth determining submodule is used to determine the rotation error of the rotating shaft of the motion mechanism by using the rotation axis length of the rotating shaft and the maximum value of the difference between the specified angle and the angle change of the feature point in the third set of images; The sixth determining submodule is used to determine the system rotation error of the motion mechanism using the rotation error of the rotating shaft and the calibration error of the rotating shaft; And / or, The device further includes: The seventh determining module is used to determine the teaching error of the motion mechanism before determining the system comprehensive error of the image acquisition device and the motion mechanism based on the camera comprehensive error and the system translation error; The comprehensive error determination module is specifically used for: Based on the camera integrated error, the system translation error, the system rotation error, and the teaching error, determine the system integrated error for the image acquisition device and the motion mechanism; And / or, The first determining submodule is specifically used for: The overall camera error of the image acquisition device is determined using the first formula. The first formula is: ; in, The overall camera error of the image acquisition device; This is a hand-eye calibration matrix relating to the image acquisition device and the motion mechanism; This is the first difference; This is the second difference; or, The overall camera error of the image acquisition device is determined using the second formula. The second formula is: ; in, The resolution per pixel is related to the image acquisition device. And / or, The second determining submodule is specifically used for: The repeatability error of the motion mechanism is determined using the third formula. The third formula is as follows: ; in, This refers to the repeatability error of the motion mechanism; The third difference, The fourth difference is M; M is the hand-eye calibration matrix for the image acquisition device and the motion mechanism. This refers to the overall error of the camera; or, The repeatability error of the motion mechanism is determined using the fourth formula. The fourth formula is: ; in, The resolution per pixel is related to the image acquisition device. And / or, The third determining submodule is specifically used for: The absolute positioning error of the motion mechanism is determined using the fifth formula. The fifth formula is as follows: ; in, The absolute positioning error of the motion mechanism; Let X be the X-coordinate of the calibration point in the i-th set of calibration coordinates in the world coordinate system; The X-coordinate of the calibration point in the i-th set of calibration coordinates in the image coordinate system; The Y-coordinate of the calibration point in the i-th set of calibration coordinates in the world coordinate system; The Y-coordinate of the calibration point in the i-th group of calibration coordinates in the image coordinate system; N is the number of calibration coordinate groups for the calibration point; The fourth determining submodule is specifically used for: Using the sixth formula, the system translation error of the motion mechanism is determined: The sixth formula is as follows: ; in, The system translation error of the motion mechanism; This refers to the repeatability error of the motion mechanism; The absolute positioning error of the motion mechanism; And / or, The sixth determining submodule is specifically used for: The system rotational error of the motion mechanism is determined using the seventh formula. The seventh formula is as follows: ; in, The system rotational error of the motion mechanism; The rotational error of the rotating shaft system; The calibration error of the rotation axis of the preset motion mechanism; The maximum value among the differences between the specified angle and the angle change of the feature point in the third set of images.

15. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the method described in any one of claims 1-12.

16. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method described in any one of claims 1-12.

Images