Systems for machine vision

Abstract

The present invention relates to systems and methods for machine vision. The machine vision system includes a motion rendering device, a first image sensor and a second image sensor. The machine vision system includes a processor configured to run a computer program stored in a memory, the program configured to determine a first transformation that allows mapping between a first coordinate system associated with the motion rendering device and a second coordinate system associated with the first image sensor, and to determine a second transformation that allows mapping between the first coordinate system associated with the motion rendering device and a third coordinate system associated with the first image sensor.

Description

[0001] This application is a divisional application of the invention application with the application date of August 1, 2016, application number 201610621798.0, and name “Machine Vision System Calibration”.

[0002] CROSS-REFERENCE TO RELATED APPLICATIONS

[0003] This application claims the benefit under 35 U.S.C. §119(e) of Provisional Patent Application No. 62 / 199,459, filed on July 31, 2015, and entitled “MACHINE VISION SYSTEM CALIBRATION,” the entire contents of which are hereby incorporated by reference in their entirety. Technical Field

[0004] The disclosed apparatus, system, and computing method generally relate to systems for machine vision (may be referred to as machine vision systems for short), and more particularly, to calibrating machine vision systems. Background Art

[0005] In a machine vision system, one or more image capture devices can be utilized to perform vision system processing on an object or surface within an imaging scene. These processing can include inspection, imaging / symbol decoding, object alignment, and / or a variety of other tasks. In some embodiments, a machine vision system can be used to inspect workpieces disposed within or passing through an imaging scene. This scene can be imaged by one or more image capture devices. Images captured by the image capture devices can be processed by a machine vision system to determine information about features within the imaging scene, such as information about the real-world position, pose, and the like of the features. In some applications, the image capture devices can be calibrated to allow the machine vision system to perform visual tasks with increased accuracy and reliability.

[0006] In some cases, a calibration target can be used to calibrate the image capture device in the vision system. The calibration target can be any object with accurately known (e.g., due to the manufacturing process) or measurable dimensional features. For example, the calibration target can include a calibration plate. The calibration plate can be a planar object with different patterns visible on its surface. Different patterns can be arranged so that the machine vision system or user can easily identify the visual features in the image of the calibration plate acquired by the image capture device. Some exemplary patterns include a dot grid, a line grid, a honeycomb pattern, a square chessboard, a triangle chessboard, etc. This pattern generally includes highly locatable features. The level of accuracy in the positioning of the features may affect the accuracy of the calibration.

[0007] The characteristics of each visible feature are known by the design of the board, such as the position and / or orientation with respect to a reference position and / or coordinate system well defined in the design. Features present in the design may include calibration features. Calibration features may be individual features in the design of the calibration board whose characteristics can be easily measured (e.g., the location, size, color, etc. of the feature) and whose expected characteristics are known by calibrating the design. For example, some possible calibration feature locations include the corners or centers of tiles in a checkerboard pattern, or the centers of dots in a dot grid, or the intersections of lines in a line grid.

[0008] The calibration plate design may include a large number of calibration features arranged in a repeating pattern. More fundamentally, the calibration features may be easily extracted from the acquired image and may provide known features. In some applications, known dimensional features are utilized to calibrate the machine vision system. Summary of the invention

[0009] Some embodiments include a machine vision system. The machine vision system includes one or more interfaces configured to provide communication with a motion rendering device, a first image sensor, and a second image sensor. The motion rendering device is configured to provide at least one of a translational movement and an in-plane rotational movement, and is associated with a first coordinate system. The motion rendering device is configured to directly or indirectly carry a first calibration plate and a second calibration plate, and the first calibration plate and the second calibration plate respectively include a plurality of first features having a known physical position with respect to the first calibration plate, and a plurality of second features having a known physical position with respect to the second calibration plate. The first image sensor and the second image sensor are configured to respectively capture images of the first calibration plate and the second calibration plate, and the first image sensor and the second image sensor are respectively associated with a second coordinate system and a third coordinate system. The machine vision system also includes a processor configured to run a computer program stored in a memory. The computer program stored in the memory is operable to cause the processor to send first data to the motion rendering device via one or more interfaces, the first data being constructed to cause the motion rendering device to move to a desired first pose; to receive a reported first pose from the motion rendering device via one or more interfaces; and to receive a first image of a first calibration plate from a first image sensor about the reported first pose via one or more interfaces; and to receive a second image of a second calibration plate from a second image sensor about the reported first pose via one or more interfaces. The computer program is also operable to cause the processor to determine a plurality of first correspondences between a plurality of first features on the first calibration plate and a first position of a plurality of first features in the first image; to determine a plurality of second correspondences between a plurality of second features on the second calibration plate and a second position of a plurality of second features in the second image; to determine a first transformation between a first coordinate system and a second coordinate system based at least in part on the plurality of first correspondences and the reported first pose; and to determine a second transformation between a first coordinate system and a third coordinate system based at least in part on a plurality of second communications and the reported first pose.

[0010] Some embodiments include a machine vision system. The machine vision system includes one or more interfaces configured to provide communication with a motion rendering device, a first image sensor, and a second image sensor. The motion rendering device is configured to provide at least one of a translational movement and an in-plane rotational movement, and is associated with a first coordinate system. The motion rendering device is configured to directly or indirectly carry a first calibration plate and a second calibration plate, and the first calibration plate and the second calibration plate respectively include a plurality of first features having a known physical position with respect to the first calibration plate, and a plurality of second features having a known physical position with respect to the second calibration plate. The first image sensor and the second image sensor are configured to respectively capture images of the first calibration plate and the second calibration plate, and the first image sensor and the second image sensor are respectively associated with a second coordinate system and a third coordinate system. The machine vision system also includes a processor configured to run a computer program stored in a memory. The computer program stored in the memory is operable to cause the processor to send first data to the motion rendering device via one or more interfaces, the first data being constructed to cause the motion rendering device to move to a desired first pose; to receive a reported first pose from the motion rendering device via one or more interfaces; and to receive a first image of a first calibration plate about the reported first pose from a first image sensor via one or more interfaces; and to receive a second image of a second calibration plate about the reported first pose from a second image sensor via one or more interfaces. The computer program is also operable to cause the processor to determine a plurality of first correspondences between a plurality of first features on the first calibration plate and a plurality of first positions of the first features in the first image; to determine a plurality of second correspondences between a plurality of second features on the second calibration plate and a plurality of second positions of the second features in the second image; to determine a first transformation that allows mapping between a first coordinate system associated with the motion rendering device and a second coordinate system associated with the first image sensor; and to determine a second transformation that allows mapping between a first coordinate system associated with the motion rendering device and a third coordinate system associated with the first image sensor.

[0011] Some embodiments include another type of machine vision system. The machine vision system includes one or more interfaces, which are configured to provide communication with a motion rendering device, a first image sensor, and a second image sensor. The motion rendering device is configured to provide at least one of a translational movement and an in-plane rotational movement, and is associated with a first coordinate system, and the motion rendering device is also configured to directly or indirectly carry a first image sensor and a second image sensor. The first image sensor and the second image sensor are configured to capture images of a first calibration plate and a second calibration plate, respectively, and the first image sensor and the second image sensor are associated with a second coordinate system and a third coordinate system, respectively. The first calibration plate and the second calibration plate respectively include a plurality of first features having known physical positions with respect to the first calibration plate, and a plurality of second features having known physical positions with respect to the second calibration plate. The machine vision system also includes a processor configured to run a computer program stored in a memory. The computer program is operable to cause the processor to send first data to the motion rendering device via one or more interfaces, the first data being constructed to cause the motion rendering device to move to a desired first pose; to receive a reported first pose from the motion rendering device via one or more interfaces; and to receive a first image of a first calibration plate about the reported first pose from a first image sensor via one or more interfaces; and to receive a second image of a second calibration plate about the reported first pose from a second image sensor via one or more interfaces. The computer program is further operable to cause the processor to determine a plurality of first correspondences between a plurality of first features on the first calibration plate and a first position of a plurality of first features in the first image; to determine a plurality of second correspondences between a plurality of second features on the second calibration plate and a second position of a plurality of second features in the second image; to determine a first transformation between a first coordinate system and a second coordinate system based at least in part on the plurality of first correspondences and the reported pose; and to determine a second transformation between the first coordinate system and a third coordinate system based at least in part on the plurality of second correspondences and the reported pose.

[0012] Some embodiments include another type of machine vision system. The machine vision system includes one or more interfaces configured to provide communication with a motion rendering device, a first image sensor, and a second image sensor. The motion rendering device is configured to provide at least one of a translational movement and an in-plane rotational movement, and is associated with a first coordinate system. The motion rendering device is further configured to directly or indirectly carry a target object including a plurality of features having unknown physical positions. The first image sensor and the second image sensor are configured to capture a first subset and a second subset of a plurality of features in the target object, respectively, and the first image sensor and the second image sensor are associated with a second coordinate system and a third coordinate system, respectively. The machine vision system also includes a processor configured to run a computer program stored in a memory. The computer program is operable to cause the processor to send first data to the motion rendering device via one or more interfaces, the first data being configured to cause the motion rendering device to move to a required first pose; receive a reported first pose from the motion rendering device via one or more interfaces; receive a first image of a first subset of a plurality of features of the reported first pose from the first image sensor via one or more interfaces; and receive a first image of a second subset of a plurality of features of the reported first pose from the second image sensor via one or more interfaces. The computer program is further operable to cause the processor to determine a first subset of features on the target object in the first image; determine a second subset of features on the target object in the second image; determine a first transformation between the first coordinate system and the second coordinate system based at least in part on the first subset of the multiple features and the reported first pose, and determine a second transformation between the first coordinate system and a third coordinate system based on the second subset of the multiple features and the reported first pose.

[0013] Some embodiments include another type of machine vision system. The machine vision system includes one or more interfaces, which are configured to provide communication with a motion rendering device, a first image sensor, and a second image sensor. The motion rendering device is configured to provide at least one of a translational movement and an in-plane rotational movement, and is associated with a first coordinate system. The motion rendering device is also configured to directly or indirectly carry a first image sensor and a second image sensor. The first image sensor and the second image sensor are configured to capture an image of a target object including multiple features with unknown physical positions, and the first image sensor and the second image sensor are respectively associated with a second coordinate system and a third coordinate system. The machine vision system also includes a processor configured to run a computer program stored in a memory. The computer program is operable to cause the processor to send first data to the motion rendering device via one or more interfaces, the first data being configured to cause the motion rendering device to move to a required first pose; receive a reported first pose from the motion rendering device via one or more interfaces; receive a first image of a first subset of multiple features of the reported first pose from the first image sensor via one or more interfaces; and receive a second image of a second subset of multiple features of the reported first pose from the second image sensor via one or more interfaces. The computer program is further operable to cause the processor to determine a first subset of features on the target object in the first image; determine a second subset of features on the target object in the second image; and determine a first transformation between the first coordinate system and the second coordinate system based at least in part on the first subset of the plurality of features and the reported first pose; and determine a second transformation between the first coordinate system and a third coordinate system based on the second subset of the plurality of features and the reported first pose.

[0014] In some embodiments, the computer program is operable to cause the processor to determine a motion correction transform that compensates for systematic motion errors associated with the motion rendering device.

[0015] In some embodiments, the computer program is operable to cause the processor to recalibrate the machine vision system after the first time period, including predetermining: a plurality of first correspondences, a plurality of second correspondences, a first transformation, and a second transformation. In some embodiments, recalibrating the machine vision system includes adjusting one or more pre-calibrated parameters.

[0016] There has thus been outlined rather broadly the features of the disclosed subject matter in order that its detailed description below may be better understood, and in order that the present contribution to the art may be better appreciated. There are of course other features of the disclosed subject matter which will be described hereinafter and which will form the subject of the claims appended hereto. It should be understood that the phraseology and terminology used herein are for the purpose of description and should not be regarded as limiting. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Various objects, features, and advantages of the disclosed subject matter may be more fully understood by reference to the following detailed description of the disclosed subject matter when considered in conjunction with the following drawings in which like reference numerals represent like elements.

[0018] Figure 1 A single-board based hand-eye (SPH) calibration system is shown in accordance with some embodiments.

[0019] Figure 2A illustrates the relationship between coordinate systems in a still image capture device mode according to some embodiments; Figure 2B The relationship between coordinate systems in a mobile image capture device mode according to some embodiments is shown.

[0020] Figure 3 The operation of an SPH calibration module in an SPH calibration system according to some embodiments is shown.

[0021] Figure 4 The line estimation stage of the SPH calibration module according to some embodiments is shown.

[0022] Figure 5 A tracking feature based hand-eye (TFH) calibration system is shown in accordance with some embodiments.

[0023] Figure 6 The operation of a TFH calibration module in a TFH calibration system according to some embodiments is shown.

[0024] Figure 7 The straight line estimation stage of the TFH calibration unit according to some embodiments is shown. DETAILED DESCRIPTION

[0025] In the following description, a number of specific details about the systems and methods of the disclosed subject matter and the environments in which such systems and methods can operate are set forth in order to provide a thorough understanding of the disclosed subject matter. However, it will be apparent to one skilled in the art that the disclosed subject matter can be performed without such specific details, and some features well known in the art are not described in detail to avoid complicating the disclosed subject matter. Furthermore, it should be understood that the examples provided below are exemplary, and it should be considered that other systems and methods are within the scope of the disclosed subject matter.

[0026] The following introduces exemplary descriptions of terms used throughout the specification according to one or more embodiments. These descriptions are not intended to be limiting.

[0027] Two commonly important components in automated industrial systems are machine vision systems and motion rendering devices. The vision system can provide important information about the runtime environment, allowing the motion rendering device to be situationally aware. In some high-precision tasks, the vision system can provide visual feedback of the state of the motion rendering device, so that adjustments can be made to improve the accuracy of the motion rendering device to form a closed-loop control system. Typically, machine vision systems use image capture devices to detect visual information around the motion rendering device.

[0028] In order to use the information obtained from the image capture device, the machine vision system should be able to transform between the coordinate system of the image capture device and the coordinate system of the motion rendering device. The relationship between (1) the coordinate system of the image capture device and (2) the coordinate system of the motion rendering device can be established through a procedure called hand-eye calibration, where the hand refers to the motion rendering device and the eye refers to the vision system.

[0029] Typically, hand-eye calibration involves moving a calibration target, such as a calibration plate, to a plurality of different positions using a motion rendering device. At each different position, the vision system can acquire an image around the calibration target. The calibration system can determine the transformation between the coordinate systems of the vision system and the motion rendering device by the expected motion of the motion rendering device and the motion observed by the vision system. In some cases, instead of moving the calibration target, the motion rendering device can move the image capture device relative to the calibration target to achieve the same goal.

[0030] A generally accurate and commonly used calibration target for hand-eye calibration is a single large calibration plate, which is typically larger than the field of view (FOV) of a single image capture device or larger than the area spanned by the FOVs of multiple image capture devices. A large calibration plate can include a large number of reliable features with known physical dimensions. Therefore, a large calibration plate allows a calibration system to provide accurate hand-eye calibration. However, for many applications, a large calibration plate is often too bulky. In practice, a calibration system based on a large calibration plate may require human intervention, may increase maintenance costs, or may be impractical in some applications.

[0031] Embodiments disclosed herein address problems associated with calibration systems based on large calibration plates. In particular, the disclosed embodiments provide two hand-eye calibration mechanisms that improve the flexibility of the hand-eye calibration system. The disclosed embodiments allow hand-eye calibration to be configured in applications that are not practical for calibration systems based on large calibration plates. These calibration mechanisms are referred to as a single calibration plate-based hand-eye (SPH) calibration mechanism and a tracking feature-based hand-eye (TFH) calibration mechanism, and calibration systems that implement these mechanisms are referred to as SPH calibration systems and TFH calibration systems, respectively.

[0032] In some embodiments, the SPH calibration system can be constructed to use multiple small calibration plates for hand-eye calibration, rather than a single large calibration plate. In some embodiments, each image capture device has a small calibration plate. In some embodiments, multiple small calibration plates can be mounted rigidly and parallel to the plane of motion. The use of smaller calibration plates can improve the flexibility of the calibration mechanism: because the SPH calibration system does not rely on bulky, large calibration plates, it can be configured in applications that were previously impractical. For example, on very large flat-panel televisions, it is more practical to mount a single calibration plate on the actual part than to construct a large calibration plate.

[0033] One of the typical challenges of using multiple small calibration plates is that the multiple calibration plates are not aligned relative to each other. In other words, one or more of the multiple small calibration plates may have different in-plane rotations, because it is difficult to perfectly align the in-plane rotations of the calibration plates. In addition, each of the small calibration plates is typically associated with its own unique coordinate system. Therefore, the SPH calibration system can be constructed to determine not only the relative in-plane rotations and translations between the calibration plates, but also the actual coordinate systems of the calibration plates. This challenge does not typically arise when using a single large calibration plate.

[0034] In some embodiments, the TFH calibration system uses a runtime object as a calibration target for hand-eye calibration. Because the runtime object is used as the calibration target, the TFH calibration system does not require the user to create, purchase, maintain, and store a calibration plate. In addition, the use of runtime objects can increase the flexibility of the calibration mechanism. Although the runtime object can provide a smaller number of reliable calibration features compared to a large calibration plate, the TFH calibration mechanism can increase the flexibility of the calibration system. Because the TFH calibration system does not rely on a bulky, large calibration plate, the TFH calibration system can be deployed in applications that were previously impractical.

[0035] Hand-eye calibration is typically a non-linear problem in image capture devices due to lens distortion. Unfortunately, this non-linear problem cannot be easily and effectively solved because it may include local solutions and existing non-linear optimization techniques may not be able to effectively identify the global solution.

[0036] To solve this problem, in some embodiments, the SPH calibration system and the TFH calibration system can use a two-step method for hand-eye calibration. For example, in the first step of the calibration method, the SPH calibration system can linearly approximate the relationship between the coordinate transformations and solve this linear relationship to determine an approximate solution for the hand-eye calibration. Subsequently, in the second step, the SPH calibration system can solve this actual non-linear problem by initializing a nonlinear optimization technique using the approximate solution from the first step. Since the approximate solution from the first step is close to the overall solution for the hand-eye calibration, the non-linear optimization technique can effectively identify the overall solution. A similar two-step method can also be used by the TFH calibration system.

[0037] In some embodiments, the SPH calibration system and the TFH calibration system can be operated in a static image capture device configuration. In this static image capture device configuration, the physical position of the image capture device remains unchanged when the motion rendering device moves. Typically, when calibrating an image capture device on this system, a calibration object or a runtime object is fixed to the motion rendering device and moved by the motion rendering device during the calibration process. In this configuration, the position in all image capture devices remains unchanged.

[0038] In some embodiments, the SPH calibration system and the TFH calibration system can be operated in a moving image capture device configuration. In the moving image capture device configuration, the image capture devices can be mounted on the motion rendering device so that their physical position changes relative to the original position of the motion rendering device. Typically, when calibrating image capture devices on this system, the calibration plate is fixed to the base of the machine and remains stationary during the calibration process while the motion rendering device moves the image capture devices around. In this configuration, the relative positions in all image capture devices remain unchanged. In some embodiments, the motion rendering device movement must be parallel to the calibration object or the runtime object.

[0039] In some embodiments, the SPH calibration system and the TFH calibration system can model system errors in the motion rendering device and determine the relationship between parameters used to control the motion rendering device, such as (x, y, θ), and the actual physical pose of the motion rendering device in the original coordinate system.

[0040] SPH Calibration System

[0041] Figure 1 An SPH calibration system according to some embodiments is shown. Figure 1A machine vision system 100 is shown with a motion rendering device 114. The machine vision system 100 includes one or more image capture devices 102 directed generally toward a scene including the motion rendering device 114. The image capture device 102 may include a charge coupled device (CCD) image sensor or a complementary metal oxide semiconductor (CMOS)-image sensor, a line scan sensor, a flying spot scanner, an electron microscope, an X-ray device including a computed tomography (CT) scanner, a magnetic resonance imaging machine, and / or other devices known to those skilled in the art.

[0042] In some embodiments, the machine vision system 100 includes a computing device 104. The computing device 104 may include a processor 106 and a storage device 108. The processor 106 may execute instructions and one or more storage devices 108 may store instructions and / or data. The storage device 108 may be a non-transitory computer readable medium, such as a dynamic random access memory (DRAM), a static random access memory (SRAM), a flash memory, a disk drive, an optical drive, a programmable read-only memory (PROM), a read-only memory (ROM), or any other memory or combination of memories. The storage device 108 may be used to temporarily store data. The storage device 108 may also be used for long-term data storage. The processor 106 and the storage device 108 may be supplemented by and / or incorporated into a special purpose logic circuit.

[0043] In some embodiments, the computing device 104 may include an SPH calibration module 110. The SPH calibration module 110 may be configured to adjust the SPH calibration mechanism. The SPH calibration module 110 may be configured to send first data configured to cause the motion rendering device to move to a desired first pose to the motion rendering device 114 via one or more interfaces 112. The SPH calibration module 110 is also configured to receive a reported first pose from the motion rendering device 114 via one or more interfaces 112; receive a first image of a first calibration plate about the reported first pose from a first image processor via one or more interfaces 112; receive a second image of a second calibration plate about the reported first pose from a second image sensor via one or more interfaces 112; determine a plurality of first correspondences between a plurality of first features on the first calibration plate and a first position of a plurality of first features in the first image; determine a plurality of second correspondences between a plurality of second features on the second calibration plate and a second position of a plurality of second features in the second image; determine a first transformation between the first coordinate system and the second coordinate system based at least in part on the plurality of first correspondences and the reported first pose; and determine a second transformation between the first coordinate system and the third coordinate system based at least in part on the plurality of second correspondences and the reported first pose.

[0044] In some embodiments, the SPH calibration module 110 can determine the first transformation and the second transformation in a two-step process. In the first step, the SPH calibration module 110 can construct a linear system that relates the unknown parameters, the correspondences of the features, and the reported pose of the motion rendering device 114. Subsequently, the SPH calibration module 110 can determine the estimated values ​​of the unknown parameters by iteratively solving the linear system and the nonlinear single camera calibration step. In the second step, the SPH calibration module 110 can construct a nonlinear system that minimizes the feature reprojection error, and solve the nonlinear system using a nonlinear solver. In this second step, the solution of the linear system can be used as an initial value for the nonlinear solver. In some embodiments, this transformation is solved simultaneously using all the correspondences and the reported pose of the motion reproduction device.

[0045] In some embodiments, the SPH calibration module 110 may be implemented in digital electronic circuitry, or in computer hardware, firmware, software, or a combination thereof. This implementation may be a computer program product as, for example, a computer program explicitly embodied in a machine-readable storage device, so as to be executed by a data processing device, such as a programmable processor, a computer, and / or a plurality of computers, or to control the operation of the data processing device. The computer program may be written in any form of computer or programming language, including source code, compiled code, interpreted code, and / or machine code, and may be deployed in any form, including as a stand-alone program or as a subroutine, element, or other unit suitable for use in a computing environment. The computer program may be deployed to be executed on one computer or on multiple computers at one or more locations.

[0046] In some implementations, the computing device 104 may include one or more interfaces 112. The one or more interfaces 112 may be configured to provide communication between the computing device 104 and the motion rendering device 114 or the image capture device 102. The one or more interfaces 112 may be implemented in hardware to send and receive signals in a variety of media such as optical, copper, and / or wireless interfaces and in a variety of different protocols, some of which may be non-transient.

[0047] In some embodiments, the computing device 104 may include a server. The server may be operated using operating system (OS) software. In some embodiments, the OS software is based on the Linux software kernel and runs specific applications in the server, such as monitoring tasks and providing protocol stacks. The OS software allows server resources to be allocated separately for control and data paths. For example, some packet accelerator cards and packet service cards are dedicated to performing wiring or security control functions, while other packet accelerator cards / packet service cards are dedicated to processing user session traffic. As network requirements change, hardware resources can be dynamically deployed to meet the requirements in some embodiments.

[0048] In some implementations, the motion rendering device 114 may be configured to provide motion to an object disposed on the motion rendering device 114. The motion rendering device 114 may include an end effector of a robotic arm, a motion stage, or any type of device capable of providing motion to an object disposed on the device.

[0049] In some embodiments, in a static image capture device configuration, the motion rendering device 114 can be configured to carry a plurality of calibration plates 116, and the plurality of calibration plates 116 can be imaged by the static image capture device 102. The motion rendering device 114 can be configured to translate along the x-axis 135 and / or the y-axis, thereby translating the plurality of calibration plates 116. The motion rendering device 114 can also provide in-plane rotation, thereby also rotating the calibration plates 116.

[0050] In some embodiments, each calibration plate 116 can be small enough so that each calibration plate 116 covers a majority of the field of view of a single image capture device 102. In some embodiments, the calibration target is flat and remains in the same physical plane while the calibration target is moved to multiple locations in the field of view of each image capture device.

[0051] In some embodiments, the optical axis of each image capture device need not be perpendicular to the plane of calibration target 116. The initial in-plane orientation of calibration target 116 relative to the image capture device image coordinate axes, as defined by positioning calibration target 116 on motion rendering device 114, may be arbitrary.

[0052] In some implementations, in a mobile image capture device configuration, the plurality of calibration plates 116 may be stationary and the image capture device 102 may be mounted on the motion rendering device 114 such that the image capture device 102 may be configured to move relative to the plurality of calibration plates 116 .

[0053] In some implementations, one or more images captured by image capture device 102 may be transmitted to computing device 104 for further processing. The one or more images may be transmitted via one or more interfaces 112.

[0054] In some embodiments, the computing device 104 may be configured to receive one or more images via the one or more interfaces 112, and receive the motion rendering device pose via the one or more interfaces 112, and provide the received images and the motion rendering device pose to the SPH calibration module 110. Subsequently, the SPH calibration module 110 may be configured to process the received images of the plurality of calibration plates 116 and the motion rendering device pose to determine a coordinate transformation between the coordinate system of the image capture device 102 and the coordinate system of the motion rendering device 114.

[0055] although Figure 1 A machine vision system 100 is described with four image capture devices 102, but in some implementations, the machine vision system 100 may include a smaller or larger number of image capture devices 102 that are all generally oriented toward a scene. In some implementations, a single image capture device may be used to image a scene.

[0056] although Figure 1 The motion rendering device 114 is described as being capable of translation along the x-axis 135 and / or the y-axis 140, but in some embodiments, the motion rendering device 114 is capable of providing three-dimensional (3D) motion, including a translation component along a z-axis perpendicular to the plane defined by the x-axis and the y-axis and / or at least one of a tilt / rotation out of the xy plane. In some embodiments, the computing device 104 can be completely self-contained within the image capture device 102, partially contained within the image capture device 102, or external to the image capture device 102.

[0057] Coordinate system

[0058] In some embodiments, the SPH calibration module 110 associates a coordinate system with one or more components in the calibration system. The coordinate system used in the SPH calibration module 110 can be set up as follows:

[0059] Home coordinate system (Home2D): Home2D is the base reference space in which all other coordinate spaces and their relationships are described. Home2D is orthogonal and is defined by the x-axis and the center of rotation of the motion rendering device 114. For example, the x-axis of Home2D is aligned with the x-axis of the motion rendering device, and Home2D is defined as 90 degrees to the x-axis. The positive y-axis direction is along the same direction as the y-axis of the motion rendering device. Depending on whether the movement of the calibration plate or the motion rendering device is considered more accurate, the length unit along the axis is provided by the pattern on the calibration plate or by the x-unit travel of the motion rendering device. When this device is in the home position, the starting point of Home2D is at the center of rotation of the motion rendering device. There is only one Home2D coordinate system in the calibration system.

[0060] Image Coordinate System (Raw2D): Raw2D is the pixel space coordinate system of a single image capture device. There is one instance of this coordinate system for each image capture device 102. Image features are initially detected in this coordinate system. Due to lens and perspective distortions, Raw2D is not an orthogonal space with respect to the Home2D coordinate system. The Raw2D coordinate system of each image capture device is independent of the Raw2D coordinate systems of other image capture devices.

[0061] Image capture device coordinate system (Camera2D): Camera2D is the physical space standard orthogonal coordinate system for a single image capture device 102 (as viewed in Home2D). There is one instance of this coordinate system for each image capture device 102. It has the same length units as the Home2D coordinate system. The origin of Camera2D is located at a position in Home2D corresponding to the center of the image capture device's image window. Its x-axis is parallel to the Raw2D x-axis at the Camera2D origin. In some embodiments, the SPH calibration module 110 linearizes the Raw2D at the center of the window, and obtains the direction of the linearized x-axis, and obtains the approximate direction of the y-axis. The Raw2D coordinate system of each image capture device is related to the Camera2D system of this image capture device through a single view, separate imaging capture device, calibration model. In the mobile image capture device configuration, since the image capture device 102 moves with the motion rendering device 114 when the motion rendering device 114 moves to a particular pose, the Camera2D coordinates of each image capture device depend on the pose of the motion rendering device as commanded by the SPH calibration module 110. Therefore, in this case, the Camera2D coordinate system also moves relative to the Home2D coordinate system. In the static image capture device configuration, Camera2D is independent of the pose of the motion rendering device as commanded by the SPH calibration module 110.

[0062] Calibration plate coordinate system (Plate2D): Plate2D is the coordinate system of the calibration plate. There is an instance of this coordinate system for each calibration plate. The calibration plate includes a well-defined calibration pattern, which typically includes at least one reference point to define the coordinate axis direction of the system origin and the calibration target. At any time, all calibration features within a single calibration plate observed by all image capture devices are described in the same Plate2D coordinate system. However, this space can be moved around the Home2D coordinates by the motion rendering device 114. Plate2D is a standard orthogonal coordinate system (as observed in Home2D), although its length unit may have a non-identical scale factor with Home2D. Plate2D and Home2D may have different handedness. Each calibration target Plate2D coordinate system is independent of the Plate2D coordinate system of other calibration plates.

[0063] In a static image capture device configuration (e.g., a moving plate), Plate2D depends on the pose of the motion rendering device 114 as commanded by the calibration system. Since the calibration target 116 moves with the motion rendering device 114, the Plate2D transform from Stage2D is a fixed transform even when the motion rendering device 114 moves to different poses. However, the Home2D transform from Plate2D is not fixed when the motion rendering device 114 moves to different poses. Therefore, the Plate2D coordinate system moves relative to the Home2D coordinate system when the motion rendering device 114 moves to different poses.

[0064] In a mobile image capture device configuration, Plate2D is independent of the pose of the motion rendering device as commanded by the SPH calibration module 110 .

[0065] Calibration object coordinate system (Part2D): This is a more general coordinate system associated with the calibration object (which may not be a calibration plate). Part2D is used to represent the relative positions of features on the calibration object. In some embodiments, Part2D can be set to be the same as Home2D or Stage2D.

[0066] Stage Coordinate System (Stage2D): Stage2D is a standard orthogonal coordinate system (as viewed in Home2D) that is attached to, and thus moves with, the motion rendering device. The relationship between Home2D and Stage2D is provided by a two-dimensional rigid transformation, which is the identity transformation when the motion rendering device 114 is in its home position.

[0067] In some embodiments, Home2D from Stage2D represents the actual physical position of the motion rendering device in Home2D. Home2D from Stage2D can define the relationship between Home2D and Stage2D and can be represented by a 2D rigid transformation.

[0068] In some embodiments, the SPH calibration module 110 is configured to operate under the assumption that the handedness of Home2D, Camera2D, and Plate2D are independent. The handedness relates to the relative position of the x-axis and the y-axis. The Home2D handedness is determined by the direction of the axis of the motion stage. The Plate2D handedness accompanies the pattern on the calibration plate. The Camera2D handedness is determined by a plurality of mirrors in the optical path between the calibration plate and the image sensor.

[0069] In some embodiments, the uncorrected Home2D from Stage2D may indicate a commanded or reported pose of the kinematic rendering device 114. The commanded or reported pose of the kinematic rendering device 114 may not match the actual physical pose of the kinematic rendering device 114 in Home2D (e.g., Home2D from Stage2D) due to systematic errors in the kinematic rendering device 114. Thus, the uncorrected Home2D from Stage2D in a sense represents a best guess at the pose of the kinematic rendering device prior to calibration.

[0070] Figure 2A illustrates the relationship between coordinate systems in a still image capture device mode according to some embodiments; Figure 2B The relationship between coordinate systems in a mobile image capture device mode according to some embodiments is shown.

[0071] Figure 3 The operation of an SPH calibration module in an SPH calibration system according to some embodiments is shown.

[0072] The SPH calibration module 110 is configured to determine a relationship between a coordinate system (e.g., Raw2D) of one or more image capture devices 102 and a coordinate system (Home2D) associated with the motion rendering device 114. In some embodiments, the SPH calibration module 110 is configured to establish a correspondence between the locations of Raw2D features found in images taken by the image capture device 102 to the physical Home2D locations of these features when the motion rendering device 114 moves through a known set of poses.

[0073] In some embodiments, the motion rendering device 114 carries a plurality of calibration plates 116. In such embodiments, the SPH calibration system is considered to operate in a stationary image capture device configuration. In other embodiments, the motion rendering device 114 carries an image capture device 102. In such embodiments, the SPH calibration system is considered to operate in a mobile image capture device configuration.

[0074] In step 302, the SPH calibration module 110 is configured to instruct the motion rendering device 114 to move to a predetermined pose. Once the motion rendering device 114 receives the instruction, the motion rendering device 114 moves to the predetermined pose and reports the reported pose of the motion rendering device 114 back to the SPH calibration module 110. Due to the limited accuracy of the motion rendering device 114, the actual pose of the motion rendering device 114 may be different from the commanded pose instructed by the SPH calibration module 110.

[0075] In step 304 , the SPH calibration module 110 is configured to instruct the image capture devices 102 to capture images of the plurality of calibration plates. Once the image capture devices 102 receive the instructions, each image capture device 102 can capture an image of the corresponding calibration plate 116 and transmit the captured image to the SPH calibration module 110 .

[0076] In some embodiments, there may be a one-to-one correspondence between the image capture devices and the calibration plate. For example, the first image capture device may be configured to capture an image of the first calibration plate; the second image capture device may be configured to capture an image of the second calibration plate; and the third image capture device may be configured to capture an image of the third calibration plate. In other embodiments, there may be a many-to-one correspondence between the image capture devices and the calibration plate. For example, the first image capture device and the second image capture device may be configured to capture an image of the first calibration plate; and the third image capture device and the fourth image capture device may be configured to capture an image of the second calibration plate.

[0077] In step 306, SPH calibration module 110 is configured to determine whether there are other poses at which to capture an image of calibration plate 116. When there are other poses at which to capture an image of calibration plate 116, SPH calibration module 110 may proceed to step 302. When there are no other poses at which to capture an image of calibration plate 116, then SPH calibration module 110 may proceed to step 308.

[0078] In step 308, the SPH calibration module 110 is configured to determine the correspondence between the physical feature locations on the calibration plate and the Raw2D feature locations captured in the image. For example, the SPH calibration module 110 may utilize feature finding techniques to determine the locations of the features captured in the image. Subsequently, the SPH calibration module 110 may determine the correspondence between the determined Raw2D feature locations captured in the image and the known physical Plate2D feature locations.

[0079] In step 310 , the SPH calibration module 110 may utilize the correspondence and one or more poses in the motion rendering device 114 to determine a transformation between the Raw2D coordinate system of the image capture device and the Home2D coordinate system.

[0080] Although the SPH calibration module 110 uses the same type of information as a hand-eye calibration system based on a single calibration plate, the SPH calibration module 110 utilizes a different method to determine the transformation between the Raw2D coordinate system and the Home2D coordinate system. In a hand-eye calibration system based on a single calibration plate, all physical positions of features on the calibration plate are defined in the same Plate2D coordinate system. However, in the SPH calibration system, the physical positions of features on different calibration plates are defined in different Plate2D coordinate systems. Therefore, the SPH calibration system can transform these Plate2D coordinate systems to the Home2D coordinate system so that all features are represented in the same coordinate system. This difficulty does not occur in a single calibration plate based hand-eye calibration system.

[0081] In some embodiments, the SPH calibration module 110 is configured to determine one or more of the following parameters:

[0082] - A single image capture device single view calibration model characterized by a non-linear transformation between the Raw2D coordinates of each image capture device and the Camera2D coordinate system.

[0083] - Arrangement pose of the image capture device: transformation between the Camera2D coordinate system and the Home2D coordinate system (eg, Home2D from Camera2D or Camera2D from Home2D).

[0084] - Arrangement pose of the calibration target: transformation between Plate2D coordinate system and Stage2D coordinate system (eg, Stage2D from Plate2D or Plate2D from Stage2D).

[0085] - A transformation from (1) parameters used to control the motion rendering device, such as (x, y, theta) or an uncorrected Home2D from Stage2D to (2) the actual physical pose of the stage in the original coordinate system (Home2D) (e.g., a motion correction matrix).

[0086] In some embodiments, the arrangement pose of the image capture devices may indicate the pose of the image capture devices in the system. In a static image capture device configuration, the arrangement pose is given from the static Camera2D via Home2D, which provides the pose of the Camera2D of each static camera in Home2D. In a mobile image capture device configuration, the arrangement pose is given from the mobile Camera2D via Stage2D, which provides the pose of the Camera2D of each mobile image capture device in Stage2D. In both cases, the arrangement pose of the image capture devices includes a 2D rigid transformation with a possible handedness flip.

[0087] In some embodiments, the arrangement pose of the calibration target may specifically define the pose of the calibration target in the calibration system. In a static camera configuration, the arrangement pose is provided from the mobile Plate2D via Stage2D, which provides the pose of the mobile calibration target's Plate2D in Stage2D. In a moving camera configuration, the arrangement pose is provided from the stationary Plate2D via Home2D, which provides the pose of the calibration target Plate2D in Home2D. In both cases, the arrangement pose includes a 2D rigid transformation and possible handedness flipping.

[0088] In some embodiments, the SPH calibration module 110 may also be configured to compensate for systematic errors in the motion rendering device 114. Some less accurate motion rendering devices may exhibit systematic errors, such as skew between the x-axis of motion and the y-axis of motion, a difference in unit travel between the two axes, and / or a scale error in unit travel from a desired nominal value. The SPH calibration module 110 may be configured to compensate for such systematic errors by compensating for the following parameters of the motion rendering device using a motion correction matrix:

[0089] -x The magnitude of the unit's travel

[0090] -y direction the unit is traveling

[0091] -y The magnitude of the unit travel

[0092] In this embodiment, the SPH calibration module 110 may assume that the direction in which the x-unit is heading is accurate by definition.

[0093] In some embodiments, the SPH calibration module 110 may be configured to determine these estimates based on the correspondences of the features determined in step 308 and the reported pose of the motion rendering device 114 .

[0094] In some embodiments, the SPH calibration module 110 assumes that in a static image capture device configuration, the calibration target is rigidly attached to the motion rendering device 114. The attachment points on the target and the orientation of the target are arbitrary. Similarly, in some embodiments, the SPH calibration module 110 assumes that in a mobile image capture device configuration, the image capture device is rigidly attached to the motion stage, and the attachment points of this image capture device and their orientation are arbitrary. In both configurations, once attached, the attachment points and orientation should remain unchanged throughout the calibration procedure. Since the calibration target (or image capture device) is rigidly attached, the layout pose remains unchanged when the motion rendering device moves the calibration target or image capture device. The layout pose includes a 2-D rigid transformation and possible handedness flipping. In most applications, the layout pose cannot be precisely controlled and is unknown before operating the SPH calibration module 110.

[0095] In some implementations, the SPH calibration module 110 may use a two-step approach to determine these estimates.

[0096] In a first step, the SPH calibration module 110 may construct a linear system relating the unknown parameters, the correspondences of the features determined in step 308, and the reported pose of the motion rendering device 114. Subsequently, the SPH calibration module 110 may determine estimates of the unknown parameters by iteratively solving the linear system and the nonlinear single camera calibration step.

[0097] In a second step, the SPH calibration module 110 may construct a nonlinear system that minimizes the feature reprojection error and solve the nonlinear system using a nonlinear solver. In this second step, the solution to the linear system may be used as an initial value for the nonlinear solver.

[0098] Step 1

[0099] Figure 4 A linear estimation stage of the SPH calibration module 110 is shown according to some embodiments.

[0100] In step 402 , the SPH calibration module 110 may be configured to estimate the correspondence between Raw2D from Plate2D, Raw2D from Camera2D, and Camera2D from Plate2D.

[0101] To this end, the SPH calibration module 110 may perform a single view, single image capture device calibration for each pose of the motion rendering device. The SPH calibration module 110 may use a single view, single image capture device calibration technique known in the art.

[0102] In some embodiments, the SPH calibration module 110, which performs single-view, single-image capture device calibration, computes Camera2DfromPlate2D and Raw2DfromCamera2D based on a single image of the calibration plate. In this way, the SPH calibration module 110 is able to account for any handedness flip between the calibration target and the image capture device.

[0103] Single-view single-camera calibration assumes that both the image capture device and the calibration plate are stationary (or stationary relative to each other). Under this assumption, single-view single-camera calibration computes the transformation between the Camera2D coordinate system and the Plate2D coordinate system based on feature correspondences (features detected in image space Raw2D, and corresponding feature positions on Plate2D), as well as the camera intrinsic parameters (Raw2D from Camera2D).

[0104] Since the feature correspondence represents the transformation between Raw2D and Plate2D, the SPH calibration module 110 can decompose Raw2DfromPlate2D into Raw2DfromCamera2D and Camera2DfromPlate2D. To determine Raw2DfromCamera2D and Camera2DfromPlate2D, the SPH calibration module 110 can use (1) the correspondence between Raw2D feature points and Plate2D feature locations and (2) the type of lens deformation pattern associated with the image capture device.

[0105] For the decomposition of Raw2DFromPlate2D into Raw2DFromCamera2D and Camera2DFromPlate2D, the SPH calibration module 110 assumes that Raw2DFromCamera2D and Camera2DFromPlate2D are in a particular form. For example, when there is a mirror in the optical path of the camera, the SPH calibration module 110 may assume that the transformation of Camera2DFromPlate2D is a rigid transformation with potential handedness flipping. Similarly, the SPH calibration module 110 may assume that Raw2DFromCamera2D indicates a stereo perspective transformation and an affine transformation. The SPH calibration module 110 may also assume that Raw2DFromCamera2D includes lens deformation. In some cases, the lens deformation may be modeled as a single parameter radial model as follows: Rx=x*(1+k*r 2 ); Ry=y*(1+k*r 2), where r 2 =x 2 +y 2 , x and y are the positions without lens distortion, and k is the coefficient of radial distortion. Since k is unknown, this coefficient is estimated by the calibration procedure. Raw2D also includes perspective and affine transformations from Camera2D.

[0106] The SPH calibration module 110 performs single-view, single image capture device calibration by forming linear and non-linear equations mapping Plate2D features to Raw2D features to resolve the unknowns in Raw2D from Camera2D and Camera2D from Plate2D.

[0107] In some embodiments, the minimum number of correspondences required to perform a single-view single image capture device calibration is nine, with non-degenerate positions. When the number of correspondences is less than nine but greater than two, then the SPH calibration module 110 may use a rigid plus scale and a handedness linear fit to determine the Raw2D from Plate2D including the handedness. In some embodiments, the SPH calibration module 110 may be configured to find the rigid plus scale and the handedness linear fit using a least squares method. The least squares method may find the Raw2D from Plate2D that minimizes the least squares error. In other embodiments, the SPH calibration module 110 may be configured to find the rigid plus scale and the handedness linear fit using an iterative closest point method.

[0108] When there are less than three nonlinear features in all views, then the SPH calibration module 110 cannot determine the arrangement of the calibration plate. Therefore, in this case, the SPH calibration module 110 may raise an exception and terminate.

[0109] In step 404 , the SPH calibration module 110 may transform the Raw2D feature coordinates to the Camera2D feature coordinates using the inversion of Raw2D from Camera2D determined in step 402 .

[0110] In step 406 , the SPH calibration module 110 may construct a linear system using the features represented in the Camera2D coordinates and solve the motion model and the arrangement pose of the calibration target and the image capture device.

[0111] This linear system can be constructed based on the features for each correspondence. There are at least two ways to map the feature positions to the Home2D coordinate system.

[0112] First, the Home2D position of the feature can be calculated by applying the plate layout transformation to the Plate2D coordinates of the feature, which transforms the Plate2D coordinates to Stage2D coordinates. The motion of the motion rendering device 114 is then applied and the coordinates transformed to Home2D: Where R represents the motion rendering device rotation matrix, which represents the rotation of the motion rendering device, and P represents the Plate2D layout transformation matrix that transforms the Plate2D coordinate system to the Stage2D coordinate system. Represents the coordinates of the feature point in the Plate2D coordinate system. Second, the Home2D position of the feature can be calculated by applying the Home2D-from-Camera2D transformation to draw the feature to the Home2D Camera2D coordinate description: Where C represents the camera placement transformation matrix that transforms the Camera2D coordinate system to the Home2D coordinate system, and Represents the coordinates of a feature point in the Camera2D coordinate system.

[0113] Since the physical position on Plate2D and the mid-image point of Camera2D should correspond to the same position when mapped to the Home2D coordinate system, the Home2D position calculated by the two methods should be the same. However, the systematic error of the motion rendering device 114 may cause a difference in the Home2D position calculated by the two methods. Therefore, the systematic error of the motion rendering device 114 can be compensated by using the motion model. In some embodiments, the motion model can be expressed as the motion correction term where M represents the motion correction matrix and Represents the uncalibrated pose of the motion rendering device.

[0114] Therefore, the relationship between the Home2D positions calculated by the two methods can be expressed as the following system:

[0115]

[0116] because and R are known values, so based on the unknowns M, P, and C, the system determined above is a linear system.

[0117] In some embodiments, the motion correction matrix M may be expressed as follows:

[0118]

[0119] Wherein i represents the x direction of the motion correction matrix, j represents the y direction of the motion correction matrix, and s represents the skew of the motion correction matrix.

[0120] In some embodiments, the motion rendering device rotation matrix R can be expressed as follows:

[0121]

[0122] Here, θ represents the in-plane rotation of the motion presentation device 114 .

[0123] In some embodiments, the Plate2D arrangement transformation matrix P may be expressed as follows:

[0124]

[0125] where a, b, c and d are the plate arrangement rotation coefficients, and e and f are the plate arrangement translation coefficients in the x and y axes, respectively. As mentioned above, the Plate2D arrangement transformation matrix P is known to be a combination of a 2D rigid transformation and a chiral transformation. Therefore, the Plate2D arrangement transformation matrix P can be simplified as follows

[0126]

[0127] In some embodiments, the camera arrangement transformation matrix C may be expressed as follows:

[0128]

[0129] Where k, l, m and n are the camera arrangement rotation coefficients, and r and t are the camera arrangement translation coefficients in the x and y axes, respectively. As described above, the camera arrangement transformation matrix C is known to be a combination of a 2D transformation and a handedness transformation. Therefore, the camera arrangement transformation matrix C can be simplified as follows:

[0130]

[0131] In some embodiments, the SPH calibration module 110 can be based on the physical feature coordinates in the Plate2D coordinate system. The imaging feature position in the Camera2D coordinate system The corresponding relationship between them solves the linear system with respect to the motion correction matrix M, the Plate2D layout transformation matrix P, and the camera layout transformation matrix C

[0132] In some embodiments, the SPH calibration module 110 can be configured to determine whether the Plate2D coordinate system is flipped in its handedness and whether the Camera2D coordinate system is flipped in its handedness. The Plate2D arrangement transformation matrix P and the camera arrangement transformation matrix C provided above assume that both the calibration target and the image capture device do not have handedness flipping. However, when the image capture device or the calibration target arrangement has handedness flipping, then the sign values ​​in the Plate2D arrangement transformation matrix P and / or the sign values ​​in the camera arrangement transformation matrix C should be appropriately modified to account for the flipped g and h axes, and the flipped p and q axes, respectively.

[0133] For example, when both the calibration target and the image capture device do not have handedness flipping, the Plate2D arrangement transformation matrix P and the camera arrangement transformation matrix C can be set as follows:

[0134]

[0135] For another example, when the calibration target has handedness flipping, but the image capture device does not have handedness flipping, the Plate2D arrangement transformation matrix P and the camera arrangement transformation matrix C can be set as follows:

[0136]

[0137] For another example, when the calibration target does not have handedness flipping, but the image capture device has handedness flipping, the Plate2D arrangement transformation matrix P and the camera arrangement transformation matrix C can be set as follows:

[0138]

[0139] For another example, when both the calibration target and the image capture device have handedness flip, the Plate2D arrangement transformation matrix P and the camera arrangement transformation matrix C can be set as follows:

[0140]

[0141] If the SPH calibration module 110 does not have any information whether the Plate2D coordinate system is flipped in its handedness and whether the Camera2D coordinate system is flipped in its handedness, then the SPH calibration module 110 needs to test all four cases provided above to determine the solution to the constructed linear system.

[0142] However, in some cases, the SPH calibration module 110 may limit the number of test cases because the SPH calibration module 110 may determine whether there is a handedness flip between Plate2D and Raw2D by examining the Raw2D transformed from Plate2D.

[0143] In some embodiments, the SPH calibration module 110 may be configured to determine whether there is a handedness flip between Plate2D and Raw2D by checking the transformation from Plate2D to Raw2D. For example, the SPH calibration module 110 may be configured to determine whether there is a handedness flip by linearizing Raw2D from Plate2D at the center of the image and checking the determinant. When the determinant is positive, then the SPH calibration module 110 has determined that there is no handedness flip.

[0144] Therefore, when the transformation of Raw2D from Plate2D indicates that there is a handedness flip between Plate2D and Raw2D, then the SPH calibration module 110 knows that the combination of P and C can only be one of the following two options:

[0145] or

[0146]

[0147] Similarly, when the transformation of Raw2D from Plate2D indicates that there is no handedness flip between Plate2D and Raw2D, then the SPH calibration module 110 knows that the combination of P and C can only be one of the following two options:

[0148] or

[0149]

[0150] Once the SPH calibration module 110 determines that the handedness of Plate2D is flipped, the SPH calibration module 110 can check the Raw2D transformed from Plate2D to determine if Raw2D is also flipped.

[0151] In some embodiments, the SPH calibration module 110 can be configured to: Estimate M, P, and C, and (2) check based on the estimated M, P, and C for the two choices and The difference between determines which of the two choices is correct. The SPH calibration module 110 can also determine the positive definiteness of the motion correction matrix M. The SPH calibration module 110 is configured to determine that the choice with (1) a smaller difference and (2) a more positive definite motion correction matrix M is the correct choice.

[0152] In step 408, once the SPH calibration module 110 estimates M, P, and C, the SPH calibration module 110 may combine the correspondences at different poses for an image capture device 102. For example, the SPH calibration module 110 may transform features in the Plate2D coordinate system to the Home2D coordinate system based on the estimated M and P. Thus, the correspondences at different poses for an image capture device 102 may be combined in the Home2D coordinate system.

[0153] In step 410, the SPH calibration module 110 may operate a single view single image capture device calibration on the combined correspondences. In this way, the SPH calibration module 110 may determine Camera2DfromHome2D and Raw2DfromCamera2D based on the correspondences from all poses. Since the Raw2DfromCamera2D transform used at the beginning of the first iteration is not calculated from all poses, its accuracy can be improved through the first iteration. In subsequent iterations, a more accurate Raw2DfromCamera2D transform will further improve the accuracy of the results. When a preset maximum number of iterations is reached, the iteration will terminate.

[0154] Step 2

[0155] The first step of the SPH calibration mechanism may generally provide a rough solution that is close to the ground truth, but may still require further refinement to improve accuracy. To refine the initial results from the first step, the SPH calibration module 110 may be configured to perform a nonlinear optimization to minimize the "reprojection error", where the results returned from the first step are used as initial values. The second step may be expressed as follows:

[0156]

[0157] Where (x, y, θ) is the incorrect position of the motion rendering device 114, and ((g, h), (p, q)) is the Plate2D coordinate and Raw2D coordinate of the feature. result is the hand-eye calibration result that needs to be solved, which includes the intrinsic and extrinsic parameters of the image capture device, the lens deformation associated with the image capture device, the motion correction transformation, and the calibration target arrangement. C is an intermediate calibration result in nonlinear optimization. Using C, physical points can be plotted to Raw2D. T(C,x,y,θ,g,h,…) can be expressed as:

[0158] T(C,x,y,θ,g,h,…)=Raw2DfromCamera2D*Camera2DfromHome2D*

[0159] Home2D from Stage2D* Stage2D from Plate2D*(g,h).

[0160] In some embodiments, the energy function Can calculate two input parameters Euclidean distance between Ω

[0161] In some embodiments, the SPH calibration module 110 may be configured to solve the above nonlinear system using a nonlinear technique such as a Levenberg-Marquardt (LM) optimization technique. For example, the SPH calibration module 110 may set the initial solution vector to a linear solution through the first step of the SPH calibration mechanism. The objective function is designed to calculate the reprojection error between the Raw2D and the drawn Raw2D using the current hand-eye calibration result. This reprojection error may be calculated by calculating the distance between the Raw2D and the drawn Raw2D.

[0162] Subsequently, the SPH calibration module 110 calculates the Jacobian matrix using a digital method. After calculating the Jacobian matrix, the SPH calibration module 110 updates the solution of the nonlinear system, calculates the residual, and continues to iterate until a default stopping criterion is met. The default stopping criterion may include, for example, that the number of such criteria is a predetermined number, or that the residual is less than a predetermined threshold. Once the SPH calibration module 110 completes the iteration, the received vector may include intrinsic and extrinsic parameters of the image capture device, lens deformation associated with the image capture device, motion correction transformation, and calibration target arrangement.

[0163] TFH Calibration System

[0164] The TFH calibration system can be used in applications where the runtime object (the object being analyzed by the machine vision system) has sufficient features that can be used as a calibration pattern. For example, when the runtime object has a large number of corners, edges, or any identifiable features, using the TFH calibration system, these features can be used to perform hand-eye calibration.

[0165] One of the challenges of using a runtime object as a calibration target is that the physical locations of the calibration features on the runtime object are often unknown. Therefore, a TFH calibration system should be constructed to (1) detect and track calibration features on the runtime object across motion rendering device poses, (2) estimate the unknown physical locations of the detected calibration features, and (3) estimate the hand-eye calibration parameters.

[0166] In some embodiments, the TFH calibration system can operate in a static image capture device configuration. In this configuration, the image capture device can be stationary and the motion rendering device can carry the runtime object through a variety of predetermined postures. In other embodiments, the TFH calibration system can operate in a moving image capture device configuration. In this configuration, the runtime object can remain stationary and the motion rendering device can carry the image capture device through a variety of predetermined postures.

[0167] In some embodiments, the TFH calibration system can associate (1) images of a calibration target captured by multiple image capture devices and (2) the pose of a motion rendering device under which the images were captured, and store the images with the associated poses in an image database.

[0168] In some embodiments, the TFH calibration system can be configured to perform TFH calibration on images stored in an image database. The TFH calibration system can operate one or more feature trackers to detect and track features that can be used as calibration features. The feature trackers can include corner detectors, edge detectors, scale-invariant feature transform (SIFT) detectors, and / or any other detectors capable of detecting reliable features suitable for calibration purposes. These calibration features are described in a Raw2D coordinate system and are therefore also referred to as tracked Raw2D features. The TFH calibration system then uses the tracked Raw2D features to perform hand-eye calibration.

[0169] Figure 5 A TFH calibration system according to some embodiments is shown. Figure 5 Including Figure 1 However, in the tracked feature hand-eye calibration system, the computing device 104 includes the TFH calibration module 502 and the motion rendering device 114 is configured to carry a runtime object 504 that is inspected by the machine vision system 100 .

[0170] In some embodiments, the TFH calibration module 502 can be configured to match the TFH calibration mechanism. The TFH calibration module 502 can be configured to send first data configured to cause the motion rendering device to move to the required first pose to the motion rendering device via one or more interfaces; receive the reported first pose from the motion rendering device via one or more interfaces; receive a first image of a first subset of multiple features of the reported first pose from the first image sensor via one or more interfaces; and receive a second image of a second subset of multiple features of the reported first pose from the second image sensor via one or more interfaces. The TFH calibration module 502 can also be configured to determine a first transformation between the first coordinate system and the second coordinate system based at least in part on the first subset of multiple features and the reported first pose, and determine a second transformation between the first coordinate system and the third coordinate system based at least in part on the second subset of multiple features and the reported first pose.

[0171] In some embodiments, the TFH calibration module 502 can determine the first transformation and the second transformation in a two-step process. In the first step, the TFH calibration module 502 can construct a linear system related to the unknown parameters, the location of the features in the captured image, and the reported pose of the motion rendering device 114. Subsequently, the TFH calibration module 502 can determine the estimated value of the unknown parameter by iteratively solving the linear system and the nonlinear single camera calibration step. In the second step, the TFH calibration module 502 can construct a nonlinear system that minimizes the feature reprojection error and solve the nonlinear system using a nonlinear solver. In this second step, this solution of the linear system can be used as a nonlinear solver.

[0172] In some embodiments, the TFH calibration module 502 may be implemented in digital electronic circuitry, or in computer hardware, firmware, software, or a combination thereof. This implementation may be a computer program product as, for example, a computer program explicitly embodied in a machine-readable storage device, so as to be executed by a data processing device, such as a programmable processor, a computer, and / or a plurality of computers, or to control the operation of the data processing device. The computer program may be written in any form of computer language or programming language, including source code, assembly code, translated code, and / or machine code, and may be deployed in any form, including as a stand-alone program or as a subroutine, element, or other unit suitable for use in a computing environment. The computer program may be deployed to be executed on one computer or on multiple computers at one or more locations.

[0173] In some embodiments, the SPH calibration module 110 and the TFH calibration module 502 may reside in the same computing device 104. In some embodiments, the SPH calibration module 110 and the TFH calibration module 502 may be implemented on the same integrated circuit such as an ASIC, PLA, DSP, or FPGA, thereby forming a system on a chip. A subroutine may represent a portion of a computer program and / or a processor / specific circuit that performs one or more functions.

[0174] In other implementations, the SPH calibration module 110 and the TFH calibration module 502 may exist in different computing devices.

[0175] Figure 6 The operation of a TFH calibration module in a TFH calibration system according to some embodiments is shown.

[0176] The TFH calibration module 502 may be configured to determine a relationship between a coordinate system of one or more image capture devices 102 (e.g., Raw2D) and a coordinate system of a motion rendering device (e.g., Stage2D). To this end, the TFH calibration module 502 is configured to establish a correspondence between Raw2D features found in images acquired by the image capture device 102 to the physical Stage2D coordinates of these features as the motion rendering device 114 moves through a known set of poses.

[0177] In some embodiments, the motion rendering device 114 hosts the runtime object 504. In this embodiment, the TFH calibration system is considered to operate in a stationary image capture device configuration. In some embodiments, the motion rendering device 114 hosts the image capture device 102. In this embodiment, the TFH calibration system is considered to operate in a moving image capture device configuration.

[0178] In step 602, the TFH calibration module 502 is configured to instruct the motion rendering device 114 to move to a predetermined pose. Once the motion rendering device 114 receives the instruction, the motion rendering device 114 moves to the predetermined pose and reports the reported pose of the motion rendering device 114 back to the TFH calibration module 502. Due to the limited accuracy of the motion rendering device 114, the actual pose of the motion rendering device 114 may be different from the reported and commanded pose indicated by the TFH calibration module 502.

[0179] In step 604, the TFH calibration module 502 is configured to instruct the image capture devices 102 to capture images of the runtime objects 504. Once the image capture devices 102 receive the instructions, each image capture device 102 may capture an image of the runtime objects 504 and transmit the captured image to the TFH calibration module 502.

[0180] In step 606, the TFH calibration module 502 is configured to determine whether there are other poses at which an image of the runtime object 504 will be captured. When there are other poses at which an image of the runtime object 504 will be captured, the TFH calibration module 502 can proceed to step 602. When there are no other poses at which an image of the runtime object 504 will be captured, then the TFH calibration module 502 can proceed to step 608.

[0181] In step 608 , the TFH calibration module 502 is configured to determine a transformation relating the Raw2D coordinate system of the image capture device to the Home2D coordinate system based on the calibration features extracted from the image of the runtime object 504 .

[0182] In some embodiments, the TFH calibration module 502 may also be configured to compensate for systematic errors in the motion rendering device 114. Some lower accuracy motion rendering devices may exhibit systematic errors, such as skew between the x-axis of motion and the y-axis of motion, a difference in unit travel between the two axes, and / or a proportional error in unit travel from the desired nominal value. The TFH calibration module 502 may be configured to compensate for this systematic error by estimating the following parameters of the motion rendering device:

[0183] y The direction the unit is traveling

[0184] The magnitude of the x unit travel

[0185] In this embodiment, the TFH calibration module 502 may assume that the direction of x travel and the x cell size are accurate by definition.

[0186] In some embodiments, the TFH calibration module 502 may be configured to determine these estimates based on Raw2D feature locations extracted from images of the runtime object 504 and the reported pose of the motion rendering device 114 .

[0187] In some embodiments, the TFH calibration module 502 assumes that, in a static image capture device configuration, the runtime object 504 is rigidly attached to the motion rendering device 114. The attachment points of the runtime object 504 and the orientation of the runtime object 504 are arbitrary. Similarly, in some embodiments, the TFH calibration module 502 assumes that, in a mobile image capture device configuration, the image capture device is rigidly attached to the motion stage, and the attachment points of this image capture device and their orientation are arbitrary. In both configurations, once attached, the attachment points and orientation should remain unchanged throughout the calibration procedure.

[0188] Since the runtime object 504 or the image capture device 102 is rigidly attached, the arrangement pose remains unchanged when the motion rendering device moves the calibration target or the image capture device. The arrangement pose includes a 2D rigid transformation with possible handedness flipping. In most applications, the arrangement pose cannot be precisely controlled and is unknown before operating the TFH calibration module 502.

[0189] In some implementations, the SPH calibration module 502 can use a two-step approach to determine these estimates.

[0190] In a first step, the TFH calibration module 502 may construct a linear system relating the unknown parameters, the Raw2D positions of the calibration features on the runtime object 504, and the reported pose of the motion rendering device 114. Subsequently, the TFH calibration module 502 may determine estimates of the unknown parameters by iteratively solving the linear system and the nonlinear single camera calibration step.

[0191] In the second step, the TFH calibration module 502 can construct a nonlinear system that minimizes the feature reprojection error and solve the nonlinear system using a nonlinear solver. In this second step, the solution to the linear system can be used as an initial value for the nonlinear solver. The second step of the TFH calibration module 502 is the same as the second step of the SPH calibration module 110.

[0192] Step 1

[0193] As explained with respect to the first step of the SPH calibration module 110, since the physical position of the feature on the Stage2D coordinate system and the image point in the Camera2D coordinate system should correspond to the same position when mapped to the Home2D coordinate system, the position of Home2D calculated by the two methods should be the same. This relationship can be expressed as the following system:

[0194]

[0195] Where M represents the motion correction matrix, represents the uncalibrated pose of the motion rendering device, R represents the motion rendering device rotation matrix, which represents the rotation of the motion rendering device in the Home2D coordinate system, P represents the Part2D layout transformation matrix that transforms the Part2D coordinate system to the Home2D coordinate system, represents the coordinates of the feature point in the Part2D coordinate system, C represents the camera layout transformation matrix that transforms the Camera2D coordinate system to the Home2D coordinate system, and Represents the coordinates of the feature point in the Camera2D coordinate system.

[0196] In the case of SPH calibration, due to and R are known values, so the above system is a linear system based on the unknown values ​​M, P and C. However, in the case of TFH calibration, the TFH calibration module 502 does not know the Part2D coordinates of the extracted features: Therefore, according to the unknown variables: M, P, C, and The above system is no longer linear.

[0197] To ensure that the system determined above is a linear system in terms of unknown variables, the TFH calibration module 502 may assume, without loss of generality, that the Part2D arrangement transformation matrix P is the identity matrix. In other words, the calibration features are measured in Stage2D without taking into account the in-plane rotation of the runtime object 504.

[0198] Based on this assumption, the system identified above can be written as follows:

[0199]

[0200] In some embodiments, there are only 2 possible handedness flips. Since Stage2D is identical from Part2D, there is no handedness flip between Part2D and Stage2D. In the case where the arrangement of the image capture device does not have handedness flip, then In the case where the arrangement of the image capture device has handedness flip, then By solving the linear system determined above, the TFH calibration module 502 can estimate the image capture device arrangement, motion model, and estimated feature positions in Stage2D.

[0201] Figure 7 The linear estimation stage of the TFH calibration module 502 according to some embodiments is summarized.

[0202] In step 702, the TFH calibration module 502 may determine the positions of calibration features on the calibration target in the Raw2D coordinate system. To this end, the TFH calibration module 502 may operate a feature tracker on the images of the calibration target acquired by the plurality of image capture devices 102 at one or more poses of the motion rendering device 114. The feature tracker may include a corner detector, an edge detector, a scale-invariant feature transform (SIFT) detector, and / or any other detector capable of detecting reliable features suitable for calibration purposes.

[0203] In step 703, the TFH calibration module 502 may set Raw2D to an identity transform from Camera2D.

[0204] In step 704, the TFH calibration module 502 may transform the Raw2D feature positions to the Camera 2D feature positions using the Raw 2D from Camera 2D transform

[0205] At step 706, the TFH calibration module 502 may establish a linear system And estimate M,C and (g,h) through a linear system.

[0206] In step 708, the TFH calibration module 502 may transform all Stage2D feature positions to Home2D positions as follows: Subsequently, the TFH calibration module 502 may construct correspondences between Home2D feature positions and Raw2D feature positions. Based on these correspondences, the TFH calibration module 502 may determine the Home2D transformation from Raw2D. In addition, based on the Home 2D feature positions and the Camera2D feature positions Based on the relationship between , the TFH calibration module 502 can estimate the transformation of Camera 2D from Home 2D.

[0207] In addition, in some embodiments, the TFH calibration module 502 can re-estimate the Raw2D from Camera 2D transformation by iterating steps 704-708 until a stopping criterion is met. The stopping criterion can include reaching a maximum number of iterations. This re-estimation process is similar to the iterative process described with respect to the SPH calibration module 110.

[0208] In some embodiments, the Camera2D coordinate system is an intermediate coordinate system, and its unit size (e.g., scale factor) is defined to be the same as the unit size of the Home2D coordinate system. This means that when the TFH calibration module 502 decomposes Raw2DfromHome2D into Raw2DfromCamera2D and Camera2DfromHome2D, the TFH calibration module 502 allocates the scale factor in Raw2DfromCamera2D and allocates all translation factors in Home2DfromCamera2D. Or, alternatively, all scales and translations can be factored into Home2DfromCamera2D to make Raw2DfromCamera2D identical. However, this may not be the most accurate decomposition. But before the physical position of the feature on Stage2D can be solved, there will be no information about Raw2DfromHome2D. Therefore, the TFH calibration module 502 may iterate steps 702 - 708 to iteratively solve for one of Raw2D from Camera2D or Camera2D from Home2D when determining the other.

[0209] For example, in a first iteration, the TFH calibration module 502 may initialize Raw2DFromCamera2D to identity and solve for Camera2DFromHome2D and other parameters / transformations. Since the Raw2DFromCamera2D used to solve for Camera2DFromHome2D is identity, the actual scale and translation of Raw2DFromCamera2D is factored into Camera2DFromHome2D. However, by definition, the scale of Camera2DFromHome2D should be unity, so this scale factor may be calculated and used to transform the calculated motion pattern (e.g., motion correction terms represented by motion correction matrices), the arrangement pose of the image capture device, and the physical location of the features in step 706 to the correct scale. Then in step 708, Raw2DFromCamera2D may be calculated to replace the initial identity transformation. In a second iteration, the TFH calibration module 502 can use the more accurate Raw2DfromCamera2D to solve for the motion correction pattern, the arrangement pose of the image capture device, and the physical location of the feature, and then update the Raw2DfromCamera2D with the more accurate physical location of the feature. This iterative process improves the accuracy of this decomposition.

[0210] Therefore, after calculating the new Raw2D from Camera2D, the TFH calibration module 502 may iterate steps 704-708 until the TFH calibration module 502 reaches a maximum number of iterations. Iterations of steps 704-708 may reduce approximation errors resulting from approximating a nonlinear relationship (eg, lens deformation) as a linear system.

[0211] During the first iteration, the scale ambiguity between Raw2D and Home2D can be resolved. In most cases, after the second iteration, the TFH calibration module 502 should be able to provide relatively accurate estimates. In some embodiments, the maximum number of iterations can be set to 4, because accuracy is not a major issue in the first step of the disclosed TFH calibration mechanism.

[0212] Step 2

[0213] The first step of the TFH calibration mechanism typically provides a rough solution that is close to the ground truth, but further refinement may still be required to improve accuracy. In order to refine the initial results from the first step, the TFH calibration module 502 can be configured to perform a nonlinear optimization to minimize the "reprojection error", where the results returned from the first step are used as initial values. The nonlinear optimization performed by the TFH calibration module 502 can be substantially the same as the nonlinear optimization performed by the SPH calibration module 110. One difference is that in the case of TFH calibration, the physical feature positions in Stage2D are unknown. Thus, the TFH calibration module 502 can be configured to iteratively estimate the physical feature positions in Stage2D in the nonlinear estimation using the estimated physical feature positions in Stage2D from the first step as initial values. The physical feature positions in Stage2D are refined in the LM solver.

[0214] Recalibration

[0215] Due to the different characteristics of the hand-eye calibration methods, different hand-eye calibration methods can be used in different scenarios. For example, image capture devices are typically pre-calibrated in the factory using a large calibration plate that provides high calibration accuracy. However, when the motion rendering device and the image capture device are deployed for an application, the motion rendering device and the image capture device still need to be calibrated regularly to compensate for changes in the physical setup. For example, some pre-calibrated parameters, such as external parameters of the image capture device, may change over time due to vibrations of the motion rendering device, and may need to be recalibrated regularly to compensate for this change. For this regular calibration application, the SPH calibration mechanism and the TFH calibration mechanism may be useful.

[0216] As explained above, the SPH calibration mechanism and the TFH calibration mechanism can provide limited accuracy, especially when there are only a few features or motion poses in the presence of lens deformation. However, when a highly accurate calibration resulting from a previously performed calibration is available, the concept of recalibration can be used to improve the accuracy of the SPH calibration mechanism and the TFH calibration mechanism by using a highly accurate previously performed camera calibration.

[0217] For example, under the premise that the image capture device or its lens is not reassembled, some calibration parameters such as the intrinsic parameters of the image capture device may not change even in the presence of vibrations of the motion rendering device. Thus, the recalibration scheme can maintain the intrinsic parameters of the image capture device and only calibrate the external parameters of the image capture device. The recalibration scheme can be substantially similar to the full calibration scheme. However, the recalibration scheme can use other input vectors including the calibration parameters of the image capture device. In the absence of prior information about the intrinsic parameters of the image capture device, especially the TFH calibration mechanism, the recalibration scheme is not only faster but also more accurate than the full hand-eye calibration.

[0218] It should be understood that the disclosed subject matter is not limited to its application to the details of the construction and the arrangement of parts set forth in the following description or described in the accompanying drawings. The disclosed subject matter can have other embodiments and be practiced and implemented in a variety of ways. In addition, it should be understood that the words and terms used herein are for descriptive purposes and should not be considered as limiting.

[0219] Likewise, it will be appreciated by those skilled in the art that the concepts underlying this disclosure can be easily used as the basis for designing other structures, methods, and devices for performing several purposes of the disclosed subject matter. Therefore, it is important that the claims are deemed to include this equivalent construction where they do not deviate from the spirit and scope of the disclosed subject matter. For example, some of the disclosed embodiments involve one or more variables. This relationship can be expressed using a mathematical equation. However, a person of ordinary skill in the art can also use different mathematical equations, by transforming the disclosed mathematical equations, to express the same relationship between one or more variables. It is important that this claim is deemed to include this equivalent relationship formula between one or more variables.

[0220] Although the disclosed subject matter has been described and illustrated in the above exemplary embodiments, it should be understood that the present disclosure is made by way of example only and that various changes may be made in the implementation details of the disclosed subject matter without departing from the spirit and scope of the disclosed subject matter.

Claims

1. A system for machine vision, comprising: A processor configured to execute a computer program stored in a memory, the computer program configured to: receiving a first image of a target object in a first pose of a motion rendering device from a first image sensor, wherein the motion rendering device is associated with a first coordinate system and is configured to directly or indirectly carry the target object, wherein the first image sensor is associated with a second coordinate system; receiving a second image of the target object in a first pose of the motion rendering device from a second image sensor, wherein the second image sensor is associated with a third coordinate system; determining a first transformation that allows mapping between a first coordinate system associated with the motion rendering device and a second coordinate system associated with the first image sensor, wherein the first transformation is determined based on a plurality of first correspondences between known physical positions of a plurality of first features of a first calibration plate in a first pose of the motion rendering device and first positions of the plurality of first features detected in a first image of the first calibration plate in the first pose of the motion rendering device obtained by the first image sensor; determining a second transformation that allows mapping between a first coordinate system associated with the motion rendering device and a third coordinate system associated with the second image sensor, wherein the second transformation is determined based on a plurality of second correspondences between known physical positions of a plurality of second features of a second calibration plate in a first pose of the motion rendering device and second positions of the plurality of second features detected in a second image of the second calibration plate in the first pose of the motion rendering device obtained by the second image sensor; as well as A correspondence between features of the target object found in the first and second images acquired by the first and second image sensors and the first coordinate system is established based on the first and second transformations.

2. The system according to claim 1, wherein: The motion rendering device is configured to provide at least one of a translational movement and an in-plane rotational movement.

3. The system according to claim 2, wherein: The computer program is operable to cause the processor to determine a motion correction transform that compensates for systematic motion errors associated with the motion rendering device.

4. The system according to claim 1, wherein: The computer program is operable to cause the processor to recalibrate the system after a first period of time, including re-determining: the plurality of first corresponding relationships; the plurality of second corresponding relationships; the first transformation; as well as The second transformation.

5. The system according to claim 4, wherein: Recalibrating the system includes adjusting one or more pre-calibrated parameters.

6. A system for machine vision, comprising: A processor configured to execute a computer program stored in a memory, the computer program configured to: receiving, from a first image sensor, a first image of the target object in a first pose of a motion rendering device, wherein the motion rendering device is associated with a first coordinate system and is configured to directly or indirectly carry a first image sensor and a second image sensor, wherein the first image sensor is associated with a second coordinate system; receiving a second image of the target object from the first pose of the motion rendering device from a second image sensor, wherein the second image sensor is associated with a third coordinate system; determining a first transformation that allows mapping between a first coordinate system associated with the motion rendering device and a second coordinate system associated with the first image sensor, wherein the first transformation is determined based on a plurality of first correspondences between known physical positions of a plurality of first features of a first calibration plate starting from a first pose of the motion rendering device and first positions of the plurality of first features detected in a first image of the first calibration plate obtained by the first image sensor starting from the first pose of the motion rendering device; determining a second transformation that allows mapping between a first coordinate system associated with the motion rendering device and a third coordinate system associated with the second image sensor, wherein the second transformation is determined based on a plurality of second correspondences between known physical positions of a plurality of second features of a second calibration plate starting at the first pose of the motion rendering device and second positions of the plurality of second features detected in a second image of the second calibration plate obtained by the second image sensor starting at the first pose of the motion rendering device; as well as A correspondence between features of the target object found in the first and second images acquired by the first and second image sensors and the first coordinate system is established based on the first and second transformations.

7. The system according to claim 6, wherein: The motion rendering device is configured to provide at least one of a translational movement and an in-plane rotational movement.

8. The system according to claim 7, wherein: The computer program is operable to cause the processor to determine a motion correction transform that compensates for systematic motion errors associated with the motion rendering device.

9. The system according to claim 6, wherein: The computer program is operable to cause the processor to recalibrate the system after a first period of time, including re-determining: the plurality of first corresponding relationships; the plurality of second corresponding relationships; the first transformation; as well as The second transformation.

10. The system according to claim 9, wherein: Recalibrating the system includes adjusting one or more pre-calibrated parameters.

11. A system for machine vision, comprising: A processor configured to execute a computer program stored in a memory, the computer program configured to: receiving a first image of a target object in a first pose of a motion rendering device from a first image sensor, wherein the motion rendering device is associated with a first coordinate system and is configured to directly or indirectly carry the target object, wherein the first image sensor is associated with a second coordinate system; receiving a second image of the target object in a first pose of the motion rendering device from a second image sensor, wherein the second image sensor is associated with a third coordinate system; determining a first transformation that allows mapping between a first coordinate system associated with the motion rendering device and a second coordinate system associated with the first image sensor, wherein the first transformation is determined based on a first subset of a plurality of features of the calibration object detected in a first image of the calibration object in a first pose of the motion rendering device obtained by the first image sensor; determining a second transformation that allows mapping between a first coordinate system associated with the motion rendering device and a third coordinate system associated with the second image sensor, wherein the second transformation is determined based on a second subset of the plurality of features of the calibration object detected in a second image of the calibration object in a first pose of the motion rendering device obtained by the second image sensor; A correspondence between features of the target object found in the first and second images acquired by the first and second image sensors and the first coordinate system is established based on the first and second transformations.

12. The system according to claim 11, wherein: The motion rendering device is configured to provide at least one of a translational movement and an in-plane rotational movement.

13. The system according to claim 12, wherein: The computer program is operable to cause the processor to determine a motion correction transform that compensates for systematic motion errors associated with the motion rendering device.

14. The system according to claim 11, wherein: The computer program is operable to cause the processor to recalibrate the system after a first period of time, including re-determining: a first subset of the plurality of features; a second subset of the plurality of features; the first transformation; as well as The second transformation.

15. The system of claim 14, wherein: Recalibrating the system includes adjusting one or more pre-calibrated parameters.

16. A system for machine vision, comprising: A processor configured to execute a computer program stored in a memory, the computer program configured to: receiving, from a first image sensor, a first image of the target object in a first pose of a motion rendering device, wherein the motion rendering device is associated with a first coordinate system and is configured to directly or indirectly carry a first image sensor and a second image sensor, wherein the first image sensor is associated with a second coordinate system; receiving a second image of the target object from the first pose of the motion rendering device from a second image sensor, wherein the second image sensor is associated with a third coordinate system; determining a first transformation that allows a mapping between a first coordinate system associated with the motion rendering device and a second coordinate system associated with the first image sensor, wherein the first transformation is determined based on a first subset of a plurality of features of the calibration object detected in a first image of the calibration object obtained by the first image sensor starting from a first pose of the motion rendering device; determining a second transformation that allows mapping between a first coordinate system associated with the motion rendering device and a third coordinate system associated with the second image sensor, wherein the second transformation is determined based on a second subset of the plurality of features of the calibration object detected in a second image of the calibration object obtained by the second image sensor starting from the first pose of the motion rendering device; A correspondence between features of the target object found in the first and second images acquired by the first and second image sensors and the first coordinate system is established based on the first and second transformations.

17. The system of claim 16, wherein: The motion rendering device is configured to provide at least one of a translational movement and an in-plane rotational movement.

18. The system of claim 17, wherein: The computer program is operable to cause the processor to determine a motion correction transform that compensates for systematic motion errors associated with the motion rendering device.

19. The system of claim 16, wherein: The computer program is operable to cause the processor to recalibrate the system after a first period of time, including re-determining: a first subset of the plurality of features; a second subset of the plurality of features; the first transformation; as well as The second transformation.

20. The system of claim 19, wherein: Recalibrating the system includes adjusting one or more pre-calibrated parameters.

Images