System and method for calibrating a camera, and object tracking system using a calibrated camera

Abstract

A method is provided for calibrating a camera without decomposing the camera parameters into external and internal components. Furthermore, a method is provided for tracking a moving object, which includes capturing one or more image frames of the moving object using one or more calibrated cameras calibrated according to a calibration method that generates and uses respective transformation matrices for mapping features of a three-dimensional (3D) real-world model to corresponding two-dimensional (2D) image features. The tracking method further includes determining the motion characteristics of the moving object based on one or more image frames captured from each of the one or more calibrated cameras using a hardware processor, wherein determining the motion characteristics is based on implicit internal camera parameters and implicit external camera parameters of each of the respective transformation matrices from each of the one or more calibrated cameras.

Description

[Technical Field]

[0001] This disclosure relates to a camera calibration method, and to a method and system for using a calibrated camera(s) to track and measure the motion of a target object in three-dimensional space. [Background technology]

[0002] Essentially, a camera provides image mapping from three-dimensional space to two-dimensional space or an image plane. Current camera calibration techniques provide model parameter values ​​necessary to calculate the line of sight in space corresponding to a point on the image plane.

[0003] The calibration, or "projection," matrix estimated during camera calibration is typically decomposed into 11 geometric parameters that define a standard pinhole camera model. Generally, camera model parameters include external and internal parameters. External camera parameters include the camera's three-dimensional position and orientation in the world, while internal camera parameters include, among other things, the focal length and the relationship between pixel coordinates and camera coordinates.

[0004] In many applications, camera calibration is necessary to reconstruct three-dimensional quantitative measurements of a scene observed from a two-dimensional image. For example, a calibrated camera can be used to determine how far an object is from the camera, or its height. Typical calibration techniques use three-dimensional, two-dimensional, or one-dimensional calibration objects whose shape in three-dimensional space is known with very good accuracy.

[0005] One of the objectives of camera calibration is to obtain a projection "matrix" from a set of world points and their image coordinates, and then to obtain the internal and external camera parameters from that matrix in a decomposition step. However, decomposition into external and internal camera parameters is one of the major challenges in calibration due to reprojection errors. Furthermore, several assumptions and constraints are set in the decomposition step to extract the external and internal camera parameters, but these may not always apply, such as the absence of lens distortion or tilt.

[0006] Therefore, it is desirable to have a camera calibration method that does not require decomposition into internal and external camera parameters. Furthermore, it is desirable to have a camera system that acquires measurements without requiring any assumptions about internal or external camera parameters. [Overview of the Initiative]

[0007] A camera system and method are provided that acquires measurements of moving objects without requiring any assumptions about internal camera parameters.

[0008] Furthermore, a camera system and method are provided for acquiring measurements of moving objects, eliminating the need to separate camera parameters from the camera's calibration matrix. This allows the camera to accommodate any lens and any shift or tilt (intentional or unintentional) in camera settings for tracking moving objects.

[0009] In addition, a camera system calibration method is provided for calibrating a camera used to acquire measurements of moving objects without decomposing the camera parameters into external and internal parts.

[0010] In one embodiment, the camera system and method include a single camera device.

[0011] In one embodiment, during the calibration process, a virtual reference is aligned to a physical object in a global reference space in order to acquire camera parameters.

[0012] A robust camera calibration system and method, along with the system itself, allows users to use any camera without having to fine-tune camera parameters to conform to global standards.

[0013] According to one embodiment, a method for tracking a moving object is provided. This method includes capturing one or more image frames of the moving object from each of one or more calibrated cameras, each of which is calibrated according to a calibration method that generates and uses its respective transformation matrix for mapping features of a three-dimensional (3D) real-world model to corresponding two-dimensional (2D) image features, and determining the motion characteristics of the moving object based on the one or more image frames captured from each of the one or more calibrated cameras using a hardware processor, wherein determining the motion characteristics is based on implicit internal camera parameters and implicit external camera parameters of each of the respective transformation matrices from each of the one or more calibrated cameras.

[0014] In a further embodiment, an object tracking system is provided. This object tracking system includes a camera system comprising one or more calibrated cameras, each camera capturing one or more image frames of the position of an object in motion, and each of the one or more calibrated cameras being calibrated according to a calibration method that generates and uses its respective transformation matrix for mapping features of a three-dimensional (3D) real-world model to corresponding two-dimensional (2D) image features; and a hardware processor coupled to memory for storing instructions, the instructions configured, when executed by the processor, to perform the task of determining the motion characteristics of an object based on one or more captured image frames, wherein the determination of the motion characteristics of the object is based on implicit internal camera parameters and implicit external camera parameters of its respective transformation matrix from each of the one or more calibrated cameras.

[0015] In a further embodiment, a method for calibrating a camera is provided. This method provides a transformation matrix (H) representing multiple camera parameters and one or more corrections (H) for multiple implicit camera parameters of the transformation matrix. Δ The method involves aligning a two-dimensional image feature (q) with a reference two-dimensional image feature (q') by applying a transformation matrix (H') to obtain an updated transformation matrix (H'), wherein the two-dimensional image feature (q) and the reference two-dimensional image feature (q') are represented in pixel coordinates.

[0016] In addition to this embodiment, the camera parameters include implicit external and implicit internal camera parameters, and the camera is calibrated without decomposing the implicit camera parameters into explicit external and explicit internal camera parameters.

[0017] In addition to this embodiment, a method for calibrating the camera includes modifying one or more implicit external camera parameters. This modification includes adjusting the implicit external parameters through affine correction. Affine correction restores six degrees of freedom (DoF) that represent explicit external camera parameters.

[0018] Further features of various embodiments, as well as their structure and operation, will be described in detail below with reference to the accompanying drawings. In the drawings, similar reference numerals indicate the same or functionally similar elements. [Brief explanation of the drawing]

[0019] [Figure 1] This is an illustrative diagram of an object measuring device employing multiple camera devices calibrated according to the method of this disclosure. [Figure 2] An example of applying the present disclosure to perform a monocular camera calibration method without performing decomposition to extract internal and external camera parameters, according to one embodiment, is shown. [Figure 3] An example of applying this disclosure to perform a field camera calibration method, which includes generating an updated matrix H' by providing affine corrections (external camera parameters), is shown. [Figure 4] This is an illustrative diagram illustrating a calibration method for the camera shown in Figure 1, according to one embodiment. [Figure 5A] In one embodiment, an example of a calibration method is shown, specifically a method for performing on-site or field calibration of an exemplary camera device. [Figure 5B] A further method for field calibration / correction in an example processing step of a calibration method according to one embodiment is shown. [Figure 6] An example of applying this disclosure to perform measurement (parameter tracking) of a moving object using a "hard synchronization" method with a synchronized camera is shown. [Figure 7]This document provides an example of applying the present disclosure to perform measurement (parameter tracking) of a moving object using a "soft synchronization" method with an asynchronous camera. [Figure 8A] A conceptual representation of an implicit three-dimensional representation for a given trajectory model assuming soft synchronization, according to one embodiment, is shown. [Figure 8B] This example shows an instance in which an inappropriate triangulation ("hard synchronization" method) is applied to "soft synchronization" data according to one embodiment. [Modes for carrying out the invention]

[0020] In particular, the technical terms used herein are for the purpose of describing specific embodiments and are not intended to impose any limitations on the scope of this disclosure. Furthermore, unless otherwise intended in context, terms used herein, even when expressed in the singular form, shall be interpreted as having a plural meaning. The phrases used herein, such as "is configured with" and "include," should not be interpreted as necessarily including all components or steps described herein, but rather as meaning that one or more components or steps from all components or steps may not be included, or that one or more other components or steps may be further included.

[0021] Furthermore, if a detailed description of known technology in the relevant technical field to which this disclosure relates is deemed to obscure the nature and essence of the technology disclosed herein, such detailed description will be omitted herein.

[0022] As mentioned in this specification, camera calibration is the process of obtaining camera parameters in the form of external and / or internal parameters. Methods commonly adopted in camera calibration include, but are not limited to, three-dimensional direct linear transformation (DLT), QR decomposition, and perspective-n-point (PnP). Three-dimensional DLT calibration obtains a homography or transformation matrix. QR decomposition decomposes the homography matrix into an external matrix and an internal matrix (a specific internal model). PnP, such as P3P (n = 3), obtains only the external camera parameters on the premise that the internal parameters are known. Camera calibration methods can utilize pre-calibration results by the same or different methods. With prior information, for example, using the gradient descent method (GD), camera parameters can be iteratively evaluated from the default to the ground truth.

[0023] As further mentioned in this specification, DLT is one method of monocular camera calibration, and as its first step, Equation (1) is as follows: H={h xx ,h xx ,h xy ,h xz ;h yx ,h yy ,h yz ;h zx ,h zy ,h zz ;h x ,h y ,h z}(1) is followed by calibration pattern, a set of feature points with known defined positions in the three-dimensional world

[0024]

Number

[0025]

Number

[0026]

Number

[0027]

number

[0028]

number

[0029] To solve the above system of equations, this relationship is reconstructed into the following form: Ah = 0, where A is a three-dimensional point.

[0030]

number

[0031]

number

[0032] As further noted herein, PnP uses a set of methods for estimating the camera's pose (e.g., 6 DoF external parameters) assuming that the intrinsic parameters are given or known, such as when the camera is pre-calibrated. For this reason, PnP is not considered a perfect calibration method because it assumes the intrinsic parameters. The minimum number of three-dimensional points for camera pose estimation is 3 (e.g., P3P, 3 pairs of corresponding points). Since no decomposition is applied in the camera calibration methods described herein, PnP, including P3P, is not used in the factory calibration step. In some embodiments, therefore, PnP or P3P is used only in the field calibration step. That is, for an "uncalibrated" or "partially calibrated" camera, if the intrinsic parameters are explicitly unknown and hidden within the homography matrix, the affine correction is formally applied to the default homography matrix.

[0033] As used herein, an “uncalibrated” or “partially calibrated” camera refers to a camera whose pose estimation is either unknown or incomplete. In some embodiments, pose estimation involves the camera's orientation R T and camera position - sR T This includes. In some embodiments, a homography matrix without resolution can result in incomplete pose estimation (e.g., position only, orientation not estimated). Typically, the orientation of the camera is defined by the optical axis. However, the optical axis is an internal camera parameter (principal point: C x ,c y (etc.)

[0034] The camera's position is the translational part of the transformation T that positions the camera from its own coordinate system to the world coordinate system. The external camera matrix C is the inverse of the transformation matrix T, given by equation (3) below: T=C -1 ={R;s} -1 ={R T ;-sR T (3)

[0035] Based on the above equation, the camera position p is obtained from the external camera matrix C. c is, p c =-sR T In some embodiments, the camera position p c It can also be obtained from the translational part of the inverse (undecomposed) of the homography matrix H. Therefore, the camera position p c =-sR T It is considered to be independent of the intrinsic matrix K.

[0036] As described herein, the decomposition of internal and external camera parameters is a process of dividing the homography matrix H, which has 11 DoFs, into external C (6 DoFs) and internal K (5 DoFs), i.e., H = {R;s}K.

[0037] When camera parameters are decomposed into external and internal parts, this decomposition allows for multiple possible solutions (for example, given an internal model: H=C n K n In the case of QR code decomposition, four parts are generated, which leads to ambiguity.

[0038] As used herein, "QR decomposition" decomposes the "rotation" portion (RK) of the homography matrix H into orthogonal rotation matrices and lower (or upper) triangles.

[0039] Different internal model K m Applying this may yield even more distinctive solutions. A certain internal model can be transformed into any other model by applying the rotation matrix R according to equation (4):

[0040]

number

[0041] As described herein, the homography matrix H is a 4x3 matrix defined to an arbitrary scale. Therefore, the homography matrix H has 11 DoFs. The homography matrix H is the product of the external matrix C and the internal matrix K, defined according to equation (5): H=CK(in the formula C={R;s}(H={RK;sK}(5))

[0042] Including projection onto the image plane,

[0043]

number

[0044]

number

[0045]

number

[0046] As described herein, the external camera matrix C is a 4x3 matrix. Matrix C contains a rotation matrix R and a translation vector s:C={R;}. The external camera matrix C has six independent parameters (six DoFs): three DoFs for rotation and three DoFs for translation. The external camera matrix C maps three-dimensional points from the world three-dimensional coordinate system to the camera three-dimensional coordinate system without applying projection onto the image plane.

[0047] The internal camera matrix K is a 3x3 matrix. It includes, in particular, the camera's internal parameters, such as focal length, principal axis (shift), and tilt. The internal camera matrix K can be expressed according to equation (6) or (7) below: K={fx,0,0;s,fy,0;cx,cy,1}("Astigmatism Skew" Model) (6) or, K={f,0,t x ;0,f,t y ;c x ,c y ,1}("Tilt-Shift" model)(7)

[0048] The internal camera matrix K has five independent parameters (five DoFs).

[0049] The three-dimensional rotation matrix "R" as used herein is a 3x3 orthonormal matrix, and is a general-purpose three-dimensional transformation T, or external camera matrix C=T -1 These are its components. The rotation matrix "s" holds three independent parameters (three DoFs). Other equivalent representations of three-dimensional rotation include axial angle method and unit quaternion, both of which also have three DoFs.

[0050] The translation vector "s" as used herein is a three-dimensional vector, and is a general three-dimensional transformation T, or an external camera matrix C = T -1 These are its components. The translation vector "s" holds three independent parameters (three DoFs).

[0051] As referred to herein, degrees of freedom (DoF) refer to the number of independent parameters (which may sometimes be hidden). For example, the angles of three-dimensional rotation representations using rotation matrices, quaternions, and axis angles each have three DoFs.

[0052] As referred to herein, and further as described in this disclosure, camera calibration is the process of obtaining camera parameters in the form of external and / or internal parameters. In some embodiments, camera calibration includes a multi-step process, as described below. In some embodiments, camera calibration includes a two-step calibration, the first step being called factory calibration and the second step being called field calibration. According to aspects of this disclosure, factory calibration is typically applied at least once to obtain a transformation matrix H that represents a plurality of camera parameters. The plurality of camera parameters include internal camera parameters and external camera parameters. In some embodiments, the transformation matrix H is for mapping features of a three-dimensional real-world model to corresponding two-dimensional image features. In some embodiments, the transformation matrix is ​​a homography matrix H. In some embodiments, the transformation matrix H represents internal and external camera parameters. In some embodiments, the transformation matrix represents implicit internal and implicit external camera parameters. In some embodiments, the transformation matrix may be obtained by the DLT method or other suitable algorithm, as described below. In some embodiments, the field calibration step may be applied once or multiple times. In some embodiments, field calibration may be referred to as calibration correction with respect to external camera parameters. In some embodiments, the second step is to implicitly correct the external camera parameters in the homography matrix H without decomposing the internal and external camera parameters. Field calibration may employ a PnP algorithm, including P3P, using the homography matrix H obtained from the first step or factory calibration as an initial estimate. Thus, in field calibration, only affine transformation correction is applied without changing the internal camera parameters implicitly defined in the homography matrix H.

[0053] Figure 1 shows multiple camera devices 1001, 1002, ... 100 calibrated according to the method of this disclosure. nThis is an illustrative diagram of an object measuring device 10 employing the above. Each of the multiple camera devices can be wired or wirelessly connected to a computing device 200 to perform object tracking and parameter measurement operations.

[0054] Each camera device, such as the exemplary camera device 100 shown in Figure 1 according to the embodiments of this disclosure, may include components such as at least a lens 101 and a sensor 105, a digital signal processor or controller 110 which may include one or more hardware microprocessors, a communication unit 120 connected to and controlled by the controller 110, an image capture unit 130 for capturing images and / or video, a calibration unit 140 for controlling camera calibration operations, a memory 150, and a tracking unit 160.

[0055] An exemplary camera device 100, shown in Figure 1 and relating to embodiments of the present disclosure, may include any device, system, component, or set of components configured to capture images and / or video. In one embodiment, the camera device 100 includes an image capture unit 130 having, for example, optical elements such as a lens 101 and a filter, and at least one image sensor 105 on which images can be recorded. Such an image sensor 105 may include any device that converts an image represented by incident light into an electronic signal. The image sensor 105 may include a plurality of pixel elements that can be arranged as a pixel array (e.g., a grid of pixel elements) for acquiring images. For example, the image sensor 105 may include a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) image sensor. The pixel array may include a two-dimensional array having any aspect ratio. The image sensor 105 may be optically aligned with various optical elements that focus light onto the pixel array, for example, a lens 101. For example, it may include any number of pixels, such as hundreds or thousands of megapixels.

[0056] In one embodiment, the image capture unit 130 acquires images under the control of the controller 110, and the acquired images are stored in the memory storage unit 150 under the control of the controller 110. Further data and information for the operation of the camera 100 may be stored in the memory 150. The images acquired by the image capture unit 130 are stored in the memory 150. Each acquired image has unique identification information elements, which may correspond to the time when the image was acquired. Furthermore, other types of tracking information detected from at least one image according to target designation information may be stored in the memory 150. Hereinafter, the storage area of ​​the memory 150 in which images are stored may also be called the image storage unit 152, and the storage area of ​​the memory 150 in which "tracking information" related to target object tracking is stored will be called the tracking information storage unit 155. The camera memory 150 also stores local camera calibration information, such as the values ​​of external and internal parameters used for camera calibration. In some embodiments, the camera memory 150 can also store values ​​of external and internal camera parameters obtained from camera calibration.

[0057] An image processing unit 170 is further provided to perform digital signal processing on raw camera image data, such as filtering, noise reduction, and image sharpening.

[0058] As further shown in Figure 1, the exemplary camera 100 may include a communication unit 120 that enables the camera to receive remote control signals for performing camera operations in connection with object tracking and measurement of motion parameters, and further include at least one module for performing wired and / or wireless communication with a remote controller processor or remote computing device 200 (remote control unit). The communication unit 120, under the control of the controller 110, can receive control signals from the remote control unit through at least one such module and transfer acquired images to the remote controller processor or computing device 200.

[0059] Whether controlled remotely or locally using the controller 110, the camera 100 can operate at a specific frame rate or capture a specific number of images in a given time. The camera 100 can operate at a frame rate of approximately 60 frames per second (fps) or higher, for example, approximately 100 to approximately 300 frames per second (fps). In some embodiments, a smaller subset of pixels available in the pixel array may be utilized to enable the camera to operate at a higher frame rate.

[0060] The tracking unit 160 operates to detect a target corresponding to the tracked received target object. In some embodiments, the target includes a ball used in golf, cricket, baseball, softball, tennis, and the like.

[0061] In one embodiment, an arbitrary number of different triggers may be used to cause the exemplary camera 100 to capture one or more images of a moving object. In non-limiting examples, the camera 100 may be triggered when it receives a timed activation signal, or when it is detected, recognized, or estimated that a moving object is within the camera's field of view, or when the moving object first begins or changes its flight (for example, when a baseball is thrown, hit, hit, hit, served, or launched), or when a moving object is detected in the first row of pixels in a pixel array.

[0062] In one embodiment, the remote controller processor or computing device 200 can be physically separated from remote cameras 100, 1001, 1002, ..., 100 to perform object tracking and measurement according to embodiments of the present disclosure. nThe overall operation can be controlled. The computing device 200 may configure the controller 110 to control the image capture unit 130 and acquire images according to the control signal 175 received through the communication unit 120. The controller 110 can then store the acquired images in the device and / or transmit the stored images, along with associated image identification and tracking information, to the computing device 200 for further processing. Further processing may include the application of one or more algorithms for receiving raw camera image data in a series of image frames. For example, according to embodiments of this specification, exemplary camera measurement of a moving object may include, but is not limited to, identifying one or more of the object's radius, object's center, velocity, elevation angle, and azimuth angle in each image, calculated, for example, based on the object's radius, object's center, and a previously measured camera alignment value.

[0063] According to embodiments of this specification, Figure 2 shows an implementation of the method of the present disclosure, which uses a calibration unit 140 to perform a factory camera calibration method 250, i.e., a first step of camera calibration. In some embodiments, the factory calibration 250 is performed using a given known three-dimensional feature and a two-dimensional image feature detected from the camera, thereby generating a transformation matrix H. Such a transformation matrix may be called the “default” transformation matrix. As described herein, the transformation or projection matrix H represents internal and external camera parameters having 11 DoFs. As seen in Figure 2, the factory calibration is performed without performing a decomposition that explicitly extracts the internal and external camera parameters. In some embodiments, following the factory calibration, the camera may undergo further calibration (i.e., field calibration) before using the calibrated camera. Thus, in some embodiments, a camera calibrated according to the factory calibration may still be considered an “uncalibrated” camera. In some embodiments, a camera calibrated according to the factory calibration refers to a “partially calibrated” camera.

[0064] In accordance with further embodiments of this specification, Figure 3 shows an example of the application of the present disclosure for performing a field calibration method 300. In some embodiments, field calibration refers to a second step of camera calibration as described herein. In some embodiments, as described above, field calibration refers to a method 300 for calibrating an “uncalibrated” camera or a “partially calibrated” camera. In some embodiments, field calibration is performed based on a given known three-dimensional feature and a two-dimensional image feature detected from the camera, as well as a given default camera parameter from a transformation matrix H, where the transformation matrix H is obtained from a first step of calibration, i.e., factory calibration (see Figure 2). In some embodiments, the calibration unit 140 performs a method for generating an updated correction matrix H' by performing affine correction to obtain updated external camera parameters. This is therefore advantageous when the calibration methods, including factory calibration and field calibration as described herein, are performed without decomposing the internal and external camera parameters. Therefore, the accuracy of the calibration method described herein is advantageously superior to that of decomposition because no assumptions were made.

[0065] In some embodiments, a method is provided for tracking a moving object. This tracking method includes capturing one or more image frames of the moving object from each of one or more calibrated cameras. In some embodiments, each of the one or more calibrated cameras may be calibrated according to the calibration method described herein. In some embodiments, this calibration generates and uses respective transformation matrices for mapping features of a three-dimensional real-world model to corresponding two-dimensional image features. The tracking method further includes using a hardware processor to determine the motion characteristics of the moving object based on one or more image frames captured from each of the one or more calibrated cameras. In some embodiments, determining the motion characteristics is based on implicit internal camera parameters and implicit external camera parameters of each transformation matrix from each of the one or more calibrated cameras. In some embodiments, the tracking is three-dimensional tracking.

[0066] In some embodiments, each of one or more calibrated cameras is calibrated without decomposing multiple camera parameters into explicit external and explicit internal camera parameters. In some embodiments, a first calibration (factory calibration) and a second calibration (field calibration) are performed without decomposing camera parameters into explicit external and explicit internal camera parameters. In some embodiments, both the first calibration step, i.e., factory calibration, and the second step calibration, i.e., field calibration, are performed without decomposing camera parameters into external and internal camera parameters.

[0067] In some embodiments, following the first step or factory calibration, each of one or more calibrated cameras is further calibrated by modifying one or more implicit external camera parameters to obtain explicit external camera parameters. In some embodiments, this modifying step includes adjusting the implicit external camera parameters through affine correction.

[0068] Figure 4 is an illustrative diagram showing a field calibration method 400 for the exemplary camera 100 of Figure 1, according to one embodiment. In some embodiments, for example, a field calibration method for restoring the camera position in a three-dimensional world using a reference pattern (natural or artificial) is performed without explicitly using external and internal camera parameters. That is, the field calibration method is performed without decomposing the camera parameters into internal and external parts. The field calibration method may be performed by processing logic that may include hardware (processor, calibration unit 140, circuitry, dedicated logic, etc.) and software (such as that which runs on the hardware processor 110 or computer system 200), or a combination of both, and this processing logic may be included in the calibration unit 140.

[0069] Referring to Figure 4, in the initial step 403, the default, or "given," camera parameters of the camera device are obtained. In some embodiments, the default camera parameters may be included in an initial calibration, projection, or "transformation" matrix ("H0") that represents a plurality of default internal and external camera parameters. In some embodiments, the camera is operated to obtain real-world three-dimensional feature points of the image, which are acquired through the camera's field of view (FoV) and represented as feature points "p". The default two-dimensional feature points of the camera image plane are q o It is expressed as, and the desired two-dimensional feature point is q k It is called [name].

[0070] In some embodiments, the three-dimensional model or reference pattern may include, but is not limited to, a stencil. In some embodiments, the three-dimensional model or reference pattern may be any suitable reference, such as a cage corner, stamp, mark, line, or boundary line. In some embodiments, the three-dimensional model or reference pattern may be static, i.e., its position does not change. In some embodiments, the three-dimensional model or reference pattern may be moving. In such embodiments, a first camera system (e.g., a stereo system having at least two cameras) is calibrated using the method described herein and used to track a moving object and obtain three-dimensional position and / or trajectory parameters. This information can be used to perform calibration of a second camera system, which may be the same as or a different stereo system as the first camera system. In such embodiments, the second camera system is also used to track the same moving object and obtain camera parameters for the second camera system.

[0071] Furthermore, based on the natural or artificial reference pattern described in step 406, the updated transformation matrix H k+1 A step is also provided to control the camera so as to update camera parameters in three-dimensional space, represented as H. This step includes updating external camera parameters having six DoFs. In step 406, a camera with known or given internal parameters (from factory calibration) is manually calibrated by adjusting camera parameters related to small axis angles and / or translation vectors. In some embodiments, this adjustment affects implicit external camera parameters having six DoFs. Next, the transformation matrix H k One or more correction matrices (H) for multiple implicit camera parameters Δ By applying this, manual alignment can be performed to obtain two-dimensional virtual image features (q2) using two-dimensional image features of the reference image (q1), thereby updating the transformation matrix H k+1The data is acquired, and then the camera is calibrated. As described above, camera calibration is performed without decomposing the camera parameters into explicit internal and external parts. In this embodiment, the two-dimensional virtual image feature (q2) and the two-dimensional image feature (q1) of the reference image are represented in pixel coordinates.

[0072] Continuing in step 410 of Figure 4, based on the manual alignment performed during the calibration in step 406, the processor applies a correction matrix "H" to the camera. Δ The following can be constructed: Then, in step 414, the constructed correction matrix H Δ In step 410, the initial transformation matrix H associated with the default camera parameters is used. k Applied to this, the updated matrix H related to the camera parameters updated according to equation (8) is obtained. k+1 Get: H k+1 =H Δ H k (8)

[0073] Furthermore, according to step 417 in Figure 4, in step 414, the updated matrix H k+1 Once obtained, the updated matrix H k+1 This is applied to a three-dimensional model, where the new two-dimensional point is given by the following equation: q k+1 ∝ pH k+1 The new two-dimensional point (camera position) can be obtained according to the updated homography matrix H k+1 It is obtained from. Since the three-dimensional point p is static, the two-dimensional point q k H k It changes according to H Δ H k+1 By adjusting desiredq K It converges to [a certain point].

[0074] Following step 420 in Figure 4, it is determined whether the two-dimensional image feature (q) and the two-dimensional image feature points (q') of the virtual reference image are less than or equal to a certain target threshold number of pixels. That is, the current transformation matrix H k The two-dimensional position (q) evaluated using and the updated correction matrix Hk+1 The difference between the desired two-dimensional position (q') obtained from 、 In other words, it determines whether the quantity |q-q'| is less than or greater than a threshold "limit point". In one embodiment, the threshold is expressed as a distance in pixels, for example, 1 pixel, and is alternatively called the "reprojection error".

[0075] If, in step 420, it is determined that the quantity |q-q'| is greater than a predetermined threshold (e.g., greater than 1 pixel), method 400 returns to step 406 and repeats, and steps 406, 410, 414, 417, and 420 are all repeated. In some embodiments, the steps of method 400 are iterative and are repeated until the quantity |q-q'| is less than or equal to the threshold, e.g., ≤1 pixel, at which point these two-dimensional points are considered to be "in place", and method 400 terminates. Subsequently, the camera is considered fully calibrated and ready for measurement purposes. The acquired correction transformation matrix H is obtained according to Equation 2. k+1 and a set of feature points in three-dimensional (orthogonal) coordinate space

[0076]

number

[0077]

number

[0078]

number

[0079] In one embodiment, automatic camera calibration correction, i.e., a second stage of calibration, may be performed. The calibration calculation method is as follows:

[0080] correction matrix H Δ =Δr k ;Δsk Determine the solution of {} from the following system of linear equations: (pΔr k +Δs k )r k ×q k =-(pr k +s k )×q k (9) Equation (9) is derived from the projection constraint described by the following equation (10) q k ∝pH k+1 =pH Δ H k =(pr Δ,k +Δs k )r k +s k (10)

[0081] Approximation of angles:

[0082]

Number

[0083] r Δ,k~ I+Δr[[ID= seventy-five]] k where Δr k is a skew-symmetric matrix. Next, set k = 0, where if k is the iteration counter, H k is initially given by the following equation (11):

[0084] ) ​

[0085] In some embodiments, this method then performs the following steps: 1. Δr k and s Δ,k Regarding (pΔr k +s Δ,k )·(q×r k )=(q×(pr k +s k Solve )), in the equation, r k represents a change in rotation, s k This represents the change in the translational movement of the camera. 2. H'=H k Assign Δr k and s Δ,k If it is small, it stops there. 3. If not, Δr k From valid r Δ,k Construct (for example, using the half-angle formula, and if the values ​​do not converge, use Cayley's formula. This is to avoid using polynomial equations). In addition, 4. H k+1 ={r k+1 =r Δ,k r k, s k+1 =s Δ,k r k +s k}, update k=k+1 and repeat.

[0086] In one embodiment, this method solves a system of linear equations that represent the relationship between a three-dimensional point in the scene and the projection of the camera onto the two-dimensional image plane, thereby solving a set of variables using the DLT method from the set of analogous relations q∝pH', where q (two-dimensional point) and p (three-dimensional point) are known vectors, "∝" indicates proportionality, and H' is a matrix (or linear transformation) containing the unknowns to be solved. According to this embodiment, from the transformation matrix H', q x and q y A two-dimensional camera position is obtained, consisting of the following elements.

[0087] Figure 5A shows an exemplary environment 550 to which a method according to one embodiment of the present disclosure is applied to perform field calibration of an exemplary camera device 100 in a second stage of an illustrative demonstration of the calibration process. The figure shown in Figure 7A represents an image in the camera viewer. As seen in Figure 7A, the camera calibration unit configures the camera to generate a virtual reference image 555 whose translational and rotational motion is controllable in order to align with a corresponding physical real-world object 565 in the camera view for affine correction in step 406 of Figure 4. In one embodiment, the camera includes physical directional buttons or an operable “joystick” 575 which the user operates to move the virtual reference image 555 so that the virtual stamp 555 matches the physical stamp 565, i.e., is aligned within a threshold distance (q-q'). The movement / alignment data obtained by aligning a calibration reference image 555 with a physical real-world object 565 within the camera's field of view is an offset correction used to generate new / updated external parameters for the camera.

[0088] In one embodiment, the method described in Figure 4 restores the camera position in the three-dimensional world by reference pattern 565, i.e., a stamp or wicket, without explicitly using external and internal camera parameters, i.e., without decomposing the camera parameters into internal and external parts. In one embodiment, affine correction can be performed using a natural or artificial reference pattern. The artificial reference pattern 565 may include other types of physical calibration objects, such as markers or cage corners.

[0089] Figure 5B illustrates a further method of field calibration / correction. In the example sports cage 580 shown in Figure 5B, the camera viewer presents a virtual reference image as marks 556 that mark geometric shapes, such as squares, which can serve as references used in calibration. Other examples of virtual references include markings on the corners of a cage (e.g., a batting cage), lines shown as field line markings 557, and field markings. The camera includes controllable physical directional buttons 592 or an operable “joystick” (not shown) to adjust translational (e.g., offset) and rotational (e.g., angle) parameters 590 related to the movement of the reference markings 556, 557 when performing calibration. The movement of virtual reference calibration markings 556 and 557 within the camera view is aligned with the corresponding physical real-world object(s) in three-dimensional space, represented, for example, by orthogonal x, y, and z directions in a Cartesian coordinate system, for affine correction. This affects the six DoFs, and consequently, updated external camera parameters are obtained.

[0090] Furthermore, in the camera view of Figure 5B, the default (H) is set as the homography matrix value 595 for each calibration. n ), correction (H Δ ), and the updated matrix (H n+1 This is shown. From the updated homography matrix value 595, the camera position in the three-dimensional world is reconstructed without explicitly using the internal and external camera parameters, i.e., without decomposing the camera parameters into internal and external parts.

[0091] Other examples of reference markings include checkerboards, AprilTag, and also the calibration device described in the pending U.S. Patent Application No. 18 / 527,245 filed on December 2, 2023, as well as moving objects including balls. A natural pattern is a known object a priori positioned relative to the camera. That is, in the first stage of calibration, the relative position of the pattern to the camera is (a priori) known. The camera parameters are reconstructed by the DLT method described above without decomposing the camera parameters into internal and external parts.

[0092] Therefore, referring to Method 400 in Figure 4, a two-stage calibration process is provided. In the first stage, the implicit camera parameters are restored using DLT, restoring 11 DoF parameters (e.g., 6 implicit DoF external parameters and 5 DoF internal parameters). Next, in step 410, the second stage of calibration requires on-site affine correction without changing the implicit internal parameters. This on-site correction assumes that: the camera's implicit internal parameters remain unchanged, i.e., the focal length, distortion profile, etc., remain the same, and only the camera's translation and / or rotation relative to world coordinates changes. Thus, the 6 external DoF parameters are implicitly changed without decomposing the combination set of calibration parameters (11 DoF) into an internal part (5 DoF) and an external part (6 DoF).

[0093] In one embodiment, for example, to extend the depth of focus along the field of a stadium, the system further provides support for a tilt lens by implicitly and unspecified models, by performing a lens tilt calibration method, for example, described by the principle of Scheinproof, on internal camera parameters.

[0094] measurement The exemplary camera system in Figure 1 is further configured as an object measurement system applicable to reconstructing the position of a three-dimensional world object using stereo (two or more) cameras without decomposing the camera parameters into internal and external parts. In other words, according to this embodiment, a measurement system and method for tracking objects are provided. In one embodiment, the object tracking measurement includes hard synchronization and soft synchronization parameter processing to reconstruct the trajectory of a projectile by a normalized higher-order trajectory model, for example, by two or more field-of-view (FOV) stereo cameras, with or without overlapping, without decomposing the camera parameters into internal and external parts. It should be understood that each of the two or more cameras in the object measurement system may be calibrated according to the method described herein.

[0095] Hard synchronization For hard-synchronized object tracking and measurement, the camera system in Figure 1 includes cameras, for example, one or more cameras 1001, 1002, ..., 100 n However, using the same external VSync signal, the camera is hardwired with general-purpose I / O (GPIO) lines, and the frames are configured to be captured synchronously with the same timestamp.

[0096] In some embodiments, Figure 6 shows an example application of performing measurements to track parameters of a moving object using, for example, a “hard synchronization” method 600. In some embodiments, this hard synchronization method uses a synchronization camera calibrated according to the method described herein. In some embodiments, the camera is calibrated without decomposing the camera parameters into internal and external camera parameters during calibration. In some embodiments, this hard synchronization method further utilizes a two-dimensional to three-dimensional transformation algorithm. Given parameters from N cameras (1, 2, ..., N), the respective projection matrix H nWhen data is provided (n=1, 2, ..., N), corresponding tracked object features are obtained in two dimensions, with each camera providing two-dimensional features (t, q1), (t, q2), ..., (t, qN). Next, by applying multiple two-dimensional to three-dimensional DLTs, the three-dimensional position of the moving object can be obtained along with the estimation error.

[0097] In one embodiment, the point-by-point triangulation method is not used and is replaced by the implicit "overlap" of trajectories in parametric space, so the requirement of "hard synchronization" may be relaxed.

[0098] As one embodiment, given two cameras (camera "a" and camera "b") calibrated according to the two-step calibration method described herein, the acquired correction transformation matrix H' is used to determine the two-dimensional feature points q for camera "a". ax and q ay Regarding camera "b", the two-dimensional feature point q bx and q by The following was calculated. Subsequently, these two-dimensional image points were converted to corresponding three-dimensional real-world points. Therefore, measurements using the calibrated camera disclosed herein do not require separation of external and internal parameters, and do not require consideration of focal length, principal axis, lens tilt, or stereo base distance in processing, thus accelerating the measurement of acquiring three-dimensional points from two-dimensional images.

[0099] Software synchronization In accordance with further embodiments herein, Figure 7 shows an example of application of the present disclosure to perform measurements for tracking parameters of a moving object in, for example, a “soft synchronization” scheme 700. In some embodiments, the soft synchronization scheme uses an asynchronous camera calibrated according to the method herein. In some embodiments, the camera is calibrated without decomposing the camera parameters into internal and external camera parameters. In some embodiments, the soft synchronization scheme utilizes a non-superimposed FOV camera. In some embodiments, the soft synchronization scheme further utilizes a two-dimensional to three-dimensional transformation algorithm. Given parameters from N cameras (1, 2, ...N), the respective projection matrix H n When a data set is provided (n=1, 2, ...N), corresponding tracked object features are obtained in two dimensions, with each camera providing two-dimensional features; for example, camera 1 provides {t[1,n], × X[1,n], × Y[1,n]}), camera 2 provides {t[2,n] × X[2,n], × Y[2,n]}, ... camera m provides {t[m,n] × X[m,n], × Y[m,n]}). Next, applying a 12p (12-point) trajectory model calculates trajectory parameters with estimation errors, from which the three-dimensional position of the moving object is obtained, along with the corresponding estimation errors.

[0100] For soft-synchronized object tracking and measurement, the camera system in Figure 1 includes cameras, for example, one or more cameras 1001, 1002, ..., 100 nHowever, the system is configured to capture image frames asynchronously. Nevertheless, the processing uses the same common time reference by clock. In such embodiments, each camera is periodically synchronized by the High Precision Time Protocol (PTP) and / or the Global Positioning System (GPS). Each camera captures frames in free-run mode by its own interval VSync, so that although the timestamps of the frames are different, the reference time is identical within the tolerance defined or required by the measurement accuracy. In some embodiments, with a ball velocity of 50 m / s and a distance measurement of 2.5 cm, the time synchronization tolerance is approximately 0.5 milliseconds (ms). In one exemplary embodiment, with a time tolerance of 0.5 ms and a clock oscillator accuracy of 10 points per million (ppm), the cameras need to be synchronized every 50 seconds.

[0101] Figure 8A conceptually illustrates an implicit three-dimensional representation 800 for a given trajectory model assuming soft synchronization. In this embodiment, the asynchronous ("soft-synchronized") ball trajectory parameters of multiple cameras are reconstructed by trajectory stitching in parametric space. As seen in Figure 8A, camera 100 A and 100 B Parameters are recovered from the data, and the trajectory is scaled until it is stitched together by shape and time. This scaling is shown as being for obtaining the trajectory of an object along path 805 and measurements (e.g., the three-dimensional position of the ball over time). However, the "soft synchronization" process is performed on camera 100 along path 805A. A Frame only, camera 100 along route 805B B The frame is used to obtain the trajectory parameters. In some embodiments, paths 805A and 805B may overlap. In some embodiments, paths 805A and 805B may not overlap in the camera view.

[0102] Figure 8B illustrates an example of object position errors resulting from the improper application of triangulation ("hard synchronization") to "soft synchronization" data. As seen in Figure 8B, frames are asynchronously captured by a free-running camera (using an internal VSync signal) without strictly matching timestamps, using the same time reference (PTP, GPS, power grid). This method selects the nearest frame in pairs to provide ground truth object positions, e.g., ball position 810. This method is used, for example, by camera 100. A , 100 B The ball's position is estimated, such as a first position 815A based on the results of triangulation applied to two adjacent frames captured by the camera, or a second position 815B based on triangulation. In one example implementation, the nearest camera image frame is selected, where the frame rate is 250 fps (2 milliseconds) and the ball velocity is 50 m / s (or 200 fps and the ball velocity is 40 m / s), which results in an object position estimation error ("shift").

[0103] This software synchronization method includes features such as 9p or 12p trajectory models, and, in some cases, non-superimposed views (e.g., the use of two, three, or more cameras). Furthermore, such object tracking methods may be referred to as non-superimposed stereo, multi-camera stereo, or (implicit) trajectory stitching. This software synchronization method can be applied to sports such as cricket, baseball, and football.

[0104] In one embodiment, "triangulation" is applied to provide a three-dimensional reconstruction of synchronization frames captured by an asynchronous (soft-synchronized) camera with a common time reference and a common VSync signal.

[0105] For all cameras, their trajectories are physically identical. In this embodiment, it is further assumed that all cameras share a common time reference and provide appropriate timestamps even if the frames are asynchronous.

[0106] In one embodiment, the requirement regarding overlapping camera views can be relaxed by applying "overlap" in parametric space. Two cameras may look in opposite directions and observe different portions of the same trajectory. In some embodiments, a sufficient base distance between cameras may be required.

[0107] Therefore, the requirement that the camera has a close focal length can be relaxed. In some embodiments, the focal lengths of the cameras vary greatly, and the overlap of the fields of view (FOV) may be small or nonexistent.

[0108] No synchronization For object tracking and measurement without synchronization, the camera system in Figure 1 consists of cameras, for example, one or more cameras 1001, 1002, ..., 100 n However, it is configured to run asynchronously using its own time reference with its own clock. In some embodiments, in the absence of synchronization, clock synchronization can be restored by post-processing using common events, such as video, audio, vibration, or radar (i.e., a conversion from “unsynchronized” to “soft synchronized”). Video events may include, but are not limited to, the ball bouncing and IR flashes. Audio and / or vibration events may include, but are not limited to, the sound of the ball hitting and bouncing. Radar events may include, but are not limited to, the ball being released and / or bouncing. In some embodiments, clock synchronization can be restored by the ball in flight. In such embodiments, certain configurations of camera positioning may be required, such as an “in-plane” configuration where the cameras are positioned side by side.

[0109] In one exemplary embodiment, a camera system having two cameras, namely camera A and camera B, is configured to determine the position of a moving object. Camera A is configured to capture first and third images at first and third time points, respectively. Camera B is configured to capture a second image at second time point. The third time point is after the first time point, and the first time point is after the second time point. In another exemplary embodiment, the second time point is after the third time point, and the third time point is after the first time point. Furthermore, in yet another exemplary embodiment, the third time point is after the second time point, and the second time point is after the first time point. In some embodiments, cameras A and B are positioned at different locations.

[0110] While specific embodiments of this disclosure have been described above, various modifications can be implemented to these specific embodiments without departing from the scope of this disclosure. However, it will be apparent to those skilled in the art that various modifications and alterations are possible without departing from the nature and essence of this disclosure. Therefore, the embodiments of this disclosure are intended to illustrate the technical idea of ​​this disclosure, and do not limit it, nor impose any limitations on the scope of the technical idea of ​​this disclosure. Accordingly, the scope of this disclosure is defined by the following claims. All equivalent technical ideas shall be construed as being within the scope of this disclosure.

Claims

1. A method for tracking a moving object, Capturing one or more image frames of a moving object from each of one or more calibrated cameras, wherein each of the one or more calibrated cameras is calibrated according to a calibration method that generates and uses a transformation matrix for mapping features of a three-dimensional (3D) real-world model to corresponding two-dimensional (2D) image features. Using a hardware processor, determine the motion characteristics of the moving object based on one or more captured image frames from each of the one or more calibrated cameras, wherein the determination of the motion characteristics is based on implicit internal camera parameters and implicit external camera parameters of the respective transformation matrices from each of the one or more calibrated cameras. A method characterized by including

2. The method according to claim 1, wherein the calibration method includes calibrating each of the one or more calibrated cameras without decomposing the plurality of camera parameters into explicit external and explicit internal camera parameters.

3. The method according to claim 2, wherein each of the one or more calibrated cameras is further calibrated by changing one or more implicit external camera parameters to obtain explicit external camera parameters, the changes comprising adjusting the implicit external camera parameters through affine correction.

4. The method according to claim 3, wherein capturing one or more image frames with a calibrated camera includes controlling the timing of a first calibrated camera and a second calibrated camera physically separated from the first calibrated camera so that each of the first and second calibrated cameras captures one or more image frames of the moving object.

5. The method according to claim 4, wherein controlling the timing of the first calibrated camera and the second calibrated camera includes hard synchronizing the timing of the first calibrated camera and the second calibrated camera so that each captures an image frame in the same time instance, and the first calibrated camera and the second calibrated camera operate according to the same internal clock reference.

6. The method according to claim 4, wherein the control of the timing of the first calibrated camera and the second calibrated camera includes soft-synchronizing the timing of the first calibrated camera and the second calibrated camera so that each captures an image frame in a different time instance, each calibrated camera captures a frame of the position of the moving object in free-run mode, and each calibrated camera has its own internal synchronization.

7. The captured camera frames are stitched together into a parametric orbital space to obtain the position of the moving object. The method according to claim 4, further comprising:

8. It is an object tracking system, A camera system comprising one or more calibrated cameras, each camera capturing one or more image frames of the position of a moving object, and each of the one or more calibrated cameras being calibrated according to a calibration method that generates and uses respective transformation matrices for mapping features of a three-dimensional real-world model to corresponding two-dimensional image features, An object tracking system comprising a hardware processor coupled to a memory for storing instructions, wherein, when executed by the processor, the instructions are configured to perform the following actions: determine the motion characteristics of an object based on one or more captured image frames, wherein the determination of the motion characteristics of the object is based on implicit internal camera parameters and implicit external camera parameters of the respective transformation matrices from each of the one or more calibrated cameras.

9. The system according to claim 8, wherein the calibration method includes calibrating each of the one or more calibrated cameras without decomposing the plurality of camera parameters into explicit external and explicit internal camera parameters.

10. The system according to claim 9, wherein in each of the calibrated cameras, the hardware processor is further configured to change one or more implicit external camera parameters by adjusting the implicit external camera parameters of the calibrated camera through affine correction.

11. The system according to claim 10, wherein the hardware processor is further configured to control the timing of the first and second calibrated cameras so that each of the first and second calibrated cameras captures the one or more image frames of the moving object.

12. The system according to claim 11, wherein, in order to control the timing of the first calibrated camera and the second calibrated camera, the hardware processor is further configured to hard synchronize the timing of the first calibrated camera and the second calibrated camera so that each of the first and second calibrated cameras synchronously captures image frames in the same time instance, and the first calibrated camera and the second calibrated camera operate according to the same internal clock reference.

13. The system according to claim 11, wherein, in order to control the timing of the first calibrated camera and the second calibrated camera, the hardware processor is further configured to soft-synchronize the timing of the first calibrated camera and the second calibrated camera so that each of the first and second calibrated cameras asynchronously captures image frames in different time instances, each calibrated camera captures frames of the position of the moving object in free-run mode, and each calibrated camera has its own internal synchronization.

14. The aforementioned hardware processor further, The system according to claim 11, configured to stitch together one or more captured image frames of the moving object into parametric space to obtain the position of the moving object.

15. A method for calibrating a camera, To provide a transformation matrix (H) that represents multiple camera parameters, One or more corrections (H) for the aforementioned multiple camera parameters Δ A method comprising aligning a two-dimensional image feature (q) with a reference two-dimensional image feature (q') by applying a transformation matrix (H') to obtain an updated transformation matrix (H'), wherein the two-dimensional image feature (q) and the reference two-dimensional image feature (q') are represented in pixel coordinates.

16. The method according to claim 15, wherein the plurality of camera parameters include implicit external and implicit internal camera parameters.

17. The method according to claim 15, wherein the camera is calibrated without decomposing the plurality of implicit camera parameters into explicit external and explicit internal camera parameters.

18. The method according to claim 17, further comprising modifying one or more implicit external camera parameters, wherein the modification includes adjusting the implicit external parameters through affine correction, and the affine correction restores six degrees of freedom (DoF) representing the explicit external camera parameters.

19. The transformation matrix (H) is the pixel coordinate (q) related to the reference camera image object within the two-dimensional camera field of view of the camera. ｋ Initial transformation matrix (H) for transforming ) into corresponding known real-world position coordinates in a three-dimensional global reference space. ｋ The above method is further, In the aforementioned three-dimensional global reference space, the calibration image is aligned to the reference camera image object to obtain implicit camera parameters, including implicit external and implicit internal camera parameters. Based on the alignment of the calibration image to the reference camera image object, the updated transformation matrix (H ｋ＋１ ) to construct and one or more corrections (H Δ ) to the initial transformation matrix (H ｋ This includes applying to the updated transformation matrix (H ｋ＋１ The method according to claim 17, wherein the method includes explicit external camera parameters.

20. Apply the updated transformation matrix (H ｋ＋１ ) to the three-dimensional model to obtain the pixel coordinates (q ｋ＋１ ) related to the reference camera image object, and The aforementioned pixel coordinates (q ｋ＋１ ) the transformed pixel coordinates (q) evaluated for the reference camera image object ｎ ) and comparing them, The acquired pixel coordinates (q ｋ＋１ ) and the converted pixel coordinates (q ｋ The process involves determining whether the difference between the two values ​​exceeds a threshold, and The acquired pixel coordinates (q ｋ＋１ ) and the converted pixel coordinates (q ｋ In response to the determination that the difference between ) exceeds the threshold, the calibration image is aligned, one or more corrections are constructed, and one or more corrections (H Δ ) to the initial transformation matrix (H ｋ Applying this to the transformation matrix (H ｋ＋１ ) to further update the above, The method according to claim 19, further comprising:

21. The method according to claim 20, wherein the threshold is a distance corresponding to a predetermined number of pixel coordinates.

22. The aforementioned initial transformation matrix ( ｋ The method according to claim 19, wherein the parameters are obtained by performing a discrete linear transformation (DLT) to restore 11 DoFs representing implicit external and implicit internal camera parameters.

Images