Camera calibration method, device and equipment based on concentric circular ring calibration plate, and medium

Abstract

The invention discloses a camera calibration method and device based on a concentric circular ring calibration plate, equipment and a medium. The concentric circular ring calibration plate comprises a plurality of concentric circular rings, each concentric circular ring comprises an inner diameter and an outer diameter, the camera calibration method comprises the following steps: obtaining a plurality of concentric circular ring calibration plate images, the concentric circular ring calibration plate images comprise a plurality of elliptical ring images, the elliptical ring images comprise an inner diameter image and an outer diameter image, and the elliptical ring images are in one-to-one correspondence with the concentric circular rings; determining a first center of the inner diameter image and a second center of the outer diameter image according to the elliptical ring image; determining the pixel position of the third center of the concentric ring on the pixel coordinate system according to the positions of the first center and the second center on the pixel coordinate system; and determining internal parameters of the camera according to the pixel position and the position of the third center on the world coordinate system, wherein the internal parameters of the camera comprise a focal length and a principal point coordinate. By adopting the camera calibration method, the camera calibration precision can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of camera calibration, and in particular to a camera calibration method, device, equipment and medium based on a concentric ring calibration plate. Background Art

[0002] Three-dimensional vision measurement technology is an advanced high-end technology that integrates computer vision and precision measurement. It has the advantages of high precision, non-contact, high quality and high resolution, and is one of the most active research fields in three-dimensional measurement. The most important step in the measurement process of this system is camera calibration. The commonly used calibration objects include one-dimensional calibration objects, two-dimensional calibration objects and three-dimensional calibration objects. Among them, one-dimensional calibration objects are less used because of their relatively poor calibration accuracy and incomplete calibration parameters; three-dimensional calibration objects have high calibration accuracy and can be used for precise industrial measurement scenes, but the production cost is high, the calibration process is relatively cumbersome, and it is not suitable for calibration in small fields of view; two-dimensional calibration objects avoid the disadvantages of high requirements for calibration objects and complex calibration process, and at the same time can achieve higher accuracy in the calibration process, so they are most widely used in practice. Plane checkerboards and concentric rings are the two most common types of calibration objects in two-dimensional calibration objects.

[0003] Since there is a certain transition zone between the black and white grid edges of the calibration board when calibrating the chessboard, it is not possible to directly and instantly transition from white to black. Therefore, compared with camera calibration based on the chessboard, camera calibration based on concentric rings has higher accuracy. Among them, the center of the ring is a very important feature point, and its positioning accuracy directly affects the measurement accuracy of the system. However, for concentric rings, according to the principles of pinhole imaging and perspective projection transformation, the concentric rings in the plane appear as two ellipses with separated centers in the image after being captured by the camera. Therefore, the existing method of directly extracting the center of the ellipse as the projection point of the center of the concentric rings will affect the accuracy of camera calibration. Summary of the invention

[0004] The present invention provides a camera calibration method, device, equipment and medium based on a concentric ring calibration plate to improve the accuracy of camera calibration.

[0005] In a first aspect, an embodiment of the present invention provides a camera calibration method based on a concentric ring calibration plate, wherein the concentric ring calibration plate includes a plurality of concentric rings, each of which includes an inner diameter and an outer diameter. The camera calibration method includes:

[0006] Acquire a plurality of concentric circular ring calibration plate images, wherein the concentric circular ring calibration plate images include a plurality of elliptical ring images, wherein the elliptical ring images include inner diameter images and outer diameter images, and the elliptical ring images correspond one-to-one to the concentric circular rings;

[0007] Determine a first center of the inner diameter image and a second center of the outer diameter image according to the elliptical ring image;

[0008] Determine the pixel position of the third center of the concentric rings in the pixel coordinate system according to the positions of the first center and the second center in the pixel coordinate system;

[0009] The camera internal parameters are determined according to the pixel position and the position of the third center in the world coordinate system, where the camera internal parameters include focal length and principal point coordinates.

[0010] Optionally, determining the first center of the inner diameter image and the second center of the outer diameter image according to the elliptical ring image includes:

[0011] Acquire a first edge of the inner diameter image and a second edge of the outer diameter image by Gaussian fitting;

[0012] The first center is determined according to the first edge, and the second center is determined according to the second edge.

[0013] Optionally, acquiring the first edge of the inner diameter image and the second edge of the outer diameter image by Gaussian fitting includes:

[0014] The first edge of the inner diameter image and the second edge of the outer diameter image are determined according to the following corresponding relationship:

[0015]

[0016] x represents the coordinate point in the direction of the edge gradient, y represents the first-order derivative of the grayscale value, μ represents the mean of the Gaussian function, σ represents the standard deviation of the Gaussian function, and k represents the amplitude of the Gaussian function.

[0017] Optionally, determining the first center according to the first edge includes:

[0018] Determine the center of the quasi-circle corresponding to the inner diameter image according to the first edge of the inner diameter image and the quasi-circular property of the ellipse;

[0019] Determining the first center of the inner diameter image according to the center of the quasi-circle corresponding to the inner diameter image;

[0020] Determining the second center according to the second edge includes:

[0021] Determine the center of the quasi-circle corresponding to the outer diameter image according to the second edge of the outer diameter image and the quasi-circular property of the ellipse;

[0022] The center of the quasi-circle corresponding to the outer diameter image determines the second center of the outer diameter image.

[0023] Optionally, determining the pixel position of the third center of the concentric rings in the pixel coordinate system according to the positions of the first center and the second center in the pixel coordinate system includes:

[0024] The pixel position of the third center in the pixel coordinate system is determined according to the positions of the first center and the second center in the pixel coordinate system and the following corresponding relationship:

[0025]

[0026] Among them, (u c , v c ) represents the pixel position of the third center in the pixel coordinate system, represents the position of the first center in the pixel coordinate system, represents the position of the second center in the pixel coordinate system, r 1 represents the radius of the inner diameter, r 2 Indicates the radius of the outer diameter.

[0027] Optionally, determining the camera internal parameters according to the pixel position and the position of the third center in the world coordinate system includes:

[0028] The camera internal parameters are determined according to the pixel position, the position of the third center in the world coordinate system, and the following corresponding relationship:

[0029]

[0030] Among them, s represents the preset scale ratio, M represents the camera intrinsic parameter matrix, and R 1 , R 2 , R 3 represents the rotation vector, t represents the translation vector, (u c , v c ) represents the pixel position, (x wc ,y wc , z wc ) represents the position of the third center in the world coordinate system.

[0031] Optionally, acquiring the first edge of the inner diameter image and the second edge of the outer diameter image by Gaussian fitting includes:

[0032] Performing grayscale processing on the concentric ring calibration plate image to obtain a grayscale image;

[0033] Performing noise filtering on the grayscale image to obtain a filtered image;

[0034] The first edge of the inner diameter image and the second edge of the outer diameter image in the filtered image are acquired through Gaussian fitting.

[0035] In a second aspect, an embodiment of the present invention provides a camera calibration device, which is used to perform the camera calibration method as described in the first aspect, and the camera calibration device includes:

[0036] An image acquisition unit, used to acquire a plurality of concentric circular ring calibration plate images, wherein the concentric circular ring calibration plate images include a plurality of elliptical ring images, wherein the elliptical ring images include an inner diameter image and an outer diameter image, and the elliptical ring images correspond one-to-one to the concentric circular rings;

[0037] An inner and outer diameter image center acquisition unit, used for determining a first center of the inner diameter image and a second center of the outer diameter image according to the elliptical ring image;

[0038] a concentric ring center pixel position determining unit, configured to determine a pixel position of a third center of the concentric rings in the pixel coordinate system according to the positions of the first center and the second center in the pixel coordinate system;

[0039] A camera internal parameter determination unit is used to determine the camera internal parameters according to the pixel position and the position of the third center in the world coordinate system, wherein the camera internal parameters include focal length and principal point coordinates.

[0040] In a third aspect, an embodiment of the present invention provides a camera calibration device, the camera calibration device comprising:

[0041] one or more processors;

[0042] a storage device for storing one or more programs,

[0043] When the one or more programs are executed by the one or more processors, the one or more processors implement the camera calibration method as described in the first aspect.

[0044] In a fourth aspect, an embodiment of the present invention provides a storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the camera calibration method as described in the first aspect.

[0045] Compared to directly selecting the position of the first center of the inner diameter image or the second center of the outer diameter image in the pixel coordinate system as the pixel position of the third center of the concentric rings in the pixel coordinate system, the technical solution of the embodiment of the present invention, by setting the pixel position of the third center of the concentric rings in the pixel coordinate system to be jointly determined according to the positions of the first center of the inner diameter image and the second center of the outer diameter image in the pixel coordinate system, can improve the center point recognition accuracy of the concentric rings, thereby helping to improve the accuracy of camera calibration.

[0046] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0048] Figure 1 A schematic structural diagram of a concentric ring calibration plate provided in an embodiment of the present invention;

[0049] Figure 2 A flowchart of a camera calibration method based on a concentric ring calibration plate provided in an embodiment of the present invention;

[0050] Figure 3 15 images of concentric ring calibration plates at different positions provided in an embodiment of the present invention;

[0051] Figure 4 A flowchart of another camera calibration method based on a concentric ring calibration plate provided in an embodiment of the present invention;

[0052] Figure 5 Schematic diagram of an ellipse circumscribed rectangle in different coordinate systems provided by an embodiment of the present invention;

[0053] Figure 6 A schematic diagram of an imaging principle provided by an embodiment of the present invention;

[0054] Figure 7 A schematic diagram of the relationship between the world coordinate system, camera coordinate system, image physical coordinate system, and pixel coordinate system provided in an embodiment of the present invention;

[0055] Figure 8 A schematic diagram of a camera calibration result provided by an embodiment of the present invention;

[0056] Fig. 9 A flowchart of another camera calibration method based on a concentric ring calibration plate provided in an embodiment of the present invention;

[0057] Fig.10 A schematic diagram of the structure of a camera calibration device provided by an embodiment of the present invention;

[0058] Fig.11 A schematic diagram of the structure of a camera calibration device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0059] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0060] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices. The orientation or positional relationship indicated by the terms "upper", "lower", "left", "right", etc. is based on the orientation or positional relationship shown in the drawings, and is only used to illustrate the relative positional relationship between the components or components, and does not particularly limit the specific installation orientation of the components or components.

[0061] Figure 1 A schematic diagram of the structure of a concentric ring calibration plate provided in an embodiment of the present invention, referring to Figure 1 The concentric ring calibration plate 10 in the embodiment of the present invention includes a plurality of concentric rings 11 , and the concentric rings 11 include an inner diameter 111 and an outer diameter 112 . Figure 1 The concentric circular ring calibration plate shown includes 63 concentric circular rings 11 arranged in an array of 7 rows and 9 columns.

[0062] Figure 2 A flowchart of a camera calibration method based on a concentric ring calibration plate provided in an embodiment of the present invention. The camera calibration method based on a concentric ring calibration plate in an embodiment of the present invention is applicable to situations where camera calibration is required. The camera calibration method based on a concentric ring calibration plate can be performed by a camera calibration device based on a concentric ring calibration plate. The device can be implemented in software and / or hardware and is specifically configured in a camera calibration device. Reference Figure 2 The camera calibration method based on the concentric ring calibration plate in the embodiment of the present invention includes:

[0063] S110, acquiring a plurality of concentric circular ring calibration plate images, wherein the concentric circular ring calibration plate images include a plurality of elliptical ring images, wherein the elliptical ring images include inner diameter images and outer diameter images, and the elliptical ring images correspond one to one to the concentric circular rings.

[0064] Figure 3 15 images of concentric ring calibration plates at different positions provided in the embodiment of the present invention, Figure 3 Each concentric ring calibration plate image shown includes 63 elliptical ring images arranged in an array of 7 rows and 9 columns. Figure 1 The 63 concentric circular rings 11 shown correspond one to one, wherein the inner diameter image of the elliptical ring image is the projection of the inner diameter 111 , and the outer diameter image of the elliptical ring image is the projection of the outer diameter 112 .

[0065] S120 , determining a first center of the inner diameter image and a second center of the outer diameter image according to the elliptical ring image.

[0066] It should be noted that the inner diameter image in the elliptical ring image can be regarded as an inner ellipse, and the first center of the inner diameter image refers to the center of the quasi-circle corresponding to the inner ellipse. The outer diameter image in the elliptical ring image can be regarded as an outer ellipse, and the second center of the outer diameter image refers to the center of the quasi-circle corresponding to the inner ellipse.

[0067] S130, determining the pixel position of the third center of the concentric rings in the pixel coordinate system according to the positions of the first center and the second center in the pixel coordinate system.

[0068] It should be noted that the positions of the first center of the inner diameter image and the second center of the outer diameter image in the pixel coordinate system are different, and both are different from the pixel position of the third center of the concentric ring 11 in the pixel coordinate system, but the pixel position of the third center of the concentric ring 11 in the pixel coordinate system can be determined based on the positions of the first center and the second center in the pixel coordinate system.

[0069] S140, determining camera internal parameters according to the pixel position and the position of the third center in the world coordinate system, where the camera internal parameters include focal length and principal point coordinates.

[0070] Exemplarily, the focal length and the principal point coordinates may be determined according to the pixel position, the position of the third center in the world coordinate system, and the conversion formula between the world coordinate system and the pixel coordinate system, thereby achieving camera calibration.

[0071] Compared with directly selecting the position of the first center of the inner diameter image or the second center of the outer diameter image in the pixel coordinate system as the pixel position of the third center of the concentric ring 11 in the pixel coordinate system, the technical solution of the embodiment of the present invention can improve the center recognition accuracy of the concentric ring 11 by setting the position of the first center of the inner diameter image and the second center of the outer diameter image in the pixel coordinate system to jointly determine the pixel position of the third center of the concentric ring 11 in the pixel coordinate system, which is beneficial to improving the accuracy of camera calibration.

[0072] Figure 4 A flowchart of another camera calibration method based on a concentric ring calibration plate provided in an embodiment of the present invention, Figure 4 The embodiment shown in the figure describes in detail how to determine the first center of the inner diameter image and the second center of the outer diameter image according to the elliptical ring image. Figure 4 The camera calibration method based on the concentric ring calibration plate in the embodiment of the present invention includes:

[0073] S210, acquiring a plurality of concentric circular ring calibration plate images, wherein the concentric circular ring calibration plate images include a plurality of elliptical ring images, wherein the elliptical ring images include inner diameter images and outer diameter images, and the elliptical ring images correspond one to one to the concentric circular rings.

[0074] S220 , obtaining a first edge of the inner diameter image and a second edge of the outer diameter image through Gaussian fitting.

[0075] As a feasible implementation manner, obtaining the first edge of the inner diameter image and the second edge of the outer diameter image by Gaussian fitting includes:

[0076] The first edge of the inner diameter image and the second edge of the outer diameter image are determined according to the following corresponding relationship:

[0077]

[0078] x represents the coordinate point in the direction of the edge gradient, y represents the first-order derivative of the grayscale value, μ represents the mean of the Gaussian function, σ represents the standard deviation of the Gaussian function, and k represents the amplitude of the Gaussian function.

[0079] It can be understood that the positioning accuracy of the first edge of the inner diameter image and the second edge of the outer diameter image can directly affect the extraction accuracy of the first center of the inner diameter image and the second center of the outer diameter image, thereby affecting the accuracy of the pixel position of the third center of the concentric ring 11 in the pixel coordinate system. In order to improve the positioning accuracy of the first edge and the second edge, the extracted edge points can be sub-pixel edge positioned. After verification by simulation experiments, a Gaussian fitting method with good noise resistance and positioning accuracy can be selected for sub-pixel edge positioning. In the gradient direction of the edge of the image, the first-order derivative of the grayscale distribution is approximately a Gaussian distribution, and the center of the Gaussian distribution is the position with the highest edge point accuracy. Combined with the above formula, it can be seen that the sub-pixel position of the edge can be obtained by obtaining the position parameter μ. Specifically, each edge is first fitted, and the normal direction of the fitted edge is used to represent the gradient direction of the edge, and then the grayscale difference is taken along the normal direction to fit the Gaussian model to obtain the sub-pixel edge position.

[0080] S230: Determine a first center according to the first edge, and determine a second center according to the second edge.

[0081] As a feasible implementation, determining the first center based on the first edge includes: determining the center of the quasi-circle corresponding to the inner diameter image based on the first edge of the inner diameter image and the quasi-circular property of the ellipse; determining the first center of the inner diameter image based on the center of the quasi-circle corresponding to the inner diameter image.

[0082] Determining the second center according to the second edge includes: determining the center of the quasi-circle corresponding to the outer diameter image according to the second edge of the outer diameter image and the quasi-circular property of the ellipse; and determining the second center of the outer diameter image from the center of the quasi-circle corresponding to the outer diameter image.

[0083] It should be noted that the quasi-circular property of an ellipse refers to the intersection of any two mutually perpendicular tangents to the edge of the ellipse with the center of the ellipse as the center. The circle with a radius of , is called the quasi-circle of the ellipse, and the center of the quasi-circle is the center of the ellipse. The circumscribed rectangle of the ellipse is constructed based on the maximum and minimum values ​​of the X and Y directions in the coordinate system in different directions. The vertices of the circumscribed rectangle can be regarded as the intersection points of the edge tangents with a vertical relationship, so as to fit the quasi-circle of the ellipse based on the vertices of the circumscribed rectangle. Figure 5 Schematic diagram of ellipse circumscribed rectangle in different coordinate systems provided by the embodiment of the present invention, such as Figure 5As shown in the figure, let the original coordinate system be (X0, Y0), O is the origin of the coordinate system, for the target ellipse in this coordinate system, the maximum and minimum values ​​in the X and Y directions can be obtained, and the four vertices A0, B0, C0 and D0 of the circumscribed rectangle (shown by the black dotted frame) can be constructed. The vertices of the rectangle are the intersection points of the mutually perpendicular edge tangents. The coordinate system is rotated counterclockwise by an angle a to obtain the coordinate system (X1, Y1) in the new posture. Similarly, a new circumscribed rectangle (shown by another black dotted frame) can be constructed, and the four vertices A1, B1, C1 and D1 can be determined. The coordinate system is rotated several times in a fixed direction and angle to obtain a set of circumscribed rectangles. The quasi-circle of the ellipse can be fitted from the vertices of these circumscribed rectangles, and then the center of the ellipse can be determined. In general, 90 posture angles are selected equidistantly in the range of 0° to 90° to complete the accurate fitting of the quasi-circle.

[0084] S240, determining the pixel position of the third center of the concentric rings in the pixel coordinate system according to the positions of the first center and the second center in the pixel coordinate system.

[0085] As a feasible implementation manner, determining the pixel position of the third center of the concentric ring 11 in the pixel coordinate system according to the positions of the first center and the second center in the pixel coordinate system includes:

[0086] The pixel position of the third center in the pixel coordinate system is determined according to the positions of the first center and the second center in the pixel coordinate system and the following corresponding relationship:

[0087]

[0088] Among them, (u c , v c ) represents the pixel position of the third center in the pixel coordinate system, represents the position of the first center in the pixel coordinate system, represents the position of the second center in the pixel coordinate system, r 1 Indicates the radius of the inner diameter, r 2 Indicates the radius of the outer diameter.

[0089] For example, we only need to know the ratio of the outer radius to the inner radius r 2 / r 1 (or the ratio of the inner radius to the outer radius r 1 / r 2 ), the position of the second center in the pixel coordinate system and the position of the first center in the pixel coordinate system, and substituting them into the above formula can obtain the pixel position of the third center of the concentric ring 11 in the pixel coordinate system.

[0090] Figure 6A schematic diagram of an imaging principle provided by an embodiment of the present invention, by calculating the coordinates (u C' , v C' ) and the coordinates (u ) of the center B' of the inner diameter image or the outer diameter image in the elliptical ring image corresponding to the concentric ring 11 on the pixel coordinate system O-uv B' , v B ) to obtain the eccentric deviation (ε u , ε v ), as shown in the following formula:

[0091]

[0092] It should be noted that C is the focal length of the camera lens; x p It is the x-coordinate of the third center of the concentric rings 11 in the world coordinate system O-xyz, ω represents the angle between the normal z-axis of the concentric rings 11 and the camera optical axis Z' axis, α represents the angle between the z-axis and the straight line OO', d represents the distance between OO', l is the projection distance of OO' in the direction of the optical axis, and r represents the inner radius or outer radius of the concentric rings 11.

[0093] In the actual camera imaging model, l>>r and (sinω) 2 <1, so the above formula can be simplified to:

[0094]

[0095] in,

[0096] First center of the inner diameter image The pixel position (u c , v c ) of the eccentric deviation (ε u1 , ε v1 ) satisfies the following corresponding relationship:

[0097]

[0098] Second center of outer diameter image The pixel position (u c , v c ) of the eccentric deviation (ε u2 , ε v2 ) satisfies the following corresponding relationship:

[0099]

[0100] The above correspondence of pixel positions can be written as:

[0101]

[0102] S250: Determine camera internal parameters according to the pixel position and the position of the third center in the world coordinate system, where the camera internal parameters include focal length and principal point coordinates.

[0103] As a feasible implementation, the camera internal parameters are determined according to the pixel position and the position of the third center in the world coordinate system, including:

[0104] The camera internal parameters are determined based on the pixel position, the position of the third center in the world coordinate system, and the following corresponding relationship:

[0105]

[0106] Among them, s represents the preset scale ratio, M represents the camera intrinsic parameter matrix, and R 1 , R 2 , R 3 represents the rotation vector, t represents the translation vector, (u c , v c ) represents the pixel position of the third center of the concentric ring 11 in the pixel coordinate system, and the position of the third center in the world coordinate system (x wc ,y wc , z wc It should be noted that since the Z axis of the world coordinate system is usually set to be perpendicular to the concentric ring calibration plate, wc Can be set directly to 0.

[0107] Figure 7 A schematic diagram of the relationship between the world coordinate system, the camera coordinate system, the image physical coordinate system, and the pixel coordinate system provided in the embodiment of the present invention, with reference to Figure 7 , the world coordinate system is a coordinate system that describes the three-dimensional real scene. The world coordinate system can be expressed as w -X w Y w Z w The direction of the coordinate axis can be arbitrarily selected, and any point can be expressed as (x w ,y w , z w ).

[0108] The camera coordinate system is based on the optical center O of the camera. c is the coordinate origin, Z c The axis is perpendicular to the imaging plane and points to the object being measured. c The axis is parallel to the x-axis of the image's physical coordinate system, and the Y c The axis is parallel to the y-axis of the image physical coordinate system and can be expressed as O c -Xc Y c Z c The focal length f of the camera is the origin of the camera coordinate system O c The distance from the origin O of the image's physical coordinate system.

[0109] The origin of the image physical coordinate system is the intersection of the imaging plane and the optical axis. xy , the x-axis and y-axis are the horizontal and vertical axes of the coordinates respectively, the x-axis is parallel to the u-axis of the image pixel coordinate system, and the y-axis is parallel to the v-axis of the image pixel coordinate system, which can be expressed as xy -xy. The unit of the image physical coordinate system is mm. Usually, the origin of the image physical coordinate system is O xy It is also called the principal point, and its coordinates in the pixel coordinate system can be recorded as (u 0 , v 0 ).

[0110] The imaging surface of the camera is represented in the form of a matrix, and the grayscale value of each pixel on the imaging surface is the value of each element in the matrix. Figure 7 As shown, the image pixel coordinate system is based on the upper left corner O of the imaging surface uv is the origin of the coordinate system, the u axis and the v axis are the horizontal axis and vertical axis of the coordinate system, respectively, which can be expressed as O uv -uv, the unit of the image pixel coordinate system is pixel.

[0111] refer to Figure 7 , the coordinates of any point P in the three-dimensional scene in the world coordinate system are P(x w ,y w , z w ), the coordinates in the camera coordinate system, image physical coordinate system and image pixel coordinate system are (x c ,y c , z c ), (x, y) and (u, v). The transformation relationship between the camera coordinate system and the world coordinate system can be expressed as:

[0112]

[0113] Where [R|T] represents the external parameters of the camera, which are determined by the spatial position of the camera relative to the world coordinate system. R is a 3×3 rotation matrix, and T is a 3×1 translation vector.

[0114] The transformation relationship between the image physical coordinate system and the camera coordinate system can be expressed as:

[0115]

[0116] Where f is the focal length of the camera.

[0117] The conversion relationship between the image physical coordinate system and the image pixel coordinate system can be expressed as:

[0118]

[0119] Among them, dx and dy represent the physical sizes of the unit pixel in the horizontal and vertical directions respectively.

[0120] Combining the above conversion relationships between the camera coordinate system and the world coordinate system, the conversion relationship between the image physical coordinate system and the camera coordinate system, and the image physical coordinate system and the image pixel coordinate system, the relationship between the world coordinates of a point in space and the pixel coordinates of its image point can be obtained as follows:

[0121]

[0122] Assume that the plane of the object to be measured (such as the concentric ring calibration plate) passes through the origin and is perpendicular to z w axis, so define z w = 0. Let t represent the 3×1 translation vector, and from the above formula we get:

[0123]

[0124] It should be noted that any point P(x w ,y w , z w ) may be the position (x) of the third center of the concentric rings 11 in the world coordinate system. wc ,y wc , z wc ). M is the camera internal parameter matrix, including internal parameters: f x 、f y 、u 0 、v 0 , f x Indicates the focal length of the camera in the u-axis direction of the pixel coordinate system, f y Indicates the focal length of the camera in the v-axis direction of the pixel coordinate system, (u 0 , v 0 ) represents the coordinates of the principal point in the pixel coordinate system.

[0125] Figure 8 A schematic diagram of a camera calibration result provided by an embodiment of the present invention, Figure 8 It shows the distribution diagram of the camera back-reprojection error in the u-axis direction and the v-axis direction of the pixel coordinate system. It should be noted that the average back-reprojection error of the camera obtained by the above camera calibration method is (0.044, 0.041), that is, the average back-reprojection error in the u-axis direction of the pixel coordinate system is 0.044, and the average back-reprojection error in the v-axis direction of the pixel coordinate system is 0.041. Figure 8It can be seen from the back-projection error distribution diagram and the final average back-reprojection error that the camera calibration method adopted in the implementation of the present invention has high accuracy.

[0126] Fig. 9 A flowchart of another camera calibration method based on a concentric ring calibration plate provided in an embodiment of the present invention, Fig. 9 The embodiment shown in the figure illustrates how to obtain the first edge of the inner diameter image and the second edge of the outer diameter image by Gaussian fitting. Fig. 9 The camera calibration method based on the concentric ring calibration plate in the embodiment of the present invention includes:

[0127] S310, acquiring a plurality of concentric circular ring calibration plate images, wherein the concentric circular ring calibration plate images include a plurality of elliptical ring images, wherein the elliptical ring images include inner diameter images and outer diameter images, and the elliptical ring images correspond one to one to the concentric circular rings.

[0128] S320, performing grayscale processing on the concentric ring calibration plate image to obtain a grayscale image.

[0129] S330 , performing noise filtering on the grayscale image to obtain a filtered image.

[0130] Before performing Gaussian fitting, grayscale processing and noise filtering are performed on the concentric ring calibration plate image to eliminate the influence of image noise on step S340, which is beneficial to improving the accuracy of camera calibration.

[0131] S340 , obtaining a first edge of the inner diameter image and a second edge of the outer diameter image in the filtered image through Gaussian fitting.

[0132] S350: Determine a first center according to the first edge, and determine a second center according to the second edge.

[0133] S360, determining the pixel position of the third center of the concentric rings in the pixel coordinate system according to the positions of the first center and the second center in the pixel coordinate system.

[0134] S370, determining camera internal parameters according to the pixel position and the position of the third center in the world coordinate system, where the camera internal parameters include focal length and principal point coordinates.

[0135] Based on the same inventive concept, an embodiment of the present invention further provides a camera calibration device. Fig.10 A schematic diagram of the structure of a camera calibration device provided by an embodiment of the present invention, referring to Fig.10 , the camera calibration device in the embodiment of the present invention includes:

[0136] The image acquisition unit 410 is used to acquire multiple concentric ring calibration plate images, the concentric ring calibration plate images include multiple elliptical ring images, the elliptical ring images include inner diameter images and outer diameter images, and the elliptical ring images correspond to the concentric rings one by one.

[0137] The inner and outer diameter image center acquisition unit 420 is used to determine the first center of the inner diameter image and the second center of the outer diameter image according to the elliptical ring image.

[0138] The concentric ring center pixel position determining unit 430 is used to determine the pixel position of the third center of the concentric rings in the pixel coordinate system according to the positions of the first center and the second center in the pixel coordinate system.

[0139] The camera internal parameter determination unit 440 is used to determine the camera internal parameters according to the pixel position and the position of the third center in the world coordinate system. The camera internal parameters include focal length and principal point coordinates.

[0140] The camera calibration device provided in the embodiment of the present invention can execute the camera calibration method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0141] Fig.11 A schematic diagram of the structure of a camera calibration device 500 that can be used to implement an embodiment of the present invention is shown. The camera calibration device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The camera calibration device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0142] like Fig.11As shown, the camera calibration device 500 includes at least one processor 510, and a memory connected to the at least one processor 510, such as a read-only memory (ROM) 520, a random access memory (RAM) 530, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 510 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 520 or the computer program loaded from the storage unit 580 to the random access memory (RAM) 530. In the RAM 530, various programs and data required for the operation of the camera calibration device 500 can also be stored. The processor 510, the ROM 520, and the RAM 530 are connected to each other through a bus 540. An input / output (I / O) interface 550 is also connected to the bus 540.

[0143] Multiple components in the camera calibration device 500 are connected to the I / O interface 550, including: an input unit 560, such as a keyboard, a mouse, etc.; an output unit 570, such as various types of displays, speakers, etc.; a storage unit 580, such as a disk, an optical disk, etc.; and a communication unit 590, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 590 allows the camera calibration device 500 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0144] The processor 510 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 510 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 510 performs the various methods and processes described above, such as a camera calibration method.

[0145] In some embodiments, the camera calibration method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 580. In some embodiments, part or all of the computer program may be loaded and / or installed on the camera calibration device 500 via the ROM 520 and / or the communication unit 590. When the computer program is loaded into the RAM 530 and executed by the processor 510, one or more steps of the camera calibration method described above may be performed. Alternatively, in other embodiments, the processor 510 may be configured to execute the camera calibration method in any other appropriate manner (e.g., by means of firmware).

[0146] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0147] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0148] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0149] To provide interaction with a user, the systems and techniques described herein may be implemented on a camera calibration device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball), through which a user can provide input to the camera calibration device. Other types of devices may also be used to provide interaction with a user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0150] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0151] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0152] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0153] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

[0154] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A camera calibration method based on a concentric ring calibration plate, wherein the concentric ring calibration plate comprises a plurality of concentric rings, each of which comprises an inner diameter and an outer diameter, wherein: The camera calibration method comprises: Acquire a plurality of concentric circular ring calibration plate images, wherein the concentric circular ring calibration plate images include a plurality of elliptical ring images, wherein the elliptical ring images include inner diameter images and outer diameter images, and the elliptical ring images correspond one-to-one to the concentric circular rings; Determine a first center of the inner diameter image and a second center of the outer diameter image according to the elliptical ring image; Determine the pixel position of the third center of the concentric rings in the pixel coordinate system according to the positions of the first center and the second center in the pixel coordinate system; The camera internal parameters are determined according to the pixel position and the position of the third center in the world coordinate system, where the camera internal parameters include focal length and principal point coordinates.

2. The camera calibration method according to claim 1, characterized in that: Determining a first center of the inner diameter image and a second center of the outer diameter image according to the elliptical ring image includes: Acquire a first edge of the inner diameter image and a second edge of the outer diameter image by Gaussian fitting; The first center is determined according to the first edge, and the second center is determined according to the second edge.

3. The camera calibration method according to claim 2, characterized in that: Acquiring a first edge of the inner diameter image and a second edge of the outer diameter image by Gaussian fitting includes: The first edge of the inner diameter image and the second edge of the outer diameter image are determined according to the following corresponding relationship: x represents the coordinate point in the direction of the edge gradient, y represents the first-order derivative of the grayscale value, μ represents the mean of the Gaussian function, σ represents the standard deviation of the Gaussian function, and k represents the amplitude of the Gaussian function.

4. The camera calibration method according to claim 2, characterized in that: Determining the first center according to the first edge includes: Determine the center of the quasi-circle corresponding to the inner diameter image according to the first edge of the inner diameter image and the quasi-circular property of the ellipse; Determining the first center of the inner diameter image according to the center of the quasi-circle corresponding to the inner diameter image; Determining the second center according to the second edge includes: Determine the center of the quasi-circle corresponding to the outer diameter image according to the second edge of the outer diameter image and the quasi-circular property of the ellipse; The center of the quasi-circle corresponding to the outer diameter image determines the second center of the outer diameter image.

5. The camera calibration method according to claim 1, characterized in that: Determining the pixel position of the third center of the concentric rings in the pixel coordinate system according to the positions of the first center and the second center in the pixel coordinate system includes: The pixel position of the third center in the pixel coordinate system is determined according to the positions of the first center and the second center in the pixel coordinate system and the following corresponding relationship: in, ( u c , v c) represents the pixel position of the third center in the pixel coordinate system, represents the position of the first center in the pixel coordinate system, represents the position of the second center in the pixel coordinate system, r1 represents the radius of the inner diameter, and r2 represents the radius of the outer diameter.

6. The camera calibration method according to claim 1, characterized in that: Determining the camera internal parameters according to the pixel position and the position of the third center in the world coordinate system includes: The camera internal parameters are determined according to the pixel position, the position of the third center in the world coordinate system, and the following corresponding relationship: Among them, s represents the preset scale ratio, M represents the camera intrinsic parameter matrix, R1, R2, R3 represent the rotation vector, t represents the translation vector, (u c , v c ) represents the pixel position, (x wc ,y wc , z wc ) represents the position of the third center in the world coordinate system.

7. The camera calibration method according to claim 2, characterized in that: Acquiring a first edge of the inner diameter image and a second edge of the outer diameter image by Gaussian fitting includes: Performing grayscale processing on the concentric ring calibration plate image to obtain a grayscale image; Performing noise filtering on the grayscale image to obtain a filtered image; The first edge of the inner diameter image and the second edge of the outer diameter image in the filtered image are acquired through Gaussian fitting.

8. A camera calibration device, used to execute the camera calibration method according to any one of claims 1 to 7, characterized in that: The camera calibration device comprises: An image acquisition unit, used to acquire a plurality of concentric circular ring calibration plate images, wherein the concentric circular ring calibration plate images include a plurality of elliptical ring images, wherein the elliptical ring images include an inner diameter image and an outer diameter image, and the elliptical ring images correspond one-to-one to the concentric circular rings; An inner and outer diameter image center acquisition unit, used for determining a first center of the inner diameter image and a second center of the outer diameter image according to the elliptical ring image; a concentric ring center pixel position determining unit, configured to determine a pixel position of a third center of the concentric rings in the pixel coordinate system according to the positions of the first center and the second center in the pixel coordinate system; A camera internal parameter determination unit is used to determine the camera internal parameters according to the pixel position and the position of the third center in the world coordinate system, wherein the camera internal parameters include focal length and principal point coordinates.

9. A camera calibration device, characterized in that: The camera calibration device comprises: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the camera calibration method as described in any one of claims 1 to 7.

10. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the camera calibration method according to any one of claims 1 to 7 is implemented.

Images