A camera calibration method, system, electronic device and storage medium

Abstract

The application discloses a camera calibration method and system, electronic equipment and storage medium. The method comprises the following steps: controlling a camera to acquire image data of an identification feature, and determining first position information of the identification feature in a camera coordinate system according to the image data and first intrinsic parameter calibration data preset for the camera; determining second position information of the identification feature in a target coordinate system according to the first position information; determining second extrinsic parameter calibration data according to the first position information and the second position information; determining second intrinsic parameter calibration data according to the first intrinsic parameter calibration data preset for the camera, first extrinsic parameter calibration data preset for the camera and the second extrinsic parameter calibration data; and determining a calibration result of the camera based on the second extrinsic parameter calibration data and the second intrinsic parameter calibration data. In this way, the originally complex high-precision calibration can be quickly, accurately and simply completed, and the calibration efficiency is improved.

Description

Technical Field

[0001] This invention relates to the field of mechanical automation technology, and in particular to a camera calibration method, a camera calibration system, an electronic device, and a computer-readable storage medium. Background Technology

[0002] In image measurement and machine vision applications, it is necessary to determine the three-dimensional spatial position of a point on the surface of a spatial object and its corresponding point in the image. The correspondence can be expressed by the camera imaging model. The process of solving the parameters in the imaging model is called camera calibration.

[0003] In practice, the inventors of this application have found that current calibration technology research mainly focuses on indoor working environment calibration. For outdoor long-distance, large target measurement scenarios, there are mainly two methods: one is to make a large calibration board according to the actual environment and complete the camera calibration solution by collecting dozens of sets of calibration board images with different placement positions and postures. This method has high requirements for the calibration board and calibration operators, and the production of large calibration boards is difficult. During calibration, large calibration boards are inconvenient to transport or move and are prone to deformation, which can introduce errors or mistakes. The second method is to complete laboratory calibration in a laboratory environment according to the distance and angle of the engineering application environment, and then directly apply the calibration results to the engineering site. This method often results in the calibration accuracy being insufficient to meet the actual application requirements. In addition, there are also situations where the target is located in an environment where it is impossible to place a calibration board, thus making calibration impossible. Summary of the Invention

[0004] The main technical problem solved by this invention is to provide a camera calibration method, system, electronic device and storage medium that can quickly, accurately and simply complete the originally complex high-precision calibration, thereby improving calibration efficiency.

[0005] To solve the above-mentioned technical problems, one technical solution adopted by the present invention is to provide a camera calibration method, the method comprising:

[0006] The camera is controlled to acquire image data of the identification feature, and the first position information of the identification feature in the camera coordinate system is determined based on the image data and the first intrinsic parameter calibration data preset by the camera.

[0007] Based on the first location information, determine the second location information of the identification feature in the target coordinate system;

[0008] The second external parameter calibration data is determined based on the first location information and the second location information;

[0009] The second intrinsic parameter calibration data is determined based on the first preset intrinsic parameter calibration data, the first preset extrinsic parameter calibration data, and the second extrinsic parameter calibration data of the camera. The calibration result of the camera is then determined based on the second extrinsic parameter calibration data and the second intrinsic parameter calibration data.

[0010] Optionally, the control camera acquires image data of the identification feature, and determines the first position information of the identification feature in the camera coordinate system based on the image data and the camera's preset first intrinsic parameter calibration data, including:

[0011] Control the camera to acquire image data of identifying features;

[0012] The pixel coordinate information corresponding to the identification feature is confirmed based on the image data.

[0013] According to the camera's preset first intrinsic parameter calibration data The pixel coordinate information Determine the normalized coordinates of the identification feature in the camera.

[0014] Obtain the distance information d between the identifier feature and the camera. i ;

[0015] Based on the standardized coordinates and the distance information, determine the first position information (x) of the identification feature in the camera coordinate system. ci ,y ci ,z ci ).

[0016] Optionally, the control camera acquires image data of the identifying features, including:

[0017] Control the laser rangefinder to emit a laser beam that falls on the target plane to form a laser point;

[0018] Acquire ranging information from a laser rangefinder and control a camera to acquire image data of the laser point, wherein the laser point is the identifying feature;

[0019] The step of obtaining the distance information between the identifier feature and the camera includes:

[0020] Based on the ranging information and the positional relationship between the laser rangefinder and the camera, the distance information between the marker feature and the camera is determined.

[0021] Optionally, after determining the standardized coordinates of the identification feature in the camera, the method further includes:

[0022] For the standardized coordinates Perform distortion correction and obtain the corrected standardized coordinates (x...cn ,y cn ,1).

[0023] Optionally, determining the second location information of the identifier feature in the target coordinate system based on the first location information includes:

[0024] Determine the Euclidean distance between the identification features based on the first location information;

[0025] The second position information (0,0,0), (x2,0,0), and (x3,y3,0) of the identification feature in the target coordinate system are determined based on the Euclidean distance; wherein the target coordinate system is established based on the identification feature.

[0026] The step of determining the second extrinsic parameter calibration data based on the first location information and the second location information includes:

[0027] Obtain the transformation relationship between the camera coordinate system and the target coordinate system, and determine the second extrinsic parameter calibration data based on the first position information, the second position information, and the transformation relationship.

[0028] Optionally, determining the second intrinsic parameter calibration data based on the camera's preset first intrinsic parameter calibration data, the camera's preset first extrinsic parameter calibration data, and the second extrinsic parameter calibration data includes:

[0029] The first object distance information d is obtained based on the first preset external parameter calibration data of the camera. o 1 d o 1 =T3 1 T3 1 T3 is the first extrinsic parameter calibration data; and the first image distance information d is obtained based on the first intrinsic parameter calibration data preset by the camera. i 1 d i 1 =α 1 ·dx=β 1 ·dy;

[0030] Determine focal length information based on the first object distance information and the first image distance information:

[0031]

[0032] Where, d x d y These represent the pixel size per unit pixel in the horizontal and vertical directions, respectively; f x f y These are the focal length information in the horizontal and vertical directions, respectively;

[0033] The second object distance information d is obtained based on the second external parameter calibration data. o *,d o *=T3 * ;

[0034] Based on the camera's preset first intrinsic parameter calibration data The focal length information and the second object distance information determine the second intrinsic parameter calibration data:

[0035]

[0036] Optionally, determining the camera calibration result based on the second extrinsic calibration data and the second intrinsic calibration data includes:

[0037] Using the second external parameter calibration data and the second internal parameter calibration data as the new first external parameter calibration data and the new first internal parameter calibration data respectively, the new second external parameter calibration data and the new second internal parameter calibration data are calculated after updating the data;

[0038] Repeat the above steps for iterative optimization until the iteration termination condition is met, and determine the final second extrinsic parameter calibration data and the final second intrinsic parameter calibration data as the calibration result of the camera.

[0039] To solve the above-mentioned technical problems, another technical solution adopted by the present invention is: to provide a camera calibration system, the system comprising:

[0040] The first acquisition module is used to control the camera to acquire image data of the identification feature, and to determine the first position information of the identification feature in the camera coordinate system based on the image data and the first intrinsic parameter calibration data preset by the camera.

[0041] The second acquisition module is used to determine the second position information of the identification feature in the target coordinate system based on the first position information;

[0042] The extrinsic parameter determination module is used to determine the second extrinsic parameter calibration data based on the first position information and the second position information;

[0043] The calibration module is used to determine the second intrinsic parameter calibration data based on the first preset intrinsic parameter calibration data, the first preset extrinsic parameter calibration data, and the second extrinsic parameter calibration data of the camera, and to determine the calibration result of the camera based on the second extrinsic parameter calibration data and the second intrinsic parameter calibration data.

[0044] To solve the above-mentioned technical problems, another technical solution adopted by the present invention is to provide an electronic device, the device comprising: a memory and a processor connected to the memory; wherein, the memory stores program data, and the processor retrieves the program data stored in the memory to execute the process steps of the camera calibration method described above.

[0045] To solve the above-mentioned technical problems, another technical solution adopted by the present invention is to provide a computer-readable storage medium that stores program instructions internally, which are executed to implement the process steps of the camera calibration method described above.

[0046] Unlike existing technologies, this application pre-calibrates to determine the first internal parameter calibration data, the first external parameter calibration data, or directly obtains the factory calibration reference data. When performing on-site engineering calibration, it is only necessary to control the camera to acquire one or several images of the identification features. Based on the images and the first internal parameter calibration data and the first external parameter calibration data, calculations can be performed to obtain the second external parameter calibration data and the second internal parameter calibration data for on-site engineering calibration. This allows for rapid determination of the camera calibration results, greatly reducing the complexity and time complexity of on-site calibration operations and improving on-site calibration efficiency. Attached Figure Description

[0047] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Wherein:

[0048] Figure 1 This is a flowchart illustrating an embodiment of the camera calibration method provided by the present invention;

[0049] Figure 2 This is a flowchart illustrating an embodiment of step S101 in this invention;

[0050] Figure 3 This is a flowchart illustrating an embodiment of step S21 in this invention;

[0051] Figure 4 This is a flowchart of an embodiment following step S23 in this invention;

[0052] Figure 5 This is a flowchart illustrating an embodiment of step S102 in this invention;

[0053] Figure 6 This is a schematic diagram of the coordinates of the identification feature PQS in the target plane in this invention;

[0054] Figure 7 This is a flowchart illustrating an embodiment of step S104 in this invention;

[0055] Figure 8 This is a flowchart illustrating another embodiment of step S104 in this invention;

[0056] Figure 9 This is a schematic diagram of an embodiment of the camera calibration system of the present invention;

[0057] Figure 10 This is a schematic diagram of the structure of an embodiment of the electronic device of the present invention;

[0058] Figure 11 This is a schematic diagram of an embodiment of a computer-readable storage medium in this invention. Detailed Implementation

[0059] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It is understood that the specific embodiments described herein are only for explaining this application and not for limiting it. Furthermore, it should be noted that, for ease of description, only the parts related to this application are shown in the accompanying drawings, not all structures. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0060] The reference to "embodiment" in this application means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0061] The steps in the embodiments of this application are not necessarily processed in the order described. The steps can be rearranged, deleted, or added as needed. The step descriptions in the embodiments of this application are only optional combinations of sequences and do not represent all possible combinations of steps in the embodiments of this application. The order of steps in the embodiments should not be considered as a limitation of this application.

[0062] The term "and / or" in the embodiments of this application refers to any and all possible combinations including one or more of the associated listed items. It should also be noted that, when used in this specification, "including / comprising" specifies the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, and / or components and / or groups thereof.

[0063] The terms "first," "second," etc., used in this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0064] Furthermore, although the terms "first," "second," etc., are used repeatedly in this application to describe various data (or various components, or various applications, or various instructions, or various operations), these data (or components, applications, instructions, or operations) should not be limited by these terms. These terms are only used to distinguish one piece of data (or component, application, instruction, or operation) from another piece of data (or component, application, instruction, or operation). For example, first position information can be called second position information, and second position information can be called first position information; the only difference is the scope they encompass, but it does not depart from the scope of this application. First position information and second position information are both collections of various position and attitude information, but they are not the same collection of position and attitude information.

[0065] Please see Figure 1 , Figure 1 This is a schematic flowchart of an embodiment of the camera calibration method of the present invention. It should be noted that if substantially the same result is obtained, the method of the present invention is not necessarily identical. Figure 1 The illustrated process sequence is limited. For example... Figure 1 As shown, the method includes the following steps:

[0066] S101. Control the camera to acquire image data of the identification feature, and determine the first position information of the identification feature in the camera coordinate system based on the image data and the camera's preset first intrinsic parameter calibration data.

[0067] Before the camera acquires image data of the marker features, it needs to be calibrated according to the actual engineering needs. The camera is set up and the target plane marker is selected. That is, based on the target distance, on-site environmental limitations, target orientation, etc., the marker is preset at a suitable position on the target. The camera is set up and the camera attitude is adjusted so that the preset marker is within the camera's field of view, generally in the center of the field of view is best. The lens aperture and lens focus ring are adjusted so that the preset marker has appropriate brightness and is in a clear focus state.

[0068] Optionally, the methods for selecting markers include selecting several marker features on the target itself, placing conventional markers, speckle patterns, etc., or laser marking, and appropriate methods can be selected according to different application scenarios.

[0069] Optionally, the camera here includes monocular cameras, binocular cameras, depth cameras, and so on.

[0070] Alternatively, during the adjustment of the lens focus ring, the lens focal length F remains constant. When using a fixed-focus lens, the focusing process only adjusts the distance from the lens optical center to the camera's image sensor, i.e., the imaging plane.

[0071] In some embodiments, adjusting the camera's attitude includes adjusting the camera's pitch angle, yaw angle, and roll angle; for binocular cameras, the angle between the cameras can also be adjusted so that the target is simultaneously in the field of view of both cameras.

[0072] In one embodiment of this application, as Figure 2 Step S101 specifically includes:

[0073] Sub-step S21: Control the camera to acquire image data of the identifying features;

[0074] The image data here is usually a two-dimensional image.

[0075] Sub-step S22: Confirm the pixel coordinate information corresponding to the identification features based on the image data;

[0076] Specifically, an appropriate recognition algorithm is selected based on the characteristics of the identifier features to confirm its corresponding pixel coordinate information. If the identifier feature is circular, a circle center detection algorithm can be used to fit the circle center, and the obtained circle center pixel coordinates are the pixel coordinate information corresponding to the identifier feature.

[0077] Optionally, for visualized image data, the corresponding pixel coordinate information can be determined by manually selecting identifier features.

[0078] Optionally, the image data can be preprocessed, such as image denoising, image augmentation, image filtering, rotation transformation, etc., to facilitate subsequent image processing.

[0079] Sub-step S23: Determine the standardized coordinates of the identification feature in the camera based on the camera's preset first intrinsic parameter calibration data and pixel coordinate information;

[0080] Specifically, the camera's preset first intrinsic parameter calibration data The known calibration data can be the camera's factory calibration results or other indoor or outdoor calibration results.

[0081] For a specific camera, the pixel coordinates (u, v) of a given imaging point in the pixel coordinate system and the normalized coordinates (x, v) of that point in the image coordinate system are... cn ,y cn The conversion formula for ,1) is as follows:

[0082]

[0083] The normalized coordinates (x, y) of this point in the camera coordinate system cn ,y cn ,1) and the three-dimensional coordinates (x,y) of the point in the camera coordinate system c y c , z c The conversion formula between () and () is as follows:

[0084]

[0085] The pixel coordinate system O-UV has its origin at the top left corner of the image matrix, with the u and v axes parallel to the two sides of the imaging target surface, i.e., the image plane. The coordinate unit is the pixel.

[0086] The camera coordinate system O-XcYcZc has the optical center of the camera lens as its origin. The Xc and Yc axes are parallel to the x and y axes of the image coordinate system, respectively, and the Zc axis coincides with the camera's optical axis. The coordinate unit is millimeters (mm) or other units.

[0087] Where u0 and v0 represent the pixel coordinates of the optical center of the camera lens in the image plane, i.e., the principal point coordinates; γ is the tilt coefficient, which represents the non-perpendicularity factor between the u-axis and v-axis in the pixel coordinate system. For a standard camera, γ is usually 0. f is the distance from the camera's optical center to the imaging target surface, which can be understood as the image distance; dx and dy are the pixel size, that is, the physical size of a unit pixel in the horizontal and vertical directions, in mm / pixel.

[0088] Specifically, the first intrinsic parameter calibration data is represented by a matrix as follows: Pixel coordinate information is Based on the above formula, the standardized coordinates in the camera coordinate system can be obtained. in, This indicates the pixel coordinates that identify lens distortion. This represents the standardized coordinates of the identifying features containing lens distortion in the camera coordinate system.

[0089] Sub-step S24: Obtain the distance information between the identifier features and the camera;

[0090] Specifically, the distance information d here i It can be measured by a ranging tool, such as radar ranging, laser ranging, tape measure ranging, etc.

[0091] Sub-step S25: Determine the first position information of the marker feature in the camera coordinate system based on the standardized coordinates and distance information.

[0092] According to the Euclidean distance formula, when the distance from the real point to the camera is d, the following relationship holds:

[0093] (x c -0) 2 +(y c -0) 2 +(z c -0) 2 =d 2

[0094] Through derivation, we can obtain that

[0095]

[0096] Therefore, according to standardized coordinates and distance information d i Then we can find z. ci Then, the coordinates (x, y) of the identifier feature in the camera coordinate system are obtained. ci y ci , z ci ).

[0097] In one embodiment of this application, as Figure 3 Step S21 specifically includes:

[0098] Sub-step S31: Control the laser rangefinder to emit a laser beam that falls on the target plane to form a laser point;

[0099] The target plane refers to the actual or approximate plane on which the monitoring target is located, as determined by visual measurement requirements.

[0100] Optionally, the laser rangefinder can be either built into the camera system or an external laser rangefinder. The two types are used in different scenarios. Built-in laser rangefinders are mainly used in close-range scenarios with low ambient light, while external laser rangefinders have a longer effective measuring distance, up to 100 meters, and can be used for long-distance, large-sized targets in scenarios with strong ambient light. Furthermore, using a rangefinder with higher power or a longer measuring range can effectively measure distances of 1-2 kilometers or more.

[0101] Sub-step S32: Obtain the ranging information of the laser rangefinder and control the camera to acquire image data of the laser point, where the laser point is an identifying feature;

[0102] Specifically, the angle of the laser rangefinder can be finely adjusted to project several different laser points onto the target plane. For each laser point projected, the rangefinder's distance measurement information is obtained, and the camera is controlled to capture its image.

[0103] Optionally, for binocular cameras, it is necessary to operate the laser rangefinders attached to each of the two cameras to make the two laser points projected by the two laser rangefinders coincide in order to perform distance measurement and shooting. For example, you can first control the laser rangefinder attached to the left camera to project a laser point onto the target plane in the common field of view, and then control the laser rangefinder attached to the right camera to project a laser point onto the same point so that the two laser points coincide. Then, you can acquire the images captured by each of the two cameras, as well as the distance measurement information from each of the two laser rangefinders.

[0104] Optionally, it is generally necessary to project at least three non-collinear laser points as identification features.

[0105] Optionally, the laser optical center can be obtained by processing the acquired image data. This can be done by manually selecting the laser optical center in the visualization interface directly from the magnified image, generally using the center of the laser spot as the laser center. Alternatively, the laser region can be locked by setting a grayscale threshold and the laser optical center can be determined through automated image processing. The laser optical center can also be identified using existing recognition algorithms, which will not be elaborated here.

[0106] Step S24 specifically includes determining the distance information between the marker feature and the camera based on the ranging information and the positional relationship between the laser rangefinder and the camera.

[0107] Optionally, for built-in laser rangefinders, the laser emission point and the camera's optical center are usually set to be on the same vertical line, and the two are very close to each other. The distance between them is negligible compared to the distance from the marker point to the camera. This makes the rangefinder's distance measurement information basically equal to the distance information between the laser point and the camera, that is, the distance information between the marker feature and the camera.

[0108] Alternatively, for a standalone external laser rangefinder, if the positional relationship between the laser rangefinder and the camera is known, the distance between the marker feature and the camera can also be calculated based on the ranging information.

[0109] Furthermore, for systems where the positional relationship between the camera and the laser rangefinder is fixed, the positional relationship between the laser rangefinder and the camera can be pre-calibrated.

[0110] In one embodiment of this application, as Figure 4 After step S23, the method further includes performing distortion correction on the standardized coordinates to obtain the corrected standardized coordinates.

[0111] Specifically, the standardized coordinates (x, y) of the corrected identification features are obtained by conventional calculation based on the distortion coefficients obtained from the known calibration results. cn ,y cn ,1).

[0112] Specifically, distortion is an image deformation that a camera cannot avoid due to its inherent imaging characteristics. Intrinsic parameters describe the camera's internal characteristics, including the image center and focal length. Distortion can be divided into two types: tangential distortion and radial distortion. Radial distortion occurs because light rays bend more when they are farther from the lens center than when they are closer to the center; radial distortion includes barrel distortion and pincushion distortion. Tangential distortion occurs because the lens and image are not perfectly parallel; the distortion system is not affected by the camera's resolution or other factors and is an inherent property of the camera itself.

[0113] Alternatively, a commonly used model for radial distortion is shown below, where k1, k2, and k3 are the radial distortion coefficients.

[0114]

[0115] A commonly used model for tangential distortion is shown below, where the tangential distortion coefficients are p1 and p2;

[0116]

[0117] Alternatively, distortion correction can be solved based on other distortion models, such as the division model, the Brown distortion model, and higher-order polynomial models, which will not be elaborated here.

[0118] Alternatively, for lenses with very low distortion, distortion can be ignored.

[0119] S102. Determine the second position information of the identification feature in the target coordinate system based on the first position information.

[0120] The target coordinate system is a three-dimensional coordinate system established in the target plane, which is generally determined based on the marking features.

[0121] In one embodiment of this application, as Figure 5 Step S102 further includes,

[0122] Sub-step S51: Determine the Euclidean distance between the identifier features based on the first location information;

[0123] Specifically, based on the principle of distance invariance, according to the coordinates (x, y) of the identifier features in the camera coordinate system... ci y ci , z ci This allows us to determine the Euclidean distance between identifying features.

[0124] Specifically, suppose there are three non-collinear marker features, such as marker points P, Q, and S, whose coordinates in the camera coordinate system are (x, y, q, s) respectively. c1 y c1 , z c1 (x)c2 y c2 , z c2 (x) c3 y c3 , z c3 The Euclidean distance between P and Q can be calculated as d. 12 The Euclidean distance between P and S is d. 13 The Euclidean distance between Q and S is d. 13 .

[0125] Sub-step S52: Determine the second position information of the identifier features in the target coordinate system based on the Euclidean distance between the identifier features; wherein, the target coordinate system is established based on the identifier features;

[0126] Specifically, a target coordinate system can be established based on three identifying features. The plane containing these three identifying features replaces the target plane, and the plane containing the three identifying features represents the O-XY plane of the target coordinate system. Assume P determines the origin of the target coordinate system, the line connecting P and Q determines the X-axis direction, a perpendicular line is drawn from S to the X-axis, and the direction of this perpendicular line determines the Y-axis direction. The Z-axis is perpendicular to the OXY plane. The target coordinate system is established according to the right-hand rule. The coordinates of P, Q, and S in the target coordinate system can be set as (0,0,0), (x2,0,0), and (x3,y3,0), respectively, and the pixel coordinates of P, Q, and S are...

[0127] Based on the invariance of distance during coordinate system transformation, that is, the Euclidean distance between P and Q calculated in the camera coordinate system is equal to the Euclidean distance between P and Q calculated in the target coordinate system, then:

[0128] x2=|QP|=d 12

[0129]

[0130]

[0131] like Figure 6 Clearly, x2 equals the Euclidean distance d between P and Q. 12 , that is, the magnitude of vector PQ is |QP|; x3 is equal to the projection of vector PS along the direction of vector PQ, that is, the value obtained by the inner product operation of vector PS and the unit vector in the direction of vector PQ; in the figure, P, Q, and S rotate counterclockwise, and y3 is equal to the projection of vector PQ on the perpendicular line of PS, that is, the value obtained by the outer product operation of vector PS and the unit vector in the direction of vector PQ.

[0132] Using the above method, the coordinates of each point in the target coordinate system can be obtained simply by defining the target world coordinate system and performing simple geometric relationship calculations. That is, the coordinates of P, Q, and S in the target coordinate system can be calculated quickly, reducing the amount of computation.

[0133] Optionally, a target coordinate system can be established according to actual needs, and the coordinates of P, Q, and S in the target coordinate system can be obtained through other geometric or analytical methods, which will not be elaborated here.

[0134] S103. Determine the second external parameter calibration data based on the first position information and the second position information.

[0135] Furthermore, the transformation relationship between the camera coordinate system and the target coordinate system is obtained, and the second external parameter calibration data is determined based on the first position information, the second position information, and the transformation relationship.

[0136] Specifically, for a given camera, the three-dimensional coordinates (x, y, z) of a real point in the camera coordinate system. c y c , z c The conversion formula between the point and its three-dimensional coordinates (x, y, z) in the target coordinate system is as follows:

[0137]

[0138] in, R is the extrinsic parameter matrix, R is the rotation matrix, which is a 3×3 identity orthogonal matrix; T is the translation vector, which represents the coordinates in the corresponding target coordinate system.

[0139] Specifically, taking P, Q, and S as examples above, based on the first position information (x) c1 y c1 , z c1 (x) c2 y c2 , z c2 (x) c3 y c3 , z c3 The second position information (0,0,0), (x2,0,0), (x3,y3,0), and the physical meaning of the identification features in the target coordinate system and the definition of the rotation matrix R are given by the following formula:

[0140]

[0141] make but R can be obtained 13 R 23 R 33 .

[0142] Among them, vectors r1, r2, and r3 satisfy ||r1|| = 1, ||r2|| = 1, and r1·r2 = 0, which can be used to verify the above.

[0143] Calculation results.

[0144] The first position information (x) c1 y c1 , z c1 (x) c2 y c2 , z c2 (x) c3 y c3 , z c3 Substituting the specific values ​​of the second position information (0,0,0), (x2,0,0), and (x3,y3,0) into the above formula and performing the calculation, we can obtain the second extrinsic parameter calibration data, which can be represented by a matrix.

[0145] Using the above method, the values ​​of each component in the extrinsic parameter calibration matrix can be quickly obtained through simple geometric calculations, without the need to solve complex multivariate equations, thus reducing the amount of computation.

[0146] S104. Determine the second intrinsic parameter calibration data based on the camera's preset first intrinsic parameter calibration data, the camera's preset first extrinsic parameter calibration data, and the second extrinsic parameter calibration data. Determine the camera's calibration result based on the second extrinsic parameter calibration data and the second intrinsic parameter calibration data.

[0147] In one embodiment of this application, as Figure 7 Step S104 further includes,

[0148] Sub-step S71: Obtain the first object distance information d based on the camera's preset first extrinsic parameter calibration data. o 1 d o 1 =T3 1 T3 1 T3 is the first extrinsic parameter calibration data; and the first image distance information d is obtained based on the first intrinsic parameter calibration data preset by the camera. i 1 d i 1 =α 1 ·dx=β 1 ·dy;

[0149] The calibration data for the first extrinsic parameter is represented by a matrix as follows: The first intrinsic parameter calibration data, represented by a matrix, is as follows: dx and dy represent the pixel size per unit pixel in the horizontal and vertical directions, respectively. Here, the horizontal and vertical directions refer to the image coordinate system OX.cn Y cn The x-axis and y-axis directions.

[0150] Specifically, in physics, object distance information refers to the distance from the object to the optical center of the lens. Here, it is equivalent to approximating the object distance information with the z-axis translation component T3 in the extrinsic parameter matrix, thus obtaining d. o 1 =T3 1 .

[0151] Specifically, image distance information is the distance between the image of the object and the optical center of the lens, according to the definition. f is the distance from the camera's optical center to the imaging target surface. Approximating f with the phase distance information, we obtain d. i 1 =α 1 ·dx=β 1 ·dy.

[0152] Sub-step S72: Determine focal length information based on the first object distance information and the first image distance information:

[0153]

[0154] Among them, f x f y These are the focal length information in the horizontal and vertical directions, respectively. Here, horizontal and vertical refer to the image coordinate system OX. cn Y cn The x-axis and y-axis directions.

[0155] Specifically, according to the Gaussian imaging formula, By substituting the relevant data, f can be obtained. x f y .

[0156] Sub-step S73: Obtain the second object distance information d based on the second external parameter calibration data. o *,d o *=T3 * ;

[0157] The calibration data for the second extrinsic parameter is represented by a matrix as follows:

[0158] Sub-step S74: Based on the camera's preset first intrinsic parameter calibration data The focal length information and the second object distance information determine the second intrinsic parameter calibration data:

[0159]

[0160] Specifically, the second intrinsic parameter calibration data is represented by a matrix as follows: Among them, u0, v0, and γ are only related to the camera's own properties, and their values ​​remain basically unchanged.

[0161] Specifically, for prime lenses, f x f y If it remains unchanged, then according to Gauss's formula, we also have...

[0162]

[0163] Here, dx and dy are also inherent properties of the camera, and their values ​​remain basically unchanged, which can be derived.

[0164] By substituting the relevant data, α* and β* can be obtained.

[0165] Specifically, if the obtained second extrinsic parameter calibration data Second internal parameter calibration data The results have been verified to meet the accuracy requirements for the target scene and can be directly used as the calibration results for the camera in that scene.

[0166] In one embodiment of this application, as Figure 8 Step S104 further includes,

[0167] Sub-step S81: Using the second external parameter calibration data and the second internal parameter calibration data as the new first external parameter calibration data and the new first internal parameter calibration data, the data is updated and the new second external parameter calibration data and the new second internal parameter calibration data are calculated.

[0168] Specifically, if the obtained second extrinsic parameter calibration data Second internal parameter calibration data If the accuracy requirements for the target scenario cannot be met after testing, the obtained data is used as the preset value, i.e. the initial value for iteration, and further optimization is carried out according to the method in this application.

[0169] Sub-step S82: Repeat the above steps for iterative optimization until the iteration termination condition is met, and determine the final second external parameter calibration data and the final second internal parameter calibration data as the calibration result of the camera.

[0170] Specifically, the iteration termination condition can be set as the value obtained after reaching a preset number of iterations n, where n can be 50 or other positive values ​​depending on the actual situation; alternatively, it can be set as the iteration error function value being less than ε, where ε can be a value close to 0 depending on the actual situation, and the error function can be selected from, but is not limited to, the classic least squares error function method. At this point, it indicates that the iteration has stabilized.

[0171] In practical applications, the method of this application, through pre-calibration to determine the first internal parameter calibration data and the first external parameter calibration data, or by directly obtaining the reference data from the factory calibration, only requires controlling the camera to acquire one or several images of the identified features during on-site engineering calibration. Based on the images and the first internal parameter calibration data and the first external parameter calibration data, calculations are performed to obtain the second external parameter calibration data and the second internal parameter calibration data for on-site engineering calibration. This quickly determines the camera calibration result, greatly reducing the complexity and time complexity of on-site calibration operations and improving on-site calibration efficiency. Practical applications include...

[0172] 1. The camera is calibrated by combining the target's own identifying features.

[0173] Specifically, this generally requires that there be one or more objects with specific textures or shapes in the target scene, which can be identified as identifying features and have a certain positional relationship with each other.

[0174] 2. Calibration of the camera in conjunction with a laser rangefinder.

[0175] Specifically, a laser rangefinder is used to project laser points onto the target plane, and these laser points are used as identifying features for identification.

[0176] 3. The camera is calibrated using a specific calibration plate / speckle map, etc.

[0177] Specifically, a specific calibration plate / speckle map is placed in the target scene, and the specific calibration plate / speckle map is used as a labeling feature for identification.

[0178] The above-mentioned application methods can be selected according to specific scenarios.

[0179] Please see Figure 9 , Figure 9 This is a schematic diagram of an embodiment of the camera calibration system of the present invention. In this embodiment, the camera calibration system 200 includes a first acquisition module 201, a second acquisition module 202, an external parameter determination module 203, and a calibration module 204.

[0180] The first acquisition module 201 is used to control the camera to acquire image data of the identification features, and to determine the first position information of the identification features in the camera coordinate system based on the image data and the camera's preset first intrinsic parameter calibration data.

[0181] The second acquisition module 202 is used to determine the second position information of the identification feature in the target coordinate system based on the first position information;

[0182] The extrinsic parameter determination module 203 is used to determine the second extrinsic parameter calibration data based on the first position information and the second position information;

[0183] The calibration module 204 is used to determine the second intrinsic parameter calibration data based on the camera's preset first intrinsic parameter calibration data, the camera's preset first extrinsic parameter calibration data, and the second extrinsic parameter calibration data, and to determine the camera's calibration result based on the second extrinsic parameter calibration data and the second intrinsic parameter calibration data.

[0184] Optionally, the first acquisition module 201 is specifically used to control the camera to acquire image data of the identification feature; to confirm the pixel coordinate information corresponding to the identification feature based on the image data, and to determine the standardized coordinates of the identification feature in the camera based on the camera's preset first intrinsic parameter calibration data and pixel coordinate information; and to acquire the distance information between the identification feature and the camera, and to determine the first position information of the identification feature in the camera coordinate system based on the standardized coordinates and distance information.

[0185] Optionally, the first acquisition module 201 is specifically used to control the laser rangefinder to emit a laser beam that falls on the target plane to form a laser point, acquire the ranging information of the laser rangefinder and control the camera to acquire the image data of the laser point, the laser point being an identification feature; and determine the distance information between the identification feature and the camera based on the ranging information and the positional relationship between the laser rangefinder and the camera.

[0186] Optionally, after determining the standardized coordinates of the identification features in the camera, the first acquisition module 201 further includes: performing lens distortion correction on the standardized coordinates to obtain the corrected standardized coordinates.

[0187] Optionally, the second acquisition module 202 is specifically used to determine the Euclidean distance between the identification features based on the first position information, and to determine the second position information of the identification features in the target coordinate system based on the Euclidean distance; wherein, the target coordinate system is established based on the identification features.

[0188] Optionally, the extrinsic parameter determination module 203 is specifically used to obtain the transformation relationship between the camera coordinate system and the target coordinate system, and to determine the second extrinsic parameter calibration data based on the first position information, the second position information, and the transformation relationship.

[0189] Optionally, the calibration module 204 is specifically used to determine the second intrinsic parameter calibration data based on the camera's preset first intrinsic parameter calibration data, the camera's preset first extrinsic parameter calibration data, and the second extrinsic parameter calibration data, including:

[0190] The first object distance information d is obtained based on the camera's preset first external parameter calibration data. o 1 d o 1 =T3 1 T3 1 T3 is the first extrinsic parameter calibration data; and the first image distance information d is obtained based on the first intrinsic parameter calibration data preset by the camera. i 1 di 1 =α 1 ·dx=β 1 ·dy;

[0191] Determine focal length information based on the first object distance information and the first image distance information:

[0192]

[0193] Where, d x d y These represent the pixel size per unit pixel in the horizontal and vertical directions, respectively; f x f y These are the focal length information in the horizontal and vertical directions, respectively;

[0194] And obtain the second object distance information d based on the second external parameter calibration data. o *,d o *=T3 * ;

[0195] Based on the camera's preset first intrinsic parameter calibration data The focal length information and the second object distance information determine the second intrinsic parameter calibration data:

[0196]

[0197] Optionally, the calibration module 204 is specifically used to use the second external parameter calibration data and the second internal parameter calibration data as the new first external parameter calibration data and the new first internal parameter calibration data, respectively, and calculate the new second external parameter calibration data and the new second internal parameter calibration data after updating the data;

[0198] Repeat the above steps for iterative optimization until the iteration termination condition is met, and determine the final second extrinsic parameter calibration data and the final second intrinsic parameter calibration data as the camera calibration results.

[0199] The other steps are the same as those in the camera calibration method, and will not be repeated here.

[0200] Please see Figure 10 , Figure 10 This is a schematic diagram of an embodiment of the electronic device of the present invention. This electronic device can perform the steps executed by the camera calibration system in the above method. For related details, please refer to the detailed description in the above method; it will not be repeated here.

[0201] The electronic device 300 includes a memory 301 and a processor 302 connected to the memory 301.

[0202] The memory 301 is used to store the operating system, instructions executed by the processor 302, and received messages, etc.

[0203] The processor 302 executes the following instructions: controlling the camera to acquire image data of the identification feature, and determining the first position information of the identification feature in the camera coordinate system based on the image data and the camera's preset first intrinsic parameter calibration data; determining the second position information of the identification feature in the target coordinate system based on the first position information; determining the second extrinsic parameter calibration data based on the first position information and the second position information; determining the second intrinsic parameter calibration data based on the camera's preset first intrinsic parameter calibration data, the camera's preset first extrinsic parameter calibration data, and the second extrinsic parameter calibration data; and determining the camera's calibration result based on the second extrinsic parameter calibration data and the second intrinsic parameter calibration data.

[0204] Optionally, the processor 302 executes the following instructions: controlling the camera to acquire image data of the identification feature, and determining the first position information of the identification feature in the camera coordinate system based on the image data and the camera's preset first intrinsic parameter calibration data, including: controlling the camera to acquire image data of the identification feature; confirming the pixel coordinate information corresponding to the identification feature based on the image data; determining the standardized coordinates of the identification feature in the camera based on the camera's preset first intrinsic parameter calibration data and pixel coordinate information; acquiring the distance information between the identification feature and the camera; and determining the first position information of the identification feature in the camera coordinate system based on the standardized coordinates and distance information.

[0205] Optionally, the processor 302 executes the following instructions: controlling the camera to acquire image data of the marker feature, including: controlling the laser rangefinder to emit a laser beam that falls on the target plane to form a laser point; acquiring the ranging information of the laser rangefinder and controlling the camera to acquire image data of the laser point, wherein the laser point is the marker feature; acquiring the distance information between the marker feature and the camera, including: determining the distance information between the marker feature and the camera based on the ranging information and the positional relationship between the laser rangefinder and the camera.

[0206] Optionally, after determining the standardized coordinates of the identification features in the camera, the processor 302 further includes: performing distortion correction on the standardized coordinates to obtain the corrected standardized coordinates.

[0207] Optionally, the processor 302 executes the instruction to: determine the second position information of the identification features in the target coordinate system based on the first position information, including: determining the Euclidean distance between the identification features based on the first position information; and determining the second position information of the identification features in the target coordinate system based on the Euclidean distance; wherein the target coordinate system is established based on the identification features;

[0208] The second extrinsic parameter calibration data is determined based on the first position information and the second position information, including: obtaining the transformation relationship between the camera coordinate system and the target coordinate system, and determining the second extrinsic parameter calibration data based on the first position information, the second position information, and the transformation relationship.

[0209] Optionally, the processor 302 executes the instruction to: determine the second intrinsic parameter calibration data based on the camera's preset first intrinsic parameter calibration data, the camera's preset first extrinsic parameter calibration data, and the second extrinsic parameter calibration data, including: obtaining the first object distance information d based on the camera's preset first extrinsic parameter calibration data. o 1 d o 1 =T3 1 T3 1 T3 is the first extrinsic parameter calibration data; and the first image distance information d is obtained based on the first intrinsic parameter calibration data preset by the camera. i 1 d i 1 =α 1 ·dx=β 1 ·dy;

[0210] Determine focal length information based on the first object distance information and the first image distance information:

[0211]

[0212] Where dx and dy are the pixel size of a unit pixel in the horizontal and vertical directions, respectively, and f x f y These are the focal length information in the horizontal and vertical directions, respectively;

[0213] The second object distance information d is obtained based on the second external parameter calibration data. o *,d o *=T3 * ;

[0214] Based on the camera's preset first intrinsic parameter calibration data The focal length information and the second object distance information determine the second intrinsic parameter calibration data:

[0215]

[0216] Optionally, the processor 302 executes the instruction to determine the camera calibration result based on the second external parameter calibration data and the second internal parameter calibration data, including: using the second external parameter calibration data and the second internal parameter calibration data as new first external parameter calibration data and new first internal parameter calibration data, updating the data and then calculating new second external parameter calibration data and new second internal parameter calibration data;

[0217] Repeat the above steps for iterative optimization until the iteration termination condition is met, and determine the final second extrinsic parameter calibration data and the final second intrinsic parameter calibration data as the calibration result of the camera.

[0218] Please see Figure 11 , Figure 11This is a schematic diagram of an embodiment of the computer-readable storage medium of the present invention. This computer-readable storage medium can perform the steps executed by the camera calibration system in the above method. For related details, please refer to the detailed description in the above method; it will not be repeated here.

[0219] The computer-readable storage medium 400 internally stores program instructions 401, which are executed to implement the steps performed by the camera calibration system in the above method.

[0220] The above solution differs from existing technologies. This application determines the first internal parameter calibration data and the first external parameter calibration data through pre-calibration, or directly obtains the reference data from the factory calibration. When performing on-site engineering calibration, it is only necessary to control the camera to acquire one or several images of the identification features. Based on the images and the first internal parameter calibration data and the first external parameter calibration data, calculations can be performed to obtain the second external parameter calibration data and the second internal parameter calibration data for on-site engineering calibration. This allows for rapid determination of the camera calibration results, greatly reducing the complexity and time complexity of on-site calibration operations and improving on-site calibration efficiency.

[0221] In the several embodiments provided by this invention, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interfaces, apparatuses, or units, and may be electrical, mechanical, or other forms.

[0222] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0223] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0224] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0225] The above description is merely an embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A camera calibration method, characterized by, The method comprises: controlling the camera to acquire image data of the identification feature, and determining first position information of the identification feature in a camera coordinate system according to the image data and first intrinsic calibration data preset for the camera; determining second position information of the identification feature in a target coordinate system according to the first position information; determining second extrinsic calibration data according to the first position information and the second position information; determining second intrinsic calibration data according to the first intrinsic calibration data preset for the camera, first extrinsic calibration data preset for the camera and the second extrinsic calibration data, and determining a calibration result of the camera based on the second extrinsic calibration data and the second intrinsic calibration data; wherein the controlling the camera to acquire image data of the identification feature, and determining first position information of the identification feature in a camera coordinate system according to the image data and first intrinsic calibration data preset for the camera comprises: controlling the camera to acquire image data of the identification feature; determining pixel coordinate information corresponding to the identification feature according to the image data; determining standardized coordinates of the identification feature in the camera according to the first intrinsic calibration data preset for the camera and the pixel coordinate information; acquiring distance information of the identification feature from the camera; determining first position information of the identification feature in a camera coordinate system according to the standardized coordinates and the distance information; wherein the determining second position information of the identification feature in a target coordinate system according to the first position information comprises: determining Euclidean distances between the identification features according to the first position information; determining second position information of the identification feature in a target coordinate system according to the Euclidean distances; wherein the target coordinate system is established based on the identification features.

2. The method of claim 1, wherein the controlling the camera to acquire image data of the identification feature comprises: controlling a laser range finder to emit laser light to fall on a target plane to form a laser point; acquiring ranging information of the laser range finder and controlling the camera to acquire image data of the laser point, the laser point being the identification feature; the acquiring distance information of the identification feature from the camera comprises: determining distance information of the identification feature from the camera according to the ranging information and a positional relationship between the laser range finder and the camera.

3. The method of claim 1, wherein after the determining standardized coordinates of the identification feature in the camera, the method further comprises: performing distortion correction on the standardized coordinates to obtain corrected standardized coordinates.

4. The method of claim 1, wherein the determining second extrinsic calibration data according to the first position information and the second position information comprises: acquiring a conversion relationship between the camera coordinate system and the target coordinate system, and determining second extrinsic calibration data according to the first position information, the second position information and the conversion relationship.

5. The method of claim 1, wherein The second intrinsic parameter calibration data is determined according to the first intrinsic parameter calibration data preset for the camera, the first extrinsic parameter calibration data preset for the camera, and the second extrinsic parameter calibration data. According to the first extrinsic parameter calibration data preset by the camera, first object distance information is acquired , , The first extrinsic parameter calibration data is used for ; and first image distance information is acquired according to first intrinsic parameter calibration data preset by the camera , ; The focal length information is determined based on the first object distance information and the first image distance information. ， ； wherein, respectively the pixel size of a unit pixel in the horizontal direction and the vertical direction, respectively the focal length information in the horizontal direction and the vertical direction; acquire second object distance information according to the second external parameter calibration data , ; based on the first intrinsic calibration data preset for the camera , the focal length information, and the second object distance information to determine second intrinsic calibration data: ， 。 6. The method of claim 1, wherein The calibration result of the camera is determined based on the second extrinsic parameter calibration data and the second intrinsic parameter calibration data. The second extrinsic parameter calibration data and the second intrinsic parameter calibration data are updated as new first extrinsic parameter calibration data and new first intrinsic parameter calibration data, respectively, and new second extrinsic parameter calibration data and new second intrinsic parameter calibration data are calculated after the update; The above steps are repeated for iterative optimization until an iteration termination condition is met, and the final second extrinsic parameter calibration data and the final second intrinsic parameter calibration data are determined as the calibration result of the camera.

7. A camera calibration system characterized by, The system comprises: The first acquisition module is configured to control the camera to acquire image data of the identification features, and determine first position information of the identification features in a camera coordinate system according to the image data and first intrinsic parameter calibration data preset for the camera; The second acquisition module is configured to determine second position information of the identification features in a target coordinate system according to the first position information; The extrinsic parameter determination module is configured to determine second extrinsic parameter calibration data according to the first position information and the second position information; The calibration module is configured to determine second intrinsic parameter calibration data according to the first intrinsic parameter calibration data preset for the camera, the first extrinsic parameter calibration data preset for the camera, and the second extrinsic parameter calibration data, and determine a calibration result of the camera based on the second extrinsic parameter calibration data and the second intrinsic parameter calibration data. The first acquisition module is further configured to control the camera to acquire image data of the identification features, determine pixel coordinate information corresponding to the identification features according to the image data, determine standardized coordinates of the identification features in the camera according to the first intrinsic parameter calibration data preset for the camera and the pixel coordinate information, acquire distance information of the identification features from the camera, and determine first position information of the identification features in a camera coordinate system according to the standardized coordinates and the distance information; the second acquisition module is further configured to determine Euclidean distances between the identification features according to the first position information, and determine second position information of the identification features in a target coordinate system according to the Euclidean distances; and the target coordinate system is established based on the identification features.

8. An electronic device, comprising: The device comprises: A memory and a processor connected to the memory; wherein the memory stores program data, and the processor invokes the program data stored in the memory to execute the camera calibration method according to any one of claims 1-6.

9. A computer-readable storage medium internally storing program instructions, characterized in that, The program instructions are executed by the processor to implement the camera calibration method according to any one of claims 1-6.

Images