Evaluation Method, System, Medium and Computing Device for Intrinsic Parameter Accuracy of Camera Module

Abstract

A method for evaluating the internal parameter accuracy of a camera module, comprising: S1: setting a calibration component with multiple feature points; S2: obtaining the three-dimensional coordinates of the multiple feature points in three-dimensional space; S3: taking a first feature image including the multiple feature points at a first position, and extracting the first projection coordinates of the multiple feature points on the imaging plane; S4: taking a second feature image including the multiple feature points at a second position, and extracting the second projection coordinates of the multiple feature points on the imaging plane; S5: deriving the first external parameter and the second external parameter of the camera module according to the projection relationship from three-dimensional space to the imaging plane, the pre-obtained internal parameters of the camera module, the three-dimensional coordinates, the first projection coordinates and the second projection coordinates corresponding to the multiple feature points; S6: deriving the estimated distance between the first position and the second position according to the first external parameter and the second external parameter; S7: evaluating the accuracy of the internal parameters of the camera module according to the magnitude of the difference between the estimated distance and the actual distance.

Description

Technical Field

[0001] The present invention relates to a method, a system, a medium, and a computing device for evaluating the internal parameter accuracy of a camera module. Background Art

[0002] Parameter calibration of a camera module, such as a camera, refers to the process of using the camera to capture an image of a calibration board and calculating the camera parameters by using the three-dimensional coordinates of known feature points in the calibration board and the corresponding image coordinates on the image. Camera calibration is a basic link in machine vision applications such as visual measurement and three-dimensional reconstruction, and the accuracy and precision of the calibration result directly determine whether the vision system can work properly.

[0003] In the prior art, the methods for evaluating the calibration accuracy of camera internal parameters mainly include the reprojection error method and the triangulation ranging method.

[0004] The reprojection error method extracts the 2D coordinates of target points on the captured image, matches them with the 3D coordinates of the target points obtained by surveying, finds the optimal external parameters according to the calibrated internal parameters, projects the 3D coordinates of the surveyed target points by using the optimal external parameters and the calibrated internal parameters, and calculates the reprojection error between the projected points and the corresponding 2D coordinates of the target points. Generally, the average reprojection error is statistically calculated. Although this method is simple and efficient in calculation, it requires high-precision 3D survey coordinates, and the external parameters and internal parameters are coupled, so the correct internal parameter evaluation cannot be carried out.

[0005] The triangulation ranging method places more than one target target in the camera's field of view, installs the camera on a fixed fixture for image capture, measures the distance between the target targets and the distance from the camera to each target target at the same time, extracts the 2D coordinates of the target targets on the captured image, calculates the distance of the target targets by using the calibrated internal parameters and the measured distance from the camera to each target target, compares it with the measured distance, and calculates the size of the error. Although the evaluation accuracy of this method is high, it has high requirements for ranging accuracy, the calculation method is cumbersome, and it is not suitable for large-scale verification;

[0006] Therefore, in the prior art, it has become a problem to improve the accuracy of camera internal parameter evaluation, simplify the evaluation method, and improve the evaluation efficiency. Summary of the Invention

[0007] The purpose of the present invention is to provide a method that can improve the accuracy of camera internal parameter evaluation and improve the evaluation efficiency. To achieve the above purpose, one solution of the present invention is a method for evaluating the internal parameter accuracy of a camera module, including:

[0008] S1: Set a calibration component with multiple feature points, where the multiple feature points are located on one side of the calibration component facing the first camera module, and the first camera module is the camera module to be evaluated for internal parameter accuracy;

[0009] S2: Construct a first coordinate system in the three-dimensional space where the calibration component is located, and obtain the three-dimensional coordinates of each of the multiple feature points in the first coordinate system;

[0010] S3: Construct a second coordinate system on the imaging plane of the first camera module. Take a first feature image including the multiple feature points at a first shooting position through the first camera module, and extract the first projection coordinates of each of the multiple feature points in the second coordinate system from the first feature image;

[0011] S4: Take a second feature image including the multiple feature points at a second shooting position through the first camera module, and extract the second projection coordinates of each of the multiple feature points in the second coordinate system from the second feature image;

[0012] S5: According to the pre-established projection relationship from the first coordinate system to the second coordinate system, the pre-acquired internal parameters of the first camera module, the three-dimensional coordinates, the first projection coordinates, and the second projection coordinates corresponding to each of the multiple feature points, deduce the first external parameters and the second external parameters of the first camera module corresponding to the first shooting position and the second shooting position respectively;

[0013] S6: Deduce the estimated distance between the first shooting position and the second shooting position according to the first external parameters and the second external parameters;

[0014] S7: Pre-acquire the actual distance between the first shooting position and the second shooting position, and evaluate the accuracy of the internal parameters of the first camera module according to the difference between the estimated distance and the actual distance.

[0015] According to the foregoing technical solution, it is possible to judge the accuracy of the pre-acquired camera internal parameters according to the external parameters of the first camera module corresponding to two shooting positions.

[0016] In a preferred manner, after the first camera module finishes shooting at the first shooting position, it is translated to the second shooting position.

[0017] According to the foregoing technical solution, the first camera module only performs a simple translation movement between two shooting positions, which can simplify the pose change form of the first camera module, and thus simplify the calculation.

[0018] In a preferred manner, the three-dimensional coordinates of each of the multiple feature points are obtained by the following method:

[0019] S21: Set up the second camera module, pre-acquire its internal parameters with known accuracy, and capture multiple feature images containing the multiple feature points through the second camera module;

[0020] S22: Perform three-dimensional reconstruction on the multiple feature points based on the multiple feature images, and obtain the three-dimensional coordinates of each feature point among the multiple feature points in the first coordinate system.

[0021] According to the foregoing technical solution, the three-dimensional coordinates of each feature point in the world coordinate system can be obtained through three-dimensional reconstruction.

[0022] In a preferred manner, the first projection coordinates / second projection coordinates of each feature point among the multiple feature points are obtained by the following method:

[0023] S41: Convert the captured first feature image / second feature image containing the multiple feature points into a first grayscale image / second grayscale image;

[0024] S42: Convert the first grayscale image / second grayscale image into a first binary image / second binary image;

[0025] S43: Obtain multiple contours included in the first binary image / second binary image, and each contour among the multiple contours is associated with each feature point among the multiple feature points in a one-to-one correspondence;

[0026] S44: Calculate the first projection coordinates / second projection coordinates of each feature point among the multiple feature points associated according to each contour among the multiple contours.

[0027] According to the foregoing technical solution, the projection coordinates of each feature point are obtained by extracting the projection contours.

[0028] In a preferred manner, before deriving the external parameters of the first camera module, it further includes a step of eliminating error points:

[0029] S51: Set multiple coding points on the surface of the calibration component facing the first camera module, and obtain the coded three-dimensional coordinates corresponding to each coding point among the multiple coding points in the first coordinate system;

[0030] S52: Capture a coded image containing the multiple coding points and the multiple feature points through the first camera module, and extract the coded projection coordinates and actual feature projection coordinates corresponding to each coding point among the multiple coding points and each feature point among the multiple feature points on the coded image respectively in the second coordinate system;

[0031] S53: deriving preliminary external parameters of the first camera module according to the pre-established projection relationship from the first coordinate system to the second coordinate system, the pre-acquired internal parameters of the first camera module, the encoded three-dimensional coordinates corresponding to each of the multiple encoding points, and the encoded projection coordinates;

[0032] S54: deriving the estimated projection coordinates corresponding to each feature point in the plurality of feature points on the encoded image according to the pre-established projection relationship from the first coordinate system to the second coordinate system, the internal parameters of the first camera module and the preliminary external parameters, and the three-dimensional coordinates of each feature point in the plurality of feature points;

[0033] S55: Calculate the Euclidean distance between the estimated projection coordinates of each feature point in the multiple feature points and the corresponding actual feature projection coordinates. If the Euclidean distance is less than a preset threshold, determine that the feature point is a valid point; if the Euclidean distance is greater than the preset threshold, determine that the feature point is an error point and eliminate it.

[0034] According to the aforementioned technical solution, the preliminary external parameters of the camera can be solved by setting coding points, and the estimated projection coordinates of each feature point on the image can be solved based on the preliminary external parameters of the camera, and compared with the extracted actual feature projection coordinates to eliminate error points.

[0035] In a preferred embodiment, de-distortion processing is performed on each of the multiple feature points on the first feature image and the second feature image.

[0036] According to the aforementioned technical solution, the influence of camera distortion can be reduced.

[0037] In a preferred embodiment, the calibration component includes a plurality of calibration plates, and the plurality of feature points are distributed on a plate surface of each of the plurality of calibration plates that faces the first camera module.

[0038] According to the aforementioned technical solution, it is possible to shoot calibration plates at different angles without manually changing the angle of a single calibration plate, thereby improving the efficiency of camera calibration.

[0039] In addition, another aspect of the present invention is a system for evaluating the internal parameter accuracy of a camera module, characterized in that it includes:

[0040] A calibration component unit, which is arranged in front of the first camera module facing the object being photographed and has a plurality of feature points, wherein the plurality of feature points are located on a side of the calibration component unit facing the first camera module, and the first camera module is a camera module whose internal reference accuracy is to be evaluated;

[0041] The fixture unit fixes the first camera module and moves the first camera module from the first shooting position to the second shooting position to respectively shoot a first feature image and a second feature image including the multiple feature points;

[0042] The coordinate acquisition unit constructs a first coordinate system in the three-dimensional space where the calibration component is located, and acquires the three-dimensional coordinates of each of the multiple feature points in the first coordinate system; a second coordinate system is constructed on the imaging plane of the first camera module, and the first projection coordinate and the second projection coordinate of each of the multiple feature points in the second coordinate system are extracted from the first feature image and the second feature image;

[0043] The calculation unit derives the first external parameter and the second external parameter of the first camera module corresponding to the first shooting position and the second shooting position according to the pre-established projection relationship from the first coordinate system to the second coordinate system, the internal parameters of the first camera module obtained in advance, the three-dimensional coordinates, the first projection coordinates, and the second projection coordinates corresponding to each of the multiple feature points; according to the first external parameter and the second external parameter, the estimated distance between the first shooting position and the second shooting position is derived;

[0044] The evaluation unit pre-acquires the actual distance between the first shooting position and the second shooting position, and evaluates the accuracy of the internal parameters of the first camera module according to the magnitude of the difference between the estimated distance and the actual distance.

[0045] According to the foregoing technical solution, the accuracy of the internal parameters of the camera can be evaluated relying on this system.

[0046] In addition, another aspect of the present invention is a computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, the method for evaluating the accuracy of the internal parameters of the above-mentioned camera module is implemented.

[0047] In addition, another aspect of the present invention is a computing device, the computing device includes: a processor; a memory storing a computer program, and when the computer program is executed by the processor, the method for evaluating the accuracy of the internal parameters of the above-mentioned camera module is implemented.

[0048] The method for evaluating the accuracy of the internal parameters of the camera module in the above embodiments of the present application obtains the three-dimensional coordinates of the feature points through three-dimensional reconstruction, and the camera only performs a simple translational movement between two shooting positions, which can improve the accuracy of the camera internal parameter evaluation, simplify the evaluation method, and improve the evaluation efficiency. Description of the Drawings

[0049] To more clearly illustrate the present invention, the accompanying drawings of the present invention will be described and explained below. Obviously, the accompanying drawings in the following description only illustrate certain aspects of some exemplary embodiments of the present invention. For those of ordinary skill in the art, other accompanying drawings can be obtained based on these drawings without creative efforts.

[0050] Figure 1 It is a schematic diagram of the positions of the exemplary camera coordinate system and the imaging plane coordinate system.

[0051] Figure 2 It is a schematic diagram of the projection of the exemplary camera coordinate system and the imaging plane coordinate system.

[0052] Figure 3 It is a schematic diagram of the relationship between the exemplary imaging plane coordinate system and the pixel coordinate system.

[0053] Figure 4 It is a flowchart for evaluating the accuracy of the camera internal parameters.

[0054] Figure 5 It is a schematic diagram of the appearance of the calibration component.

[0055] Figure 6 It is a schematic diagram of the circular coded points.

[0056] Figure 7 It is a block diagram of the evaluation system for the internal parameter accuracy of the imaging module.

[0057] Explanation of the text in the accompanying drawings:

[0058] 1 Calibration component

[0059] 11 First calibration plate

[0060] 12 Second calibration plate

[0061] 13 Third calibration plate

[0062] 121 Center of the circle

[0063] 2 Ring-shaped coded points

[0064] 21 Bright ring band

[0065] 22 Dark ring band

[0066] 23 Central circle

[0067] 24 First position

[0068] 25 Last position

[0069] 10 Calibration component unit

[0070] 20 Fixture unit

[0071] 30 Coordinate acquisition unit

[0072] 40 Calculation unit

[0073] 50 Evaluation unit Detailed implementation manners

[0074] Various exemplary embodiments of the present disclosure are described in detail below with reference to the accompanying drawings. The description of the exemplary embodiments is merely illustrative and in no way limits the present disclosure and its application or use. The present disclosure can be implemented in many different forms and is not limited to the embodiments described herein. These embodiments are provided to make the present disclosure thorough and complete, and to fully convey the scope of the present disclosure to those skilled in the art. It should be noted that: unless otherwise specified, the relative arrangements, numerical expressions and numerical values, etc. of the components and steps set forth in these embodiments should be construed as merely exemplary and not as limitations.

[0075] The words such as "including" or "comprising" used in the present disclosure mean that the elements before this word cover the elements listed after this word, and do not exclude the possibility of also covering other elements.

[0076] All terms used in the present disclosure (including technical terms or scientific terms) have the same meaning as understood by those of ordinary skill in the art to which the present disclosure pertains, unless otherwise specifically defined. It should also be understood that terms defined in a general dictionary should be understood to have a meaning consistent with their meaning in the context of the relevant art, and should not be interpreted in an idealized or overly formal sense unless clearly defined as such herein.

[0077] For the components not described in detail in this part, the specific model parameters of the components, the mutual relationship between the components, and the control circuit, they can be considered as the technologies, methods and devices known to those of ordinary skill in the relevant art. However, under appropriate circumstances, the said technologies, methods and devices should be regarded as a part of the specification.

[0078] (Coordinate system transformation)

[0079] In the process of image measurement and machine vision applications, in order to determine the mutual relationship between the three-dimensional geometric position of a certain point on the surface of a spatial object and its corresponding point in the image, it is necessary to establish a geometric model of camera imaging, and these geometric model parameters are camera parameters. Under most conditions, these camera parameters must be obtained through experiments and calculations, and this process of solving the parameters is called camera calibration. Whether in image measurement or machine vision applications, the calibration of camera parameters is a very crucial link, and the accuracy of its calibration results and the stability of the algorithm directly affect the accuracy of the results generated by the camera operation.

[0080] Camera calibration usually involves four types of coordinate systems: world coordinate system, camera coordinate system, imaging plane coordinate system, and pixel coordinate system. The following combines Figures 1-3 to illustrate the projection transformation between coordinate systems.

[0081] Figure 1 is a schematic diagram of the positions of the camera coordinate system and the imaging plane coordinate system. Figure 2 is a schematic diagram of the projection of the camera coordinate system and the imaging plane coordinate system. Figure 3 is a schematic diagram of the relationship between the imaging plane coordinate system and the pixel coordinate system.

[0082] The camera and the object to be photographed can be placed at any position in the environment. Thus, it is necessary to establish a coordinate system in the environment to represent the relative positions between the camera and the object to be photographed. This coordinate system is called the world coordinate system (Ow-Xw-Yw-Zw, Figures 1-3 not shown in ), where Ow is the origin, and Xw, Yw, and Zw are the mutually perpendicular horizontal axis, vertical axis, and longitudinal axis, and their measurement values are usually in meters. The world coordinate system can be arbitrarily selected, which is a hypothetical coordinate system. Once specified, it remains unchanged and unique, that is, the absolute coordinate system.

[0083] As Figure 1 shown, the camera coordinate system (Oc-Xc-Yc-Zc) takes the optical center Oc of the camera as the origin, the Xc axis as the horizontal axis, the Yc axis as the vertical axis, and the Zc axis points in the direction that the camera observes along the optical axis of the camera lens (i.e., perpendicular to the imaging plane). Its measurement values are usually in meters and change with the movement of the camera, which is a relative coordinate system.

[0084] As Figure 1 shown, the imaging plane coordinate system (o-x-y): also called the plane coordinate system, is a transitional coordinate system introduced to derive the pixel coordinate system. The origin is the intersection point o of the optical axis of the camera lens and the imaging plane. The x-axis and the y-axis are respectively parallel to the Xc axis and the Yc axis of the camera coordinate system. It is a two-dimensional plane rectangular coordinate system. Its measurement values are usually in meters and rely on the camera coordinate system, which is also a relative coordinate system. Among them, the distance between the optical center Oc of the camera and the origin o of the imaging plane coordinate system is the focal length f of this camera.

[0085] As Figure 3 shown, the pixel coordinate system (o’-u-v) is a plane rectangular coordinate system fixed on the imaging plane with pixels as the unit. Its origin o’ is usually located at a corner of the imaging plane, such as the lower left corner. The horizontal axis u and the vertical axis v are respectively parallel to the x-axis and the y-axis of the imaging plane coordinate system. Its measurement values are usually the number of pixels and rely on the imaging plane coordinate system, which is also a relative coordinate system.

[0086] Camera calibration, simply put, is the process of projecting from the world coordinate system to the pixel coordinate system, that is, the process of finding the final projection matrix. Generally, the process of coordinate system projection is divided into three steps: The first step is to convert from the world coordinate system to the camera coordinate system, which is a projection from a three-dimensional point to a three-dimensional point; the second step is to convert from the camera coordinate system to the imaging plane coordinate system, which is a projection from a three-dimensional point to a two-dimensional point; the third step is to convert from the imaging plane coordinate system to the pixel coordinate system, which is a projection from a two-dimensional point to a two-dimensional point.

[0087] The following is a specific description of the coordinate system projection transformation.

[0088] The first step is to convert from the world coordinate system to the camera coordinate system. This transformation process only involves the rotation and translation of the three-dimensional coordinate system and can be achieved through the following first relational expression:

[0089]

[0090] R is the overall rotation projection matrix, and the expression is R = R x R y R z . Among them, R x , R y , R z respectively represent the rotation projection matrices around the X-axis, Y-axis, and Z-axis.

[0091] T is the translation projection matrix:

[0092] Among them, T x , T y , T z respectively represent the translation amounts along the X-axis, Y-axis, and Z-axis.

[0093] The above R and T respectively reflect the pose of the camera relative to the world coordinate system, that is, the position and attitude of rotation and translation. Therefore, R and T together constitute the external parameter matrix of the camera.

[0094] Thus, according to the first relational expression, when the external parameter matrices R and T of the camera and the coordinates of a certain point in the world coordinate system are known, the coordinates of the corresponding projection point of this point in the camera coordinate system can be obtained. Conversely, when the coordinates of a certain point in the world coordinate system and the coordinates of the corresponding projection point of this point in the camera coordinate system are known, the external parameter matrices R and T of this camera can be obtained.

[0095] The second step is to convert from the camera coordinate system to the imaging plane coordinate system, which involves a projection from a three-dimensional point to a two-dimensional point.

[0096] As Figure 2 shown, the point B(X c Y c Z c), and the projection point of the corresponding imaging plane is point P(x, y). According to the similar triangle theorem, it can be known that:

[0097]

[0098] Writing it in homogeneous coordinate form gives the second relation:

[0099] The third step is to convert from the imaging plane coordinate system to the pixel coordinate system, which is a projection from a two-dimensional point to a two-dimensional point.

[0100] As Figure 3 shown, o is the origin of the imaging plane coordinate system, but its position in the pixel coordinate system is (u0, v0). Here, the origin of the pixel coordinate system is at the lower left corner of the imaging plane.

[0101] It should be noted that the imaging plane coordinate system is measured in meters, while the pixel coordinate system is measured in the number of pixels. Here, a new concept needs to be introduced: d x represents the number of pixels contained in a length distance of 1m in the x-axis direction; d y represents the number of pixels contained in a length distance of 1m in the y-axis direction.

[0102] From this, it can be obtained that:

[0103] Writing it in homogeneous coordinate form gives the third relation:

[0104] Furthermore, based on the second relation and the third relation, it can be obtained that:

[0105]

[0106] Furthermore, the fourth relation can be obtained:

[0107] Among them, let

[0108] K is the internal parameter matrix of the camera.

[0109] Thus, according to the fourth relation, when the internal parameter matrix K of the camera and the coordinates of a certain point in the camera coordinate system are known, the coordinates of the corresponding projection point of this point in the pixel coordinate system can be obtained. Or, when the coordinates of a certain point in the camera coordinate system and the coordinates of the corresponding projection point of this point in the pixel coordinate system are known, the internal parameter K of this camera can be obtained.

[0110] In summary, according to the first relation and the fourth relation, there are a total of four variables, namely the external parameter matrix (R, T) of the camera, the internal parameter matrix K, the coordinates of a point in the world coordinate system, and the coordinates of the projection point corresponding to the point in the pixel coordinate system. Only by obtaining the values of three of these variables can the value of the fourth variable be deduced.

[0111] (Step S1)

[0112] This embodiment provides a method for evaluating the accuracy of camera internal parameters, which will be specifically described below in conjunction with Figures 4-5 for illustration. Figure 4 is a flowchart for evaluating the accuracy of camera internal parameters. Figure 5 is a schematic diagram of the appearance of the calibration component.

[0113] In step S1, a calibration component 1 with multiple feature points as shown in Figure 5 is set. The multiple feature points are located on one side of the calibration component 1 facing the camera to be measured (not shown in the figure), and the camera to be measured is the camera whose internal parameter accuracy is to be evaluated.

[0114] In a general example, the calibration component 1 is an object with a calibration pattern, such as a single calibration board, or can be composed of multiple calibration boards. Generally speaking, the more the number of calibration boards, the higher the calibration accuracy of the camera parameters. Therefore, in the specific implementation process, the implementer can determine the number of the calibration components according to the actual conditions or the calibration accuracy requirements, and this application does not make any limitations in this regard. For simplicity, only Figure 5 the calibration component 1 composed of the first calibration board 11, the second calibration board 12, and the third calibration board 13 shown in the figure will be taken as an example for illustration.

[0115] On the plate surface of the first calibration board 11, the second calibration board 12, and the third calibration board 13 facing the camera to be measured, there are calibration patterns for calibrating the camera parameters, such as Figure 5 the solid circles shown in the figure. The actual shape of the pattern can be various, such as a checkerboard, and no limitation is made here. There are multiple feature points in the calibration pattern, such as the center 121 of the solid circle.

[0116] In actual applications, feature points usually have certain significant features in the image, such as local maximum or minimum gray levels, certain gradient features, etc. For example, the corner points formed by the intersections of the black and white checkerboard grids, where two line segments intersect at the corner points, and there are also large changes in the image gradient here. Therefore, they can be easily detected and located in the image. In other words, feature points usually have two-dimensional features that can be accurately located, so the efficiency and accuracy of detection and analysis can be improved. For simplicity, this application only takes Figure 5 the center 121 shown in the figure as an example for illustration.

[0117] Due to the existence of the center point 121 of the above-mentioned feature points, in image processing and computer vision, we no longer need to observe the entire image, but select some center points 121 of the feature points in the image. As long as a sufficient number of the above-mentioned center points 121 of the feature points are detected and then accurately positioned and modeled and analyzed, it can lay a good foundation for calibrating the camera parameters.

[0118] In addition, as an example, there is a certain angle between the first calibration plate 11, the second calibration plate 12, and the third calibration plate 13, and the calibration patterns on the three calibration plates are all within the shooting field of view of the camera to be measured, so that the center points 121 of the feature points can be stably extracted from all three calibration plates.

[0119] Compared with the scheme of a single calibration plate, the calibration assembly 1 is composed of multiple calibration plates at different angles. On the one hand, it is no longer necessary to manually change the angle of a single calibration plate to achieve shooting of calibration plates at different angles, overcoming the problem of complex operation and improving the efficiency of camera calibration. On the other hand, compared with the two-dimensional information of a single calibration plate, the center points 121 of the feature points on the calibration assembly 1 add depth information, which can further improve the accuracy of camera calibration and accuracy inspection.

[0120] (Step S2)

[0121] Next, in step S2, a world coordinate system (Ow-Xw-Yw-Zw) is constructed in the three-dimensional space where the calibration assembly 1 is located, and the three-dimensional coordinates of each feature point among the multiple feature points in the world coordinate system are obtained.

[0122] As an example, the three-dimensional reconstruction of the present application adopts the Structure from Motion (SFM) method. Through a monocular camera with known high-precision internal parameters, multiple images containing multiple center points 121 of the feature points are taken at different positions. By matching the multiple feature points in these multiple images, the corresponding relationship between the feature points of the images from different perspectives is established; then, according to the principles of multi-view geometry, the three-dimensional coordinates of these feature points and the pose parameters of the camera are optimized and calculated.

[0123] Structure from Motion (SFM), that is, determining the spatial and geometric relationships of the target by the movement of the camera, is a common method for three-dimensional reconstruction. Compared with 3D cameras, it only requires an ordinary RGB camera to achieve three-dimensional reconstruction, with lower costs and less environmental constraints.

[0124] (Step S3)

[0125] Next, in step S3, an imaging plane coordinate system is constructed on the imaging plane of the camera to be tested, a first feature image including a plurality of feature point centers 121 is captured by the camera to be tested at a first shooting position, and a first projection coordinate of each feature point in the plurality of feature point centers 121 in the imaging plane coordinate system is extracted from the first feature image. A detailed description is given below.

[0126] First, the captured first feature image containing multiple feature point centers 121 is converted into a first grayscale image. By graying the image, unimportant color information can be removed, and the gradient information that plays a key role in identifying the image can be retained, thereby simplifying the recognition process and improving the operation speed.

[0127] Secondly, the first grayscale image is converted into a first binary image. By binarizing the image, the grayscale value of each pixel of the image is set to 0 (black) or 255 (white), that is, the entire image is presented as only black and white, which greatly reduces the amount of data in the image, thereby highlighting the outline of the target.

[0128] Next, a plurality of contours included in the first binary image are obtained, and each contour in the plurality of contours is associated with each feature point circle center 121 in a one-to-one correspondence.

[0129] Finally, according to each of the multiple contours, the first projection coordinates of the center 121 of each associated feature point are calculated.

[0130] As an embodiment, since there is a certain angle between the calibration plate and the lens optical axis of the camera to be tested, the solid circle on the calibration plate appears as an ellipse when projected onto the imaging plane of the camera to be tested.

[0131] At this time, the elliptical contour corresponding to the solid circle on the imaging plane can be detected based on the canny edge detection algorithm, and the curvature of a single contour can be judged to filter out the interfering contour. The reason for the curvature judgment is that due to the influence of the digitization error and noise of the image, the contour line obtained after the image is edge detected and contour extracted is often not smooth. Therefore, before extracting the contour feature points, it is necessary to smooth the curve by Gaussian smoothing or other methods, or directly remove the contour with too large curvature.

[0132] After that, the minimum circumscribed rectangle of the single ellipse contour is calculated, and the single ellipse image is projected according to the minimum circumscribed rectangle, that is, the ellipse on the imaging plane is projected into a perfect circle, the coordinates of the center of the perfect circle are obtained, and the perfect circle is then inversely projected back into an ellipse, and the coordinates of the center of the perfect circle are projected to obtain the center coordinates of the ellipse, which are the first projection coordinates of the center of the feature point 121 on the imaging plane. Through the above-mentioned projection transformation from ellipse to circle, the eccentricity error caused by the ellipse can be reduced.

[0133] (Step S4)

[0134] As an implementation manner, in step S4, after the camera under test finishes shooting at the first shooting position, it is translated to the second shooting position, and continues to shoot the calibration component 1 to obtain a second feature image including the centers 121 of multiple feature points, and extracts the second projection coordinates of each feature point among the centers 121 of multiple feature points in the second coordinate system from the second feature image.

[0135] The method for extracting the second projection coordinates from the second feature image is the same as the method for extracting the first projection coordinates in the foregoing step S3, and will not be elaborated here.

[0136] As an example, the camera under test is fixedly installed on a jig, and the jig is designed according to the structure of the camera housing to meet the anti-fooling requirements. This anti-fooling design enables the operator to directly and correctly complete the correct operation without spending too much attention, experience, and professional knowledge by using a limiting method to avoid errors.

[0137] The jig is installed on a bracket, and the bracket is fixed to the side of the calibration component 1 facing the camera under test. First shooting positions and second shooting positions with known distances are set on the bracket. The jig stops at the first shooting position and the second shooting position in sequence for the camera under test to take pictures.

[0138] Through the above translation operation, the external parameters of the camera under test can be simplified, and the operation efficiency can be improved. In fact, the movement of the camera under test can be more diverse, but this will inevitably increase the complexity of the operation.

[0139] (Steps S5, S6)

[0140] Next, in step S5, according to the pre-established projection relationship from the first coordinate system to the second coordinate system, the internal parameters of the camera under test obtained in advance, the three-dimensional coordinates corresponding to each feature point among the centers 121 of multiple feature points, the first projection coordinates, and the second projection coordinates, the first external parameters and the second external parameters of the camera under test corresponding to the first shooting position and the second shooting position are deduced.

[0141] In step S6, according to the first external parameters and the second external parameters, the estimated distance between the first shooting position and the second shooting position is deduced.

[0142] The following is a specific description of the derivation process.

[0143] As an implementation, the initial pose of the camera to be measured can be adjusted so that the horizontal and vertical axes of the camera coordinate system are parallel to the horizontal and vertical axes of the world coordinate system respectively, and the vertical axis of the camera coordinate system coincides with the vertical axis of the world coordinate system and has the same direction, preferably along the optical axis of the camera to be measured towards the object to be photographed.

[0144] As a preferred method, during the movement from the first shooting position to the second shooting position, the camera to be measured only translates along the Z-axis. Assuming that the coordinate of the first shooting position on the Z-axis of the world coordinate system is t1 and the coordinate of the second shooting position on the Z-axis of the world coordinate system is t2, then the translation matrices of the camera to be measured at the two shooting positions relative to the world coordinate system are respectively

[0145]

[0146] At the same time, since the camera to be measured does not perform rotational movement during the process from the first shooting position to the second shooting position, its rotation matrix relative to the world coordinate system is a unit matrix, that is:

[0147]

[0148] At this time, the internal parameter matrix K of the camera to be measured is pre-calibrated. The three-dimensional coordinates of each feature point among the multiple feature point centers 121 are obtained through the three-dimensional reconstruction in step S2, and the first projection coordinates and the second projection coordinates corresponding to the first shooting position and the second shooting position of each feature point among the multiple feature point centers 121 are obtained through steps S4 and S5.

[0149] On this basis, according to the foregoing first relation and fourth relation, the first external parameters (R, T1) and the second external parameters (R, T2) of the camera to be measured at the first shooting position and the second shooting position can be deduced.

[0150] In this embodiment, since the rotation matrix R is a unit matrix, the translation matrices T1 and T2 of the camera to be measured can be deduced from the first external parameters (R, T1) and the second external parameters (R, T2), and then the coordinate values t1 and t2 can be obtained. The difference between the two can be used to obtain the estimated distance △t between the two shooting positions.

[0151] (Step S7)

[0152] Next, in step S7, the actual distance △t' between the first shooting position and the second shooting position is pre-obtained, and the accuracy of the internal parameters of the camera to be measured is evaluated according to the difference between the estimated distance △t and the actual distance △t'.

[0153] As an example, a precision threshold can be preset to determine whether the difference between the calculated spacing △t and the actual spacing △t' is less than the precision threshold. If it is satisfied, it is considered that the current internal parameter calibration result meets the precision requirements and can be used normally. If the difference between the calculated spacing △t and the actual spacing △t' is greater than the precision threshold, it is considered that the current internal parameter calibration result does not meet the precision requirements, and the internal parameters of the camera to be measured need to be recalibrated, and the precision verification is carried out again according to the steps of S1 - S7 above.

[0154] (Undistortion and error point removal)

[0155] Due to the manufacturing precision of the lens and the deviation of the assembly process, the images captured by the camera will be distorted, that is, image distortion. Image distortion is mainly divided into radial distortion and tangential distortion. Radial distortion is caused by the inherent characteristics of the convex lens of the camera. The reason is that light is more bent at places far from the center of the lens than at places close to the center. Tangential distortion is caused by the non - parallelism between the lens itself and the camera imaging plane, and this situation is mostly due to the installation deviation when the lens is installed on the lens module.

[0156] Preferably, after the camera to be measured captures the characteristic image of the calibration component 1, the characteristic image is undistorted.

[0157] In addition, in a real application scenario, we match the center 121 of the feature point and its projection point on the camera imaging plane into a one - to - one corresponding point pair. Inevitably, such point pairs will contain errors. For example, the position of the point deviates by several pixels, or even the phenomenon of incorrect matching of point pairs occurs. Therefore, it is also necessary to remove the error points of incorrect matching.

[0158] The following combines Figure 6 to specifically illustrate the method of removing error points. Figure 6 is a schematic diagram of the circular coding points.

[0159] As Figure 6 shown, a plurality of circular coding points 2 are arranged on the side plate surface of the calibration component 1 facing the camera to be measured. The circular coding points 2 include 1 central circle 23, and the central circle 23 is mainly used for the positioning of the circular coding points 2. Around the central circle 23 are concentric, segmented coding bands (21, 22), and the coding bands (21, 22) are used to determine the coding information of the circular coding points 2, mainly for the identification of the circular coding points 2. Among them, the coding bands (21, 22) are further divided into bright coding bands 21 and dark coding bands 22. In fact, there are various arrangements of the bright coding bands 21 and dark coding bands 22, not limited to the Figure 6 style shown in Figure 6 For simplicity, only

[0160] Preferably, the circular coding point 2 is made of a reflective material, and its reflectivity is much stronger than that of the background board. Thus, by setting a gray threshold, the interference of miscellaneous points on the background board can be effectively removed.

[0161] As an example, the specific coding rule is that the segmented coding bands (21, 22) are evenly divided into Figure 6 the 8 parts shown, and each part can be made into a bright band 21 or a dark band 22, and the corresponding binary code is 1 or 0; starting from Figure 6 the first bit 24 shown as the start and the last bit 25 shown as the end and reading counterclockwise, the binary code 00001011 can be obtained, and converting it to decimal is 11, which is the coding information corresponding to the circular coding point 2.

[0162] By positioning and decoding multiple circular coding points 2, and using the unique coding information of each circular coding point 2 itself, high-precision matching between multiple circular coding points 2 in multiple images can be achieved.

[0163] Next, return to the method for removing error points. It mainly includes the following steps:

[0164] S51: On the surface of the calibration component 1 facing the camera to be measured, set multiple circular coding points 2, and obtain the coded three-dimensional coordinates corresponding to each circular coding point 2 in the first coordinate system;

[0165] S52: Use the camera to be measured to take a coded image including each circular coding point 2 and the centers 121 of multiple feature points, and extract the coded projection coordinates corresponding to each circular coding point 2 in the second coordinate system and the actual feature projection coordinates corresponding to each center 121 of the feature points in the second coordinate system from the coded image;

[0166] S53: According to the pre-established projection relationship from the first coordinate system to the second coordinate system, the internal parameters of the camera to be measured obtained in advance, the coded three-dimensional coordinates and coded projection coordinates corresponding to each circular coding point 2, deduce the preliminary external parameters of the camera to be measured;

[0167] S54: According to the pre-established projection relationship from the first coordinate system to the second coordinate system, the internal parameters and preliminary external parameters of the camera to be measured, and the three-dimensional coordinates of each feature point among multiple centers 121 of feature points, deduce the estimated projection coordinates corresponding to each feature point among multiple centers 121 of feature points on the coded image;

[0168] S55: Calculate the Euclidean distance between the estimated projection coordinates and the corresponding actual feature projection coordinates of each feature point among multiple centers 121 of feature points. If the Euclidean distance is less than the preset threshold, determine that the feature point is a valid point; if the Euclidean distance is greater than the preset threshold, determine that the feature point is an error point and remove it.

[0169] (Evaluation System for Intrinsic Parameter Accuracy)

[0170] Next, in combination with Figure 7 an evaluation system for the accuracy of the intrinsic parameters will be described. Figure 7 is a block diagram of an evaluation system for the accuracy of the intrinsic parameters of a camera module, including:

[0171] A calibration component unit 10 is arranged in front of the object to be photographed by the camera to be measured and has a plurality of feature point centers 121. The plurality of feature point centers 121 are located on one side of the calibration component 1 facing the camera to be measured, and the camera to be measured is the camera to be evaluated for the accuracy of the intrinsic parameters;

[0172] A jig unit 20 fixes the camera to be measured and moves the camera to be measured from the first shooting position to the second shooting position, and respectively shoots a first feature image and a second feature image including a plurality of feature point centers 121;

[0173] A coordinate acquisition unit 30 constructs a first coordinate system in the three-dimensional space where the calibration component 1 is located, and acquires the three-dimensional coordinates of each feature point among the plurality of feature point centers 121 in the first coordinate system; constructs a second coordinate system on the imaging plane of the camera to be measured, and extracts the first projection coordinates and the second projection coordinates of each feature point among the plurality of feature point centers 121 in the second coordinate system from the first feature image and the second feature image;

[0174] A calculation unit 40 derives the first extrinsic parameter and the second extrinsic parameter corresponding to the first shooting position and the second shooting position of the camera to be measured according to the pre-established projection relationship from the first coordinate system to the second coordinate system, the pre-acquired intrinsic parameters of the camera to be measured, the three-dimensional coordinates, the first projection coordinates, and the second projection coordinates corresponding to each feature point among the plurality of feature point centers 121; derives the estimated distance between the first shooting position and the second shooting position according to the first extrinsic parameter and the second extrinsic parameter;

[0175] An evaluation unit 50 pre-acquires the actual distance between the first shooting position and the second shooting position, and evaluates the accuracy of the intrinsic parameters of the camera to be measured according to the magnitude of the difference between the estimated distance and the actual distance.

[0176] In addition, in combination with the evaluation method for the accuracy of the intrinsic parameters of the camera module provided in the above embodiments, a storage medium can also be provided in this embodiment to implement it. A computer program is stored on the storage medium; when the computer program is executed by a processor, the evaluation method for the accuracy of the intrinsic parameters of the camera module in the above embodiments is implemented.

[0177] In one embodiment, a computing device is further provided, and the computing device may be a server. The computing device includes a processor, a memory, a network interface, and a database connected through a system bus. Among them, the processor of the computing device is used to provide computing and control capabilities. The memory of the computing device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computing device is used to store feature point detection data. The network interface of the computing device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, it implements the above method for evaluating the internal parameter accuracy of the camera module.

[0178] It should be understood that the above specific embodiments are only used to explain the present invention, and the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention, according to the technical solution and inventive concept of the present invention, makes changes, substitutions, and combinations, and all should be covered within the protection scope of the present invention.

Claims

1. An evaluation method for the internal parameter accuracy of a camera module, characterized in that, Including: S1: Set a calibration component with multiple feature points, where the multiple feature points are located on the side of the calibration component facing the first camera module, and the first camera module is the camera module whose internal parameter accuracy is to be evaluated; S2: Construct a first coordinate system in the three-dimensional space where the calibration component is located, and obtain the three-dimensional coordinates of each of the multiple feature points in the first coordinate system; S3: Construct a second coordinate system on the imaging plane of the first camera module. Take a first feature image containing the multiple feature points at a first shooting position through the first camera module, and extract the first projection coordinates of each of the multiple feature points in the second coordinate system from the first feature image; S4: Take a second feature image containing the multiple feature points at a second shooting position through the first camera module, and extract the second projection coordinates of each of the multiple feature points in the second coordinate system from the second feature image; S5: According to the pre-established projection relationship from the first coordinate system to the second coordinate system, the pre-acquired internal parameters of the first camera module, the three-dimensional coordinates, the first projection coordinates, and the second projection coordinates corresponding to each of the multiple feature points, deduce the first external parameters and the second external parameters of the first camera module corresponding to the first shooting position and the second shooting position respectively; S6: Deduce the estimated distance between the first shooting position and the second shooting position according to the first external parameter and the second external parameter; S7: Pre-acquire the actual distance between the first shooting position and the second shooting position, and evaluate the accuracy of the internal parameters of the first camera module according to the magnitude of the difference between the estimated distance and the actual distance.

2. The method for evaluating the internal parameter accuracy of the camera module according to claim 1, wherein: After the first camera module finishes shooting at the first shooting position, it is translated to the second shooting position.

3. The evaluation method for the internal parameter accuracy of the camera module according to claim 1, characterized in that, The three-dimensional coordinates of each of the multiple feature points are obtained by the following method: S21: Set a second camera module, pre-acquire its internal parameters with known accuracy, and take multiple feature images containing the multiple feature points through the second camera module; S22: Perform three-dimensional reconstruction on the multiple feature points based on the multiple feature images, and obtain the three-dimensional coordinates of each of the multiple feature points in the first coordinate system.

4. The method for evaluating the internal parameter accuracy of the camera module according to claim 1, wherein The first projection coordinates / second projection coordinates of each of the multiple feature points are obtained by the following method: S41: Convert the taken first feature image / second feature image containing the multiple feature points into a first grayscale image / second grayscale image; S42: Convert the first grayscale image / second grayscale image into a first binary image / second binary image; S43: Obtain multiple contours included in the first binary image / second binary image, and each of the multiple contours is associated with each of the multiple feature points in a one-to-one correspondence; S44: Calculate the first projection coordinates / second projection coordinates of each of the multiple feature points associated with each of the multiple contours.

5. The evaluation method for the internal parameter accuracy of the camera module according to claim 4, characterized in that, Before deriving the external parameters of the first imaging module, it further includes a step of removing error points: S51: On the surface of the calibration component facing the first imaging module, set a plurality of coding points, and obtain the coded three-dimensional coordinates corresponding to each coding point in the first coordinate system; S52: Use the first imaging module to capture a coded image including the plurality of coding points and the plurality of feature points, and extract the coded projection coordinates and actual feature projection coordinates corresponding to each coding point and each feature point in the plurality of coding points and the plurality of feature points respectively in the second coordinate system from the coded image; S53: Derive the preliminary external parameters of the first imaging module according to the pre-established projection relationship from the first coordinate system to the second coordinate system, the pre-obtained internal parameters of the first imaging module, the coded three-dimensional coordinates and the coded projection coordinates corresponding to each coding point in the plurality of coding points; S54: Derive the estimated projection coordinates corresponding to each feature point in the plurality of feature points on the coded image according to the pre-established projection relationship from the first coordinate system to the second coordinate system, the internal parameters of the first imaging module, the preliminary external parameters, and the three-dimensional coordinates of each feature point in the plurality of feature points; S55: Calculate the Euclidean distance between the estimated projection coordinates of each feature point in the plurality of feature points and the corresponding actual feature projection coordinates. If the Euclidean distance is less than a preset threshold, determine that the feature point is a valid point; if the Euclidean distance is greater than the preset threshold, determine that the feature point is an error point and remove it.

6. The method for evaluating the internal parameter accuracy of the imaging module according to claim 1, wherein: Perform distortion correction on each of the multiple feature points on the first feature image and the second feature image.

7. The method for evaluating the internal parameter accuracy of the imaging module according to claim 1, wherein: The calibration component includes a plurality of calibration plates, and the plurality of feature points are distributed on the plate surface of each calibration plate in the plurality of calibration plates facing the first imaging module.

8. An evaluation system for the internal parameter accuracy of a camera module, characterized in that, It includes: A calibration component unit, arranged in front of the first imaging module facing the object to be photographed and having a plurality of feature points, the plurality of feature points being located on the side of the calibration component unit facing the first imaging module, and the first imaging module being the imaging module to be evaluated for internal parameter accuracy; A jig unit, fixing the first imaging module and moving the first imaging module from a first shooting position to a second shooting position to respectively capture a first feature image and a second feature image including the plurality of feature points; A coordinate acquisition unit constructs a first coordinate system in the three-dimensional space where the calibration component is located, and acquires the three-dimensional coordinates of each of the multiple feature points in the first coordinate system; constructs a second coordinate system on the imaging plane of the first imaging module, and extracts the first projection coordinates and the second projection coordinates of each of the multiple feature points in the second coordinate system from the first feature image and the second feature image; A calculation unit derives the first external parameter and the second external parameter of the first imaging module corresponding to the first shooting position and the second shooting position according to the pre-established projection relationship from the first coordinate system to the second coordinate system, the pre-acquired internal parameters of the first imaging module, the three-dimensional coordinates, the first projection coordinates, and the second projection coordinates corresponding to each of the multiple feature points; Derive the estimated distance between the first shooting position and the second shooting position according to the first external parameter and the second external parameter; An evaluation unit pre-acquires the actual distance between the first shooting position and the second shooting position, and evaluates the accuracy of the internal parameters of the first imaging module according to the magnitude of the difference between the estimated distance and the actual distance.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method for evaluating the accuracy of the internal parameters of the imaging module according to any one of claims 1 to 7.

10. A computing device, characterized in that, The computing device includes: A processor; A memory storing a computer program, which when executed by the processor, implements the method for evaluating the accuracy of the internal parameters of the imaging module according to any one of claims 1 to 7.

Images