A camera distortion correction effect evaluation method, device, medium and equipment

Abstract

This invention discloses a method, apparatus, medium, and device for evaluating camera distortion correction effects. The method includes: performing affine transformation processing on a target object image after distortion removal, ensuring the target object is within the image range and that the feature points in the middle and edge portions of the processed target object image have consistent scales; detecting all feature points in the processed target object image and performing line fitting on feature points located on the same straight line to obtain the straight line containing each feature point; calculating the distance error from the feature point to the straight line containing the feature point for each feature point, and converting the distance error corresponding to the feature point into an angular error from the feature point to the principal point of the camera lens; and evaluating the distortion correction effect based on the angular errors corresponding to all feature points. This invention, by converting the straightness error from distance error to angular error, enables the use of a unified evaluation standard, thereby achieving a unified evaluation of the camera distortion correction effect.

Description

Technical Field

[0001] This invention relates to the field of machine vision correction technology, and in particular to a method, apparatus, computer-readable storage medium, and terminal device for evaluating the effect of camera distortion correction. Background Technology

[0002] In camera calibration, calibrating the camera's distortion parameters is a crucial part of the calibration process, and it's also necessary to evaluate the calibration error of these parameters, i.e., to assess the camera's distortion correction effectiveness. Existing technology provides an evaluation method that involves first capturing an image containing a calibration template, then using the calibrated distortion parameters to remove distortion from the image, extracting all feature points from the removed image, and finally fitting a straight line to each feature point based on the features points that should theoretically be on the same line. The vertical distance between each feature point and its corresponding line is calculated, and the straightness error is statistically analyzed based on this vertical distance to assess whether the distortion correction effect meets the requirements.

[0003] However, the above scheme uses straightness error measured in absolute distance. As the shooting distance and shooting angle between the camera and the calibration template change, the straightness error will also change. For example, in cases of large distortion and large FOV (Field of View) (such as with a fisheye camera), the central part of the image is shrunk, making the straightness error value smaller accordingly. At the same time, the distortion at the edge of the image is generally more severe and is the part of interest in evaluating the distortion correction effect. However, if the image is directly dedistorted, the edge part will be out of range, and it is impossible to obtain the straightness error corresponding to the edge part. It can be seen that the straightness error in the above scheme cannot be used as a unified evaluation standard and cannot uniformly evaluate the distortion correction effect under different conditions. Summary of the Invention

[0004] The purpose of this invention is to provide a method, apparatus, computer-readable storage medium, and terminal device for evaluating camera distortion correction effects. By converting straightness error from distance error to angle error, a unified evaluation standard can be used to achieve a unified evaluation of camera distortion correction effects.

[0005] To achieve the above objectives, embodiments of the present invention provide a method for evaluating the effect of camera distortion correction, including:

[0006] The target object image is captured using a camera; wherein the target object image includes several extractable feature points, and the feature points are located on the same straight line or multiple straight lines;

[0007] After distortion removal, the target image is subjected to affine transformation to ensure that the target object is within the image range and that the feature points in the middle and edge parts of the processed target image have the same scale.

[0008] Detect all feature points in the processed target image, and perform line fitting on feature points located on the same straight line to obtain the straight line where each feature point is located;

[0009] For each feature point, calculate the distance error from the feature point to the line containing the feature point;

[0010] For each feature point, the distance error corresponding to the feature point is converted into the angle error from the feature point to the principal point of the camera lens;

[0011] The distortion correction effect is evaluated based on the angular errors corresponding to all feature points.

[0012] Furthermore, for each feature point, converting the distance error corresponding to the feature point into the angular error from the feature point to the principal point of the camera lens specifically includes:

[0013] The coordinates of the camera in the world coordinate system are calculated.

[0014] For each feature point, according to the formula The distance error corresponding to the feature point is converted into the angle error from the feature point to the principal point of the camera lens; where θ represents the angle error corresponding to the feature point, d represents the distance error corresponding to the feature point, (u, v, 0) represents the coordinates of the feature point in the world coordinate system, and (x, y, z) represents the coordinates of the camera in the world coordinate system.

[0015] Furthermore, the calculation of the camera's coordinates in the world coordinate system specifically includes:

[0016] According to the formula The coordinates of the camera in the world coordinate system are calculated; where R represents the rotation matrix in the calibration extrinsic parameters of the camera, and T represents the translation vector in the calibration extrinsic parameters of the camera.

[0017] Furthermore, the evaluation of the distortion correction effect based on the angular errors corresponding to all feature points specifically includes:

[0018] Statistical analysis is performed on the angle errors corresponding to all feature points to determine the average and / or maximum angle errors;

[0019] The distortion correction effect of the camera is evaluated based on the average and / or maximum angle error; wherein, the larger the average angle error, the worse the distortion correction effect; the larger the maximum angle error, the worse the distortion correction effect.

[0020] Further, the step of detecting all feature points in the processed target image and performing line fitting on feature points located on the same straight line to obtain the straight line containing each feature point specifically includes:

[0021] Feature point detection is performed on the processed target image to obtain all feature points in the processed target image;

[0022] The least squares method is used to fit the feature points located on the same straight line to obtain the straight line where each feature point is located; where the feature points located on the same straight line correspond to the points on the target object being photographed that are located on the same straight line.

[0023] Furthermore, the target image is a checkerboard image; therefore,

[0024] The process involves detecting all feature points in the processed target image and performing line fitting on feature points located on the same straight line to obtain the straight line containing each feature point. Specifically:

[0025] Detect all feature points in the processed checkerboard image, and perform line fitting on feature points located on the same straight line to obtain the horizontal and vertical straight lines of each feature point.

[0026] For each feature point, the distance error from the feature point to the line containing the feature point is calculated, specifically as follows:

[0027] For each feature point, calculate the first distance error from the feature point to the horizontal line containing the feature point, and the second distance error from the feature point to the vertical line containing the feature point.

[0028] For each feature point, the distance error corresponding to the feature point is converted into the angular error from the feature point to the principal point of the camera lens, specifically as follows:

[0029] For each feature point, the first distance error corresponding to the feature point is converted into the first angle error from the feature point to the principal point of the camera lens, and the second distance error corresponding to the feature point is converted into the second angle error from the feature point to the principal point of the camera lens.

[0030] The distortion correction effect is evaluated based on the angle errors corresponding to all feature points, specifically as follows:

[0031] The distortion correction effect is evaluated based on the first and second angle errors corresponding to all feature points.

[0032] Furthermore, the evaluation of the distortion correction effect based on the first and second angular errors corresponding to all feature points specifically includes:

[0033] Statistical analysis is performed on the first and second angle errors corresponding to all feature points to determine the average and / or maximum angle errors;

[0034] The distortion correction effect of the camera is evaluated based on the average and / or maximum angle error; wherein, the larger the average angle error, the worse the distortion correction effect; the larger the maximum angle error, the worse the distortion correction effect.

[0035] To achieve the above objectives, embodiments of the present invention also provide a camera distortion correction effect evaluation device, used to implement the camera distortion correction effect evaluation method described in any of the above claims, the device comprising:

[0036] The target image acquisition module is used to capture images of the target object using a camera; wherein the target image includes several extractable feature points, and the feature points are located on the same straight line or multiple straight lines;

[0037] The distortion correction and affine transformation processing module is used to perform affine transformation processing on the target image after distortion correction, so that the target object is within the image range and the feature points of the middle part and the edge part of the processed target image have the same scale.

[0038] The line fitting module is used to detect all feature points in the processed target image and perform line fitting on feature points located on the same straight line to obtain the straight line where each feature point is located.

[0039] The distance error calculation module is used to calculate the distance error from each feature point to the line containing the feature point.

[0040] An angle error calculation module is used to convert the distance error corresponding to each feature point into the angle error from the feature point to the lens principal point of the camera.

[0041] The distortion correction effect evaluation module is used to evaluate the distortion correction effect based on the angular errors corresponding to all feature points.

[0042] This invention also provides a computer-readable storage medium, which includes a stored computer program; wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to perform the camera distortion correction effect evaluation method described above.

[0043] This invention also provides a terminal device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor implements the camera distortion correction effect evaluation method described in any of the above embodiments when executing the computer program.

[0044] Compared with existing technologies, this invention provides a method, apparatus, computer-readable storage medium, and terminal device for evaluating camera distortion correction effects. First, a target object image is captured using a camera. The target object image includes several extractable feature points, and these feature points are located on the same or multiple straight lines. The target object image is then distorted and subjected to affine transformation processing to ensure the target object is within the image range, and the feature points in the middle and edge parts of the processed target object image have consistent scale. Next, all feature points in the processed target object image are detected, and straight line fitting is performed on feature points located on the same straight line to obtain the straight line containing each feature point. For each feature point, the distance error from the feature point to the straight line containing the feature point is calculated, and this distance error is converted into an angular error from the feature point to the principal point of the camera lens. Finally, the distortion correction effect is evaluated based on the angular errors corresponding to all feature points. This invention, by converting straightness error from distance error to angular error, enables the use of a unified evaluation standard, thereby achieving a unified evaluation of camera distortion correction effects. Attached Figure Description

[0045] Figure 1 This is a flowchart of a preferred embodiment of a camera distortion correction effect evaluation method provided by the present invention;

[0046] Figure 2 This is a schematic diagram illustrating the conversion of distance error into angle error provided in an embodiment of the present invention;

[0047] Figures 3A to 3B This is a schematic diagram of a chessboard image provided in an embodiment of the present invention;

[0048] Figure 4 This is a structural block diagram of a preferred embodiment of a camera distortion correction effect evaluation device provided by the present invention;

[0049] Figure 5 This is a structural block diagram of a preferred embodiment of a terminal device provided by the present invention. Detailed Implementation

[0050] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0051] This invention provides a method for evaluating the effect of camera distortion correction. See [link to relevant documentation]. Figure 1 The diagram shown is a flowchart of a preferred embodiment of a camera distortion correction effect evaluation method provided by the present invention, the method comprising steps S11 to S16:

[0052] Step S11: Take an image of the target object using a camera; wherein the image of the target object includes several extractable feature points, and the feature points are located on the same straight line or multiple straight lines;

[0053] Step S12: After the target image is distorted, an affine transformation is performed to ensure that the target object is within the image range and that the feature points in the middle and edge parts of the processed target image have the same scale.

[0054] Step S13: Detect all feature points in the processed target image, and perform line fitting on feature points located on the same straight line to obtain the straight line where each feature point is located;

[0055] Step S14: For each feature point, calculate the distance error from the feature point to the line containing the feature point;

[0056] Step S15: For each feature point, convert the distance error corresponding to the feature point into the angle error from the feature point to the principal point of the camera lens;

[0057] Step S16: Evaluate the distortion correction effect based on the angle errors corresponding to all feature points.

[0058] Specifically, first, a target object is selected as the target. An image containing the target object is captured using a camera whose parameters have been calibrated (including intrinsic, extrinsic, and distortion parameter calibration), thus obtaining a target object image. This target object image contains multiple extractable feature points, and these feature points lie on the same or multiple straight lines. Next, the obtained target object image is subjected to distortion correction and affine transformation using the camera's calibration parameters, resulting in a processed target object image. This processed image ensures that the viewpoint of the target object is directly facing the target object plane, guaranteeing that the target object is within the image range, and that the feature points in the middle and edge parts of the processed target object image maintain the same scale. Then… Next, all feature points in the processed target image are detected, and straight lines are fitted to feature points that should theoretically be on the same straight line, thus obtaining the straight line (i.e., the fitted line) for each feature point. Then, for each feature point, the distance error (i.e., straightness error, or the perpendicular distance from the feature point to the fitted line) is calculated. For each feature point, the calculated distance error is converted into the angle error from the feature point to the principal point of the camera lens. Finally, the distortion correction effect of the camera is evaluated based on the angle errors corresponding to all the feature points.

[0059] It should be noted that, in the embodiments of the present invention, the camera parameter calibration method can be implemented using the calibration scheme provided by the prior art. Furthermore, the distortion correction and affine transformation processing of the target object image can be performed using the camera calibration parameters (camera intrinsic parameters, rotation matrix, and translation vector), or can be implemented using the distortion correction and affine transformation scheme provided by the prior art. The embodiments of the present invention do not impose specific limitations.

[0060] For example, there are two options for the target image: (1) the image used in the camera intrinsic calibration process can be used directly as the target image; (2) as described above, after the camera intrinsic calibration is completed, the calibrated camera can be used to separately acquire the target image for the distortion correction effect evaluation.

[0061] For example, the distortion correction and affine transformation processing of the target image utilizes the intrinsic parameters in the camera calibration parameters. The intrinsic parameters are used for distortion correction processing, while the affine transformation processing requires additional extrinsic parameters from the camera to the target. Depending on the target image used, there are two cases for obtaining the extrinsic parameters: (1) If the image calibrated with the camera intrinsic parameters is used as the target image, the extrinsic parameters will be obtained during the intrinsic parameter calibration process and do not need to be calculated separately; (2) If a new image is taken as the target image, then when specifically executing step S12, the following should be done: detect feature points in the original image, calculate the extrinsic parameters from the camera to the target based on the feature points in the original image, the camera intrinsic parameters, and the actual coordinates of the feature points on the target, and then perform distortion correction and affine transformation processing on the original image using the obtained intrinsic and extrinsic parameters.

[0062] This invention provides a method for evaluating camera distortion correction effectiveness. First, a target image is captured using a camera. The target image includes several extractable feature points located on the same or multiple straight lines. The image is then distorted and subjected to an affine transformation to ensure the target object is within the image's range, and the feature points in the middle and edge portions of the processed image are of consistent scale. Next, all feature points in the processed image are detected, and straight-line fitting is performed on feature points located on the same straight line to obtain the straight line containing each feature point. For each feature point, the distance error from the feature point to the straight line is calculated, and this distance error is converted into an angular error from the feature point to the principal point of the camera lens. Finally, the distortion correction effectiveness is evaluated based on the angular errors corresponding to all feature points. This invention, by converting straightness error from distance error to angular error, enables the use of a unified evaluation standard, thereby achieving a unified evaluation of camera distortion correction effectiveness.

[0063] In another preferred embodiment, the step of converting the distance error corresponding to each feature point into an angular error from the feature point to the principal point of the camera lens specifically includes:

[0064] The coordinates of the camera in the world coordinate system are calculated.

[0065] For each feature point, according to the formula The distance error corresponding to the feature point is converted into the angle error from the feature point to the principal point of the camera lens; where θ represents the angle error corresponding to the feature point, d represents the distance error corresponding to the feature point, (u, v, 0) represents the coordinates of the feature point in the world coordinate system, and (x, y, z) represents the coordinates of the camera in the world coordinate system.

[0066] Specifically, in conjunction with the above embodiments, when converting the distance error corresponding to each feature point into the corresponding angle error, the coordinates of the camera in the world coordinate system can be calculated first; then, for each feature point, the formula can be used... The distance error d corresponding to the feature point is converted into the angle error θ from the feature point to the principal point of the camera lens; where (u, v, 0) represents the coordinates of the feature point in the world coordinate system, and (x, y, z) represents the coordinates of the camera in the world coordinate system.

[0067] For example, see Figure 2 The diagram shown is a schematic representation of the conversion of distance error into angle error according to an embodiment of the present invention. Figure 2 In the diagram, point P(u, v, 0) represents the current feature point on the target (i.e., the feature point on the target image when the target is used as the target), point P' represents the foot of the perpendicular from point P to the line it lies on, and the distance d between point P' and point P is the distance error d from the current feature point to the line it lies on. Point O(x, y, z) represents the principal point of the camera lens (i.e., the coordinates corresponding to the camera in the world coordinate system), (ux) 2 +(vy) 2 +z 2 Let O represent the distance between point O and point P, which is also the distance between the camera and the current feature point. Then we have:

[0068] It should be noted that, in the case of Figure 2 In the triangle formed by points P, P', and O, the parameters should satisfy certain trigonometric function relationships. However, under normal circumstances, the distortion correction effect of a camera after parameter calibration is generally quite good, and the error (i.e., the value of d) will not be large. Therefore, there is no need to use trigonometric function relationships for calculation; d can be directly compared with... Dividing by d yields an approximate angle value, reducing computational complexity in practical applications. Furthermore, even if the error (i.e., the value of d) is large, directly dividing d by d provides a more accurate result. The quotient obtained by division is still monotonic and still meets the evaluation criterion that "the larger θ is, the worse the distortion correction effect".

[0069] It should be noted that the coordinates of each feature point in the world coordinate system can be obtained by detecting the coordinates of each feature point in the world coordinate system when performing feature point detection on the target object image after distortion removal and affine transformation processing. The feature point detection method can be implemented using the detection scheme provided by existing technology, and the embodiments of the present invention do not make specific limitations.

[0070] In yet another preferred embodiment, the calculation of the camera's coordinates in the world coordinate system specifically includes:

[0071] According to the formula The coordinates of the camera in the world coordinate system are calculated; where R represents the rotation matrix in the calibration extrinsic parameters of the camera, and T represents the translation vector in the calibration extrinsic parameters of the camera.

[0072] Specifically, in conjunction with the above embodiments, when actually calculating the coordinates of the camera in the world coordinate system, the rotation matrix R and translation vector T in the camera's calibration extrinsic parameters can be used, according to the formula... The calculation involves multiplying the inverse (or transpose) of the rotation matrix R from the world coordinate system to the camera coordinate system by the translation vector T from the world coordinate system to the camera coordinate system, and then taking the negative of the product to obtain the camera's coordinates (x, y, z) in the world coordinate system.

[0073] In yet another preferred embodiment, the evaluation of the distortion correction effect based on the angular errors corresponding to all feature points specifically includes:

[0074] Statistical analysis is performed on the angle errors corresponding to all feature points to determine the average and / or maximum angle errors;

[0075] The distortion correction effect of the camera is evaluated based on the average and / or maximum angle error; wherein, the larger the average angle error, the worse the distortion correction effect; the larger the maximum angle error, the worse the distortion correction effect.

[0076] Specifically, in conjunction with the above embodiments, when evaluating the distortion correction effect based on the angle errors corresponding to all the obtained feature points, statistical analysis can be performed first on the angle errors corresponding to all the obtained feature points to obtain the average and / or maximum values ​​of the angle errors. Then, the average and / or maximum values ​​of the obtained angle errors can be used to evaluate whether the distortion correction effect of the camera meets the requirements.

[0077] Understandably, the larger the average angle error, the worse the distortion correction effect of the camera; the larger the maximum angle error, the worse the distortion correction effect of the camera. Among them, the average angle error is mainly used to evaluate the overall distortion correction effect, while the maximum angle error is mainly used to evaluate the distortion correction effect of the worst local area.

[0078] In another preferred embodiment, the step of detecting all feature points in the processed target image and performing line fitting on feature points located on the same straight line to obtain the straight line containing each feature point specifically includes:

[0079] Feature point detection is performed on the processed target image to obtain all feature points in the processed target image;

[0080] The least squares method is used to fit the feature points located on the same straight line to obtain the straight line where each feature point is located; where the feature points located on the same straight line correspond to the points on the target object being photographed that are located on the same straight line.

[0081] Specifically, in conjunction with the above embodiments, after obtaining the target image after distortion correction and affine transformation processing, feature point detection can be performed on the processed target image to obtain all feature points in the processed target image; then, the least squares method is used to perform line fitting on the feature points that should theoretically be located on the same straight line to obtain the straight line where each feature point is located; wherein, the feature points located on the same straight line correspond to the points on the actual target object that are photographed and located on the same straight line. In other words, whether the feature points are located on the same straight line can be determined based on whether the points corresponding to the feature points on the actual target object are located on the same straight line.

[0082] In yet another preferred embodiment, the target image is a checkerboard image; therefore,

[0083] The process involves detecting all feature points in the processed target image and performing line fitting on feature points located on the same straight line to obtain the straight line containing each feature point. Specifically:

[0084] Detect all feature points in the processed checkerboard image, and perform line fitting on feature points located on the same straight line to obtain the horizontal and vertical straight lines of each feature point.

[0085] For each feature point, the distance error from the feature point to the line containing the feature point is calculated, specifically as follows:

[0086] For each feature point, calculate the first distance error from the feature point to the horizontal line containing the feature point, and the second distance error from the feature point to the vertical line containing the feature point.

[0087] For each feature point, the distance error corresponding to the feature point is converted into the angular error from the feature point to the principal point of the camera lens, specifically as follows:

[0088] For each feature point, the first distance error corresponding to the feature point is converted into the first angle error from the feature point to the principal point of the camera lens, and the second distance error corresponding to the feature point is converted into the second angle error from the feature point to the principal point of the camera lens.

[0089] The distortion correction effect is evaluated based on the angle errors corresponding to all feature points, specifically as follows:

[0090] The distortion correction effect is evaluated based on the first and second angle errors corresponding to all feature points.

[0091] Specifically, in conjunction with the above embodiments, if a checkerboard target is selected, firstly, an image containing the checkerboard target is captured using a camera whose parameters have been calibrated, thus obtaining a checkerboard image; then, distortion correction and affine transformation processing are performed on the obtained checkerboard image using the camera's calibration parameters, thus obtaining a processed checkerboard image; next, all feature points in the processed checkerboard image are detected, and straight line fitting is performed on feature points that should theoretically be located on the same straight line, thus obtaining the horizontal and vertical straight lines corresponding to each feature point (in the checkerboard target, the fitted straight line corresponding to each feature point includes straight lines in both the horizontal and vertical directions); then... For each feature point, the first distance error from the feature point to the horizontal line containing the feature point and the second distance error from the feature point to the vertical line containing the feature point are calculated. For each feature point, the calculated first distance error is converted into a first angle error from the feature point to the principal point of the camera lens, and the calculated second distance error is converted into a second angle error from the feature point to the principal point of the camera lens. Finally, the distortion correction effect of the camera is evaluated based on the first and second angle errors obtained for all feature points.

[0092] It should be noted that the target selected in this embodiment of the invention is a checkerboard pattern. However, in addition to the checkerboard pattern, any target object with a regular dot pattern or a target object after replacing the dots in the dot pattern with any shape can be used as a target. This embodiment of the invention does not make any specific limitation.

[0093] See Figures 3A to 3B The image shown is a schematic diagram of a chessboard pattern provided in an embodiment of the present invention. Figure 3A This refers to a checkerboard image taken with a camera. Figure 3B This represents a checkerboard image after distortion correction and affine transformation. After distortion correction and affine transformation, the viewpoint of the processed checkerboard image is directly facing the target plane, ensuring that the checkerboard target is within the image range, and that the feature points in the middle and edge parts of the image maintain the same scale.

[0094] exist Figure 3BIn the chessboard, the corner points correspond to the feature points in the chessboard image. Taking the corner point P (i.e. the feature point P) as an example, by performing straight line fitting on the feature points that should theoretically be located on the same straight line, the horizontal line L1 and the vertical line L2 where the feature point P is located are obtained. By calculating the vertical distance from the feature point P to the horizontal line L1, the first distance error can be obtained. And by calculating the vertical distance from the feature point P to the vertical line L2, the second distance error can be obtained.

[0095] It should be noted that, for Figure 3B For the chessboard image shown, theoretically, feature points located on the same straight line are already approximately on the same straight line in the image itself. Therefore, the first distance error and the second distance error corresponding to feature point P are very small. Figure 3B The first and second distance errors are not specifically shown in the document, and Figure 3B Only some feature points located on the horizontal line L1 and the vertical line L2 are shown in the image; not all feature points are shown.

[0096] Combination Figure 2 and Figure 3B As shown, when converting the first distance error and second distance error corresponding to each feature point into the corresponding first angle error and second angle error, similarly to the above embodiment, for each feature point, the formula can be used. The first distance error d1 corresponding to the feature point is converted into the first angular error θ1 from the feature point to the principal point of the camera lens, according to the formula... The second distance error d2 corresponding to the feature point is converted into the second angle error θ2 from the feature point to the principal point of the camera lens; where (u, v, 0) represents the coordinates of the feature point in the world coordinate system, and (x, y, z) represents the coordinates of the camera in the world coordinate system.

[0097] In yet another preferred embodiment, the evaluation of the distortion correction effect based on the first and second angular errors corresponding to all feature points specifically includes:

[0098] Statistical analysis is performed on the first and second angle errors corresponding to all feature points to determine the average and / or maximum angle errors;

[0099] The distortion correction effect of the camera is evaluated based on the average and / or maximum angle error; wherein, the larger the average angle error, the worse the distortion correction effect; the larger the maximum angle error, the worse the distortion correction effect.

[0100] Specifically, in conjunction with the above embodiments, when evaluating the distortion correction effect based on the first angle error and the second angle error corresponding to all the obtained feature points, statistical analysis can be performed first on the first angle error and the second angle error corresponding to all the obtained feature points to obtain the average value and / or maximum value of the angle error. Then, the average value and / or maximum value of the obtained angle error can be used to evaluate whether the distortion correction effect of the camera meets the requirements.

[0101] It should be noted that statistical analysis is performed based on the first and second angle errors corresponding to all the obtained feature points. There is no need to perform statistical analysis separately for all the first angle errors and for all the second angle errors. Instead, the first and second angle errors corresponding to all feature points can be treated as a whole, and the average value can be calculated to find the maximum value.

[0102] This invention also provides a camera distortion correction effect evaluation device for implementing the camera distortion correction effect evaluation method described in any of the above embodiments. See [link to relevant documentation]. Figure 4 The diagram shown is a structural block diagram of a preferred embodiment of a camera distortion correction effect evaluation device provided by the present invention. The device includes:

[0103] The target image acquisition module 11 is used to capture an image of the target object using a camera; wherein the target image includes a number of extractable feature points, and the feature points are located on the same straight line or multiple straight lines;

[0104] The distortion correction and affine transformation processing module 12 is used to perform affine transformation processing on the target image after distortion correction, so that the target object is within the image range and the feature point scale of the middle part and the edge part of the processed target image is consistent.

[0105] The line fitting module 13 is used to detect all feature points in the processed target image and perform line fitting on feature points located on the same straight line to obtain the straight line where each feature point is located.

[0106] The distance error calculation module 14 is used to calculate the distance error from the feature point to the line where the feature point is located for each feature point.

[0107] Angle error calculation module 15 is used to convert the distance error corresponding to each feature point into the angle error from the feature point to the lens principal point of the camera.

[0108] The distortion correction effect evaluation module 16 is used to evaluate the distortion correction effect based on the angular error corresponding to all feature points.

[0109] Preferably, the angle error calculation module 15 specifically includes:

[0110] A camera coordinate calculation unit is used to calculate the coordinates of the camera in the world coordinate system;

[0111] Angle error calculation unit, used for each feature point, according to the formula The distance error corresponding to the feature point is converted into the angle error from the feature point to the principal point of the camera lens; where θ represents the angle error corresponding to the feature point, d represents the distance error corresponding to the feature point, (u, v, 0) represents the coordinates of the feature point in the world coordinate system, and (x, y, z) represents the coordinates of the camera in the world coordinate system.

[0112] Preferably, the camera coordinate calculation unit is specifically used for:

[0113] According to the formula The coordinates of the camera in the world coordinate system are calculated; where R represents the rotation matrix in the calibration extrinsic parameters of the camera, and T represents the translation vector in the calibration extrinsic parameters of the camera.

[0114] Preferably, the distortion correction effect evaluation module 16 specifically includes:

[0115] The first angle error statistical analysis unit is used to perform statistical analysis on the angle errors corresponding to all feature points to determine the average and / or maximum angle errors.

[0116] The first distortion correction effect evaluation unit is used to evaluate the distortion correction effect of the camera based on the average value and / or maximum value of the angle error; wherein, the larger the average value of the angle error, the worse the distortion correction effect; the larger the maximum value of the angle error, the worse the distortion correction effect.

[0117] Preferably, the line fitting module 13 specifically includes:

[0118] The feature point detection unit is used to perform feature point detection on the processed target image to obtain all feature points in the processed target image.

[0119] The line fitting unit is used to perform line fitting on feature points located on the same straight line using the least squares method to obtain the straight line where each feature point is located; wherein, the feature points located on the same straight line correspond to the points on the target object being photographed that are located on the same straight line.

[0120] Preferably, the target image is a checkerboard image; then,

[0121] The line fitting module 13 is specifically used for:

[0122] Detect all feature points in the processed checkerboard image, and perform line fitting on feature points located on the same straight line to obtain the horizontal and vertical straight lines of each feature point.

[0123] The distance error calculation module 14 is specifically used for:

[0124] For each feature point, calculate the first distance error from the feature point to the horizontal line containing the feature point, and the second distance error from the feature point to the vertical line containing the feature point.

[0125] The angle error calculation module 15 is specifically used for:

[0126] For each feature point, the first distance error corresponding to the feature point is converted into the first angle error from the feature point to the principal point of the camera lens, and the second distance error corresponding to the feature point is converted into the second angle error from the feature point to the principal point of the camera lens.

[0127] The distortion correction effect evaluation module 16 is specifically used for:

[0128] The distortion correction effect is evaluated based on the first and second angle errors corresponding to all feature points.

[0129] Preferably, the distortion correction effect evaluation module 16 specifically includes:

[0130] The second angle error statistical analysis unit is used to perform statistical analysis on the first angle error and the second angle error corresponding to all feature points to determine the average and / or maximum angle error.

[0131] The second distortion correction effect evaluation unit is used to evaluate the distortion correction effect of the camera based on the average value and / or maximum value of the angle error; wherein, the larger the average value of the angle error, the worse the distortion correction effect; the larger the maximum value of the angle error, the worse the distortion correction effect.

[0132] It should be noted that the camera distortion correction effect evaluation device provided in the embodiments of the present invention can realize all the processes of the camera distortion correction effect evaluation method described in any of the above embodiments. The functions and technical effects of each module and unit in the device are the same as the functions and technical effects of the camera distortion correction effect evaluation method described in the above embodiments, and will not be repeated here.

[0133] This invention also provides a computer-readable storage medium, which includes a stored computer program; wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to perform the camera distortion correction effect evaluation method described in any of the above embodiments.

[0134] This invention also provides a terminal device, see [link to relevant documentation]. Figure 5 The diagram shown is a structural block diagram of a preferred embodiment of a terminal device provided by the present invention. The terminal device includes a processor 10, a memory 20, and a computer program stored in the memory 20 and configured to be executed by the processor 10. When the processor 10 executes the computer program, it implements the camera distortion correction effect evaluation method described in any of the above embodiments.

[0135] Preferably, the computer program can be divided into one or more modules / units (such as computer program 1, computer program 2, ...), and the one or more modules / units are stored in the memory 20 and executed by the processor 10 to complete the present invention. The one or more modules / units can be a series of computer program instruction segments capable of performing specific functions, and the instruction segments are used to describe the execution process of the computer program in the terminal device.

[0136] The processor 10 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor, or the processor 10 may be any conventional processor. The processor 10 is the control center of the terminal device, connecting various parts of the terminal device through various interfaces and lines.

[0137] The memory 20 mainly includes a program storage area and a data storage area. The program storage area can store the operating system, applications required for at least one function, etc., while the data storage area can store related data, etc. Furthermore, the memory 20 can be a high-speed random access memory, or a non-volatile memory, such as a plug-in hard disk, a smart media card (SMC), a secure digital card (SD), and a flash card, or other volatile solid-state storage devices.

[0138] It should be noted that the aforementioned terminal devices may include, but are not limited to, processors and memory, as will be understood by those skilled in the art. Figure 5 The structural block diagram is merely an example of the terminal device described above and does not constitute a limitation on the terminal device. It may include more or fewer components than shown in the diagram, or combine certain components, or use different components.

[0139] In summary, the camera distortion correction effect evaluation method, apparatus, computer-readable storage medium, and terminal device provided by the embodiments of the present invention first use a camera to capture an image of a target object, the target object image including several extractable feature points, and the feature points located on the same straight line or multiple straight lines; then, after distortion correction, the target object image is subjected to affine transformation processing to ensure that the target object is within the image range, and the feature points in the middle and edge parts of the processed target object image have consistent scales; then, all feature points in the processed target object image are detected, and straight line fitting is performed on the feature points located on the same straight line to obtain the straight line where each feature point is located; for each feature point, the distance error from the feature point to the straight line where the feature point is located is calculated, and the distance error corresponding to the feature point is converted into the angle error from the feature point to the principal point of the camera lens; finally, the distortion correction effect is evaluated based on the angle errors corresponding to all feature points; the embodiments of the present invention, by converting the straightness error from distance error to angle error, can use a unified evaluation standard, thereby achieving a unified evaluation of the camera distortion correction effect.

[0140] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for evaluating the effect of camera distortion correction, characterized in that, include: The target object image is captured using a camera; wherein the target object image includes several extractable feature points, and the feature points are located on the same straight line or multiple straight lines; After distortion removal, the target image is subjected to affine transformation to ensure that the target object is within the image range and that the feature points in the middle and edge parts of the processed target image have the same scale. Detect all feature points in the processed target image, and perform line fitting on feature points located on the same straight line to obtain the straight line where each feature point is located; For each feature point, calculate the distance error from the feature point to the line containing the feature point; For each feature point, the distance error corresponding to the feature point is converted into the angle error from the feature point to the principal point of the camera lens; The distortion correction effect is evaluated based on the angular errors corresponding to all feature points.

2. The method for evaluating camera distortion correction effect as described in claim 1, characterized in that, For each feature point, the distance error corresponding to the feature point is converted into the angular error from the feature point to the principal point of the camera lens, specifically including: The coordinates of the camera in the world coordinate system are calculated. For each feature point, according to the formula The distance error corresponding to the feature point is converted into the angle error from the feature point to the principal point of the camera lens; where θ represents the angle error corresponding to the feature point, d represents the distance error corresponding to the feature point, (u, v, 0) represents the coordinates of the feature point in the world coordinate system, and (x, y, z) represents the coordinates of the camera in the world coordinate system.

3. The method for evaluating camera distortion correction effect as described in claim 2, characterized in that, The calculation to obtain the camera's coordinates in the world coordinate system specifically includes: According to the formula The coordinates of the camera in the world coordinate system are calculated; where R represents the rotation matrix in the calibration extrinsic parameters of the camera, and T represents the translation vector in the calibration extrinsic parameters of the camera.

4. The method for evaluating camera distortion correction effect as described in claim 1, characterized in that, The evaluation of the distortion correction effect based on the angular errors corresponding to all feature points specifically includes: Statistical analysis is performed on the angle errors corresponding to all feature points to determine the average and / or maximum angle errors; The distortion correction effect of the camera is evaluated based on the average and / or maximum angle error; wherein, the larger the average angle error, the worse the distortion correction effect; the larger the maximum angle error, the worse the distortion correction effect.

5. The method for evaluating camera distortion correction effect as described in claim 1, characterized in that, The process of detecting all feature points in the processed target image and performing line fitting on feature points located on the same straight line to obtain the straight line containing each feature point specifically includes: Feature point detection is performed on the processed target image to obtain all feature points in the processed target image; The least squares method is used to fit the feature points located on the same straight line to obtain the straight line where each feature point is located; where the feature points located on the same straight line correspond to the points on the target object being photographed that are located on the same straight line.

6. The method for evaluating camera distortion correction effect as described in claim 1, characterized in that, The target image is a checkerboard image; therefore, The process involves detecting all feature points in the processed target image and performing line fitting on feature points located on the same straight line to obtain the straight line containing each feature point. Specifically: Detect all feature points in the processed checkerboard image, and perform line fitting on feature points located on the same straight line to obtain the horizontal and vertical straight lines of each feature point. For each feature point, the distance error from the feature point to the line containing the feature point is calculated, specifically as follows: For each feature point, calculate the first distance error from the feature point to the horizontal line containing the feature point, and the second distance error from the feature point to the vertical line containing the feature point. For each feature point, the distance error corresponding to the feature point is converted into the angular error from the feature point to the principal point of the camera lens, specifically as follows: For each feature point, the first distance error corresponding to the feature point is converted into the first angle error from the feature point to the principal point of the camera lens, and the second distance error corresponding to the feature point is converted into the second angle error from the feature point to the principal point of the camera lens. The distortion correction effect is evaluated based on the angle errors corresponding to all feature points, specifically as follows: The distortion correction effect is evaluated based on the first and second angle errors corresponding to all feature points.

7. The method for evaluating camera distortion correction effect as described in claim 6, characterized in that, The evaluation of the distortion correction effect based on the first and second angle errors corresponding to all feature points specifically includes: Statistical analysis is performed on the first and second angle errors corresponding to all feature points to determine the average and / or maximum angle errors; The distortion correction effect of the camera is evaluated based on the average and / or maximum angle error; wherein, the larger the average angle error, the worse the distortion correction effect; the larger the maximum angle error, the worse the distortion correction effect.

8. A device for evaluating the effect of camera distortion correction, characterized in that, The apparatus for implementing the camera distortion correction effect evaluation method as described in any one of claims 1 to 7, the apparatus comprising: The target image acquisition module is used to capture images of the target object using a camera; wherein the target image includes several extractable feature points, and the feature points are located on the same straight line or multiple straight lines; The distortion correction and affine transformation processing module is used to perform affine transformation processing on the target image after distortion correction, so that the target object is within the image range and the feature points of the middle part and the edge part of the processed target image have the same scale. The line fitting module is used to detect all feature points in the processed target image and perform line fitting on feature points located on the same straight line to obtain the straight line where each feature point is located. The distance error calculation module is used to calculate the distance error from each feature point to the line containing the feature point. An angle error calculation module is used to convert the distance error corresponding to each feature point into the angle error from the feature point to the lens principal point of the camera. The distortion correction effect evaluation module is used to evaluate the distortion correction effect based on the angular errors corresponding to all feature points.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program; wherein, when the computer program is executed, it controls the device on which the computer-readable storage medium is located to perform the camera distortion correction effect evaluation method as described in any one of claims 1 to 7.

10. A terminal device, characterized in that, The system includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor, when executing the computer program, implements the camera distortion correction effect evaluation method as described in any one of claims 1 to 7.

Images