Automatic camera intrinsic parameter calibration method

Abstract

The application provides a camera intrinsic parameter automatic calibration method, comprising the following steps: S1, acquiring a set of original images of a general scene shot by a camera; S2, extracting first edge information of each original image in the set of original images through an edge detection algorithm, wherein the first edge information comprises image coordinates and gradient directions of first edge points; S3, performing straight line detection on the first edge information based on a 4D Hough space, and acquiring an initial estimated value of a lens distortion parameter; S4, counting a total number of effective straight lines in the set of original images; S5, judging whether the total number of effective straight lines is greater than a given first threshold value, if yes, entering S6, otherwise, returning to S1; S6, iteratively optimizing camera intrinsic parameters based on the 4D Hough space by using the effective straight lines; and S7, updating the camera intrinsic parameters and performing image correction. The application calibrates a principal point of the camera and the lens distortion parameter by using images of a general scene, and can simplify a calibration process of camera intrinsic parameters.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of camera calibration, in particular to a camera intrinsic parameter automatic calibration method. BACKGROUND

[0002] In the field of machine vision, camera calibration is an important step in many vision applications. The article "A Flexible New Technique for Camera Calibration" presents a classic camera calibration method, which can estimate the intrinsic parameters of the camera by shooting multiple calibration patterns of regular checkerboard patterns. However, many complex field environments do not have the conditions to use a checkerboard for calibration. In addition, due to the impact of external forces, even the influence of temperature and humidity in the environment, the intrinsic parameters of the camera may be affected. Therefore, a more convenient method is needed to update the camera intrinsic parameters regularly. SUMMARY

[0003] The purpose of the present application is to provide a camera intrinsic parameter automatic calibration method, which does not rely on calibration tools such as checkerboards, can simplify the camera intrinsic parameter calibration process, and is suitable for long focal length lenses, zoom lenses and various complex and harsh environments.

[0004] In order to achieve the above purpose, the present application provides a camera intrinsic parameter automatic calibration method, comprising:

[0005] S1, obtaining a set of original images of a general scene shot by a camera, the set of original images comprising N original images, wherein N≥1;

[0006] S2, extracting first edge information of each original image in the set of original images by an edge detection algorithm, the first edge information comprising image coordinates and gradient directions of first edge points, the first edge points being edge points of the original image;

[0007] S3, performing straight line detection on the first edge information based on a 4D Hough space, and obtaining an initial estimate value of lens distortion parameters;

[0008] S4, counting the total number of valid straight lines in the set of original images;

[0009] S5, judging whether the total number of valid straight lines is greater than a given first threshold value, if yes, entering S6, otherwise, returning to S1;

[0010] S6, performing iterative optimization on camera intrinsic parameters based on the valid straight lines and the 4D Hough space;

[0011] S7, updating the camera intrinsic parameters and performing image correction.

[0012] Optionally, when extracting the first edge information of each original image in the original image set by the edge detection algorithm, the edge points of the original image are screened to determine the first edge points, specifically comprising:

[0013] For each edge point p of the original image, the cosine sum of the gradient direction difference values of the edge point p and all other edge points in the neighborhood is calculated, denoted as EOC p ;

[0014] When EOC p is less than a second threshold value, the edge point p is removed.

[0015] Optionally, only the edge point with the maximum EOC p is retained in the neighborhood with a set radius as the center of the edge point p as the first edge point.

[0016] Optionally, the 4D Hough space is based on the traditional 2D Hough space and increases the image dimension and the distortion dimension, the length of the image dimension is at least 1, each original image corresponds to a 3D Hough space, and the S3 specifically comprises:

[0017] S31, initializing the sampling values of the distance dimension, the angle dimension and the distortion dimension in the corresponding 3D Hough space for each original image, and each distortion dimension corresponds to a 2D Hough space;

[0018] S32, in the 4D Hough space, performing operations from the image dimension and the distortion dimension, based on the sampling value of the i-th distortion dimension, transforming the first edge information of the j-th original image by a distortion model to generate the second edge information of the j-th original image, the second edge information including the image coordinates and the gradient direction of the second edge points, and the second edge points being the edge points generated by the transformation of the first edge points by the distortion model;

[0019] S33, in the i-th 2D Hough space corresponding to the j-th original image, using the second edge information to vote for a group of straight lines and associating the first edge points and the second edge points with the voted straight lines to calculate the score of the i-th 2D Hough space corresponding to the j-th original image;

[0020] S34, calculating the total score of all original images corresponding to the i-th 2D Hough space in the 4D Hough space, and selecting the 2D Hough space i max with the highest score as the initial estimation value of the lens distortion parameter.

[0021] Optionally, the S31 specifically comprises:

[0022] generating the distance sampling values of the distance dimension and the angle sampling values of the angle dimension according to a conventional 2D Hough space initialization manner;

[0023] The distortion dimension is represented by a first radial distortion coefficient k1, and according to a given range [k min max ] of the first radial distortion coefficient and a step size δ k , the sampling value of the i-th distortion dimension is represented as:

[0024]

[0025] wherein i=0, 1, 2…(k max -k min ) / δ k .

[0026] Optionally, the distortion dimension is represented by a first normalized distortion parameter p1;

[0027] The first normalized distortion coefficient p1 is represented under a polynomial distortion model as:

[0028]

[0029] The first normalized distortion coefficient p1 is represented under a division distortion model as:

[0030]

[0031] wherein r max represents the farthest distance from the original image coordinate to the distortion center;

[0032] According to a given range [p min , p max ] of the first normalized distortion coefficient and a step size δ p , the sampling value of the i-th distortion dimension under the 4D Hough space is represented as:

[0033]

[0034] wherein i=0, 1, 2…(p max -p min ) / δ p .

[0035] Optionally, the S4 specifically comprises:

[0036] Given a minimum number threshold T p of the second edge points of the valid straight line association, p when the number of the second edge points of a straight line association is not less than T p , the straight line is determined as a valid straight line;

[0037] In the 4D Hough space, the number of the valid straight lines corresponding to the i max th 2D Hough space of the jth image is counted and recorded

[0038] The total number of the valid straight lines in the original image set is counted

[0039] Optionally, in the S5, the original images in which the valid straight lines exist in the original image set are reserved.

[0040] Optionally, the S6 specifically includes:

[0041] S61, in the 4D Hough space, the sum of the number of the second edge points associated with all the valid straight lines of all the original images corresponding to the i max th 2D Hough space of each original image of the original image set is counted and taken as the initial number of the valid second edge points Np last ;

[0042] S62, the camera intrinsic parameters include the distortion center (u c , v c ) and the lens distortion parameters (k1, k2, k3), denoted as Dp(u c , v c , k1, k2, k3), and the initial value of the camera intrinsic parameters is set as w and h are respectively the pixel width and height of the original image;

[0043] S63, the objective function to be optimized is the weighted distance square sum of the second edge points of all the original images in the 4D Hough space to the valid straight lines associated therewith, and the objective function is defined as:

[0044]

[0045] wherein, represents the second edge point corresponding to the first edge point k associated with the valid straight line m in the jth original image under the camera intrinsic parameters Dp, represents the distance of the second edge point to the valid straight line m associated therewith in the jth original image, β jm represents the weight value of the valid straight line m in the jth original image, which can be determined according to the number of the edge points associated with the valid straight line m, α j represents the weight value of the jth image, which can be determined according to the number of the valid straight lines of the jth image;

[0046] S64, in each optimization process, the effective straight line m remains unchanged, the first edge point k remains unchanged, the second edge point changes according to the camera internal parameter Dp, the target function is minimized by changing the camera internal parameter Dp, and the optimal camera internal parameter Dp of this optimization is output opt ;

[0047] S65, the first edge information of each original image is transformed according to the optimal camera internal parameter of this optimization, the second edge information is regenerated, and the straight line detection is performed again according to the second edge information, and the total number of effective straight lines Nl in the original image set is counted opt and the sum of the number of second edge points associated with all the effective straight lines Np out ;

[0048] S66, when the total number of effective straight lines Nl opt is greater than the first threshold value and Np opt >Np last , this optimization is effective, and the next iteration will continue to optimize with Dp opt as the initial value according to the regenerated effective straight line and the associated second edge point, the initial effective second edge point number Np last is updated to Np opt ; otherwise, this optimization is invalid, the iteration is ended, and the camera internal parameter selects the optimal camera internal parameter Dp opt .

[0049] Optionally, the S7 specifically comprises:

[0050] S71, using the optimal camera internal parameter Dp opt as the camera internal parameter;

[0051] S72, traversing each coordinate (x', y') of the corrected image, obtaining the coordinate (x, y) of the coordinate (x', y') in the original image according to the distortion model;

[0052] S73, obtaining the pixel value at the coordinate (x', y') of the corrected image in an interpolation manner.

[0053] In the camera internal parameter automatic calibration method provided by the application, the original image set captured by a general scene is used to calibrate the principal point of the camera and the lens distortion parameter, and the calibration process of the camera internal parameter can be simplified without relying on a calibration tool such as a checkerboard, and the stability and accuracy are higher than those of a single image. BRIEF DESCRIPTION OF DRAWINGS

[0054] Those skilled in the art should understand that the provided drawings are used to better understand the application, and do not constitute any limitation on the scope of the application. Among them:

[0055] Figure 1 The flow chart of the camera intrinsic parameter automatic calibration method provided in the embodiment. DETAILED DESCRIPTION

[0056] The camera intrinsic parameter mainly includes focal length, principal point (u c ,v c ) and lens distortion parameters. The radial distortion model of the lens is as follows:

[0057]

[0058] wherein (u,v) is the original image coordinate, is the transformed image coordinate, r = ‖(u,v)-(u c ,v c )‖2 represents the distance from the image coordinate to the distortion center, and D(r) represents the radial distortion model of the lens. The distortion model of the lens includes but is not limited to the polynomial model and the division model, and the polynomial model is as follows:

[0059] D(r) = 1 + k1r 2 +k2r 4 +k3r 6 (2)

[0060] The division model is as follows:

[0061]

[0062] The lens distortion parameters include the radial distortion coefficients k1, k2 and k3, wherein k2 and k3 are optional.

[0063] Generally, the camera intrinsic parameter needs to be calibrated by a standard pattern such as a checkerboard, which is inconvenient for use in a long-focus lens, a zoom lens or a complex and harsh environment. Based on this, the purpose of the present application is to provide a camera intrinsic parameter automatic calibration method, which calibrates the principal point and the lens distortion parameters of the camera by using a set of original images taken in a general scene, does not rely on a calibration tool such as a checkerboard, can simplify the calibration process of the camera intrinsic parameter, and has higher stability and precision compared with a single image.

[0064] To make the purpose, advantages and characteristics of the present application clearer, the present application is further described in detail below in combination with the drawings and specific embodiments. It should be noted that the drawings are very simplified and all use non-precise proportions, only to facilitate and clearly assist the purpose of describing the embodiments of the present application, and are not used to limit the defined conditions of the embodiments of the present application, so they do not have substantial technical significance, any modification of the structure, change of the proportional relationship or adjustment of the size, without affecting the effects and purposes that can be achieved by the present application, should still fall within the scope of the technical content disclosed by the present application.

[0065] It should also be understood that the terms "first", "second", "third", etc. in the description merely identify different components, elements, steps, etc. in the description for the purpose of differentiating one component, element, step, etc. from another, and do not imply a logical or chronological relationship among different components, elements, steps, etc. Moreover, the terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without more limitations, an element preceded by "comprises... a" does not, without more limitations, foreclose the existence of

[0066] Reference is made to Figure 1 , Figure 1 A flowchart of a camera intrinsic automatic calibration method provided by the present embodiment. The present embodiment provides a camera intrinsic automatic calibration method, comprising the following steps:

[0067] S1, obtaining an original image set of a general scene photographed by a camera, the original image set comprising N original images, where N≥1;

[0068] S2, extracting first edge information of each original image in the original image set by an edge detection algorithm, the first edge information comprising image coordinates and gradient directions of first edge points, the first edge points being edge points of the original image;

[0069] S3, performing straight line detection on the first edge information based on a 4D Hough space, and obtaining an initial estimation value of a lens distortion parameter;

[0070] S4, counting a total number of effective straight lines in the original image set;

[0071] S5, judging whether the total number of effective straight lines is greater than a given first threshold value, if yes, entering S6, otherwise, returning to S1;

[0072] S6, performing iterative optimization on camera intrinsic based on the effective straight lines and the 4D Hough space;

[0073] S7, updating the camera intrinsic and performing image correction.

[0074] Firstly, step S1 is performed to obtain an original image set of a general scene photographed by a camera, the original image set comprising N original images, where N≥1.

[0075] Then, step S2 is performed to extract first edge information of each original image in the original image set by an edge detection algorithm, the first edge information including image coordinates (x, y) and a gradient direction θ of a first edge point, the first edge point being an edge point of the original image. The first edge detection algorithm includes but is not limited to a first-order edge detection method such as a Roberts operator, a Prewitt operator, a Sobel operator, a Canny operator, and the like, and a second-order edge detection method such as a Laplacian operator.

[0076] Further, in order to quickly utilize the first edge points for line detection and effectively remove redundant edge points and abnormal edge points, when the first edge information of each original image in the original image set is extracted by the edge detection algorithm, the edge points of the original image are screened to determine the first edge points.

[0077] Specifically, the step of determining the first edge points specifically includes:

[0078] S21, for each edge point p of the original image, a cosine sum of gradient direction difference values of the edge point p and all other edge points in a neighborhood is calculated and denoted as EOC p , which is used to measure the edge direction change of the neighborhood of the point p.

[0079] EOC p =∑ q cos(θ q -θ p ) (4)

[0080] wherein θ p and θ q are gradient directions of the edge point p and the edge point q, respectively.

[0081] S22, when EOC p is less than a third threshold EOC0, the edge point p is removed.

[0082] Further, in order to reduce edge point redundancy, only the edge point with the maximum EOC p is retained as the first edge point in a neighborhood with a set radius and with the edge point p as the center.

[0083] Then, step S3 is performed to perform line detection on the first edge information based on a 4D Hough space and obtain an initial estimation value of a lens distortion parameter. Specifically, the step S3 includes:

[0084] S31, for each original image, sampling values of a distance dimension, an angle dimension, and a distortion dimension in a corresponding 3D Hough space are initialized, and each distortion dimension corresponds to a 2D Hough space;

[0085] S32, in the 4D Hough space, from the image dimension and the distortion dimension, based on the sampling value of the i-th distortion dimension, the first edge information of the j-th original image is transformed by the distortion model to generate the second edge information of the j-th original image, the second edge information includes the image coordinates and gradient direction of the second edge point, the second edge point is the edge point generated by the transformation of the first edge point by the distortion model;

[0086] S33, in the i-th 2D Hough space corresponding to the j-th original image, a group of straight lines are voted using the second edge information, and the first edge point and the second edge point are associated with the voted straight lines, and the score of the i-th 2D Hough space corresponding to the j-th original image is calculated;

[0087] S34, the total score of all original images corresponding to the i-th 2D Hough space in the 4D Hough space is calculated, and the 2D Hough space i with the highest score is selected max The sampling value of the corresponding distortion dimension is used as the initial estimation value of the lens distortion parameter.

[0088] After obtaining the first edge information, step S31 is performed first to initialize the 4D Hough space. The 4D Hough space is based on the traditional 2D Hough space and adds image dimensions and distortion dimensions. As for the image dimension, the length of the image dimension is at least 1, each original image corresponds to a 3D Hough space, and the sampling value of the distance dimension, the angle dimension and the distortion dimension in the corresponding 3D Hough space is initialized for each original image. Each distortion dimension corresponds to a 2D Hough space containing the distance dimension and the angle dimension.

[0089] Wherein, the distance dimension and the angle dimension generate the distance sampling value of the distance dimension and the angle sampling value of the angle dimension according to the initialization method of the traditional 2D Hough space; the distortion dimension uses the first radial distortion coefficient k1 to represent, according to the given range [k min , k max ] and step δ k , the sampling value of the i-th distortion dimension is represented as:

[0090]

[0091] Wherein i=0, 1, 2...(k max -k min ) / δ k .

[0092] Since the lens distortion parameters include radial distortion coefficients k1, k2, k3, wherein k2 and k3 are optional, when the distortion dimension uses the first radial distortion coefficient k1 to represent, k2, k3=0.

[0093] Preferably, in order to avoid the distortion dimension sampling value too small and related to the image resolution, the present application uses a normalized distortion parameter instead of the radial distortion parameter, which is defined as the percentage change of the distance from the distortion center of a point in the original image before and after the distortion model transformation. The first normalized distortion parameter p1 is defined by the farthest point from the distortion center on the original image. For example, in a polynomial model, the first normalized distortion parameter p1 satisfies where r max represents the farthest distance from the distortion center of the original image coordinates. The second normalized distortion parameter p2 and the third normalized distortion parameter p3 corresponding to the distance r2 and r3 of the points to the distortion center can be defined as r2=ρ2r max and r3=ρ3r max , for example. etc. For example, in a polynomial model, p1, p2, p3 satisfy:

[0094]

[0095] If the distortion dimension uses the first normalized distortion parameter p1 (p2, p3=0) described above, the first distortion parameter a1 in the polynomial model can be obtained using the above equation (5): i.e. According to the given first normalized distortion coefficient range [p min , p max ] and the step size δ p , the sampling value of the i-th distortion dimension in the 4D Hough space is represented as:

[0096]

[0097] where i=0, 1, 2...(p max -p min ) / δ p .

[0098] Then step S32 is performed, and in the 4D Hough space, the image dimension and the distortion dimension are operated, and based on the sampling value of the i-th distortion dimension or The image coordinates (x ij , y ij ) and the gradient direction θ ij of the first edge information of the j-th original image are transformed by the distortion model to generate second edge information, and the second edge information includes the image coordinates (x′ ij , y′ ij ) and the gradient direction θ′ ij, the second edge point being an edge point generated by the first edge point through a distortion model transformation;

[0099] Then step S33 is performed, in which a set of lines is voted on the i-th 2D Hough space corresponding to the j-th original image using the second edge information, and the first edge point and the second edge point are associated with the voted lines, and the score of the i-th 2D Hough space corresponding to the j-th original image is calculated.

[0100] Preferably, the set of lines can be limited according to the position and gradient direction of the second edge point. The (x' ij , y' ij ) and gradient direction θ' ij of the second edge point can limit the angle interval of line voting in the i-th 2D Hough space corresponding to the j-th original image to [θ' ij - Δθ, θ' ij + Δθ], where Δθ is the maximum difference between the set line gradient direction and the gradient direction of the second edge point. Each The distance interval of line voting in the i-th 2D Hough space corresponding to the j-th original image can be limited to , which represents the distance from the origin to the line passing through the second edge point (x' ij , y' ij ) and having a gradient direction of θ' in the i-th 2D Hough space corresponding to the j-th original image, and Δρ is the maximum distance allowed for the second edge point to the line.

[0101] Preferably, the voting score is represented according to the distance of the second edge point to the line in the i-th 2D Hough space corresponding to the j-th original image, the farther the distance, the lower the score. For example or , etc. , where is the voting score of the second edge point to the line in the i-th 2D Hough space corresponding to the j-th original image.

[0102] The total score of the line in the i-th 2D Hough space corresponding to the j-th original image is:

[0103]

[0104] where is the line The total number of associated second edge points.

[0105] Then calculate the score in the i-th 2D Hough space corresponding to the j-th original image. Select all valid lines (number of lines) in the i-th 2D Hough space corresponding to the j-th original image. The scores of the j-th original image are summed to obtain the score of the i-th 2D Hough space corresponding to the j-th original image.

[0106]

[0107] Next, step S34 is executed to calculate the total score s of all original images corresponding to the i-th 2D Hough space in the 4D Hough space. i :

[0108]

[0109] Select the highest-scoring 2D Hough space i max The sampled values ​​corresponding to the distortion dimension or This serves as the initial estimate for the lens distortion parameters.

[0110] Next, step S4 is executed to count the total number of valid lines in the original image set, specifically including:

[0111] Given the minimum number of second edge points T associated with the effective straight line p When the number of second edge points associated with a certain straight line is not less than T p When the line is in a valid line, it is determined to be a valid line.

[0112] In the 4D Hough space, for the j-th image corresponding to the i-th... max In a 2D Hough space, the number of valid lines is counted and recorded.

[0113] Count the total number of valid straight lines in the original image set.

[0114] Next, step S5 is executed to determine whether the total number of valid lines is greater than a given first threshold. If it is, proceed to S6; otherwise, it is determined that the number of valid lines in the 4D Hough space is insufficient, and the process returns to S1 to obtain more original images or replace the original image dataset. When retaining original images containing valid lines in the original image set, only the newly added original images in the 4D Hough space need to be calculated, and the retained original images can directly utilize the previous calculation results.

[0115] Then, step S6 is executed, which uses the effective straight line to iteratively optimize the camera intrinsic parameters based on 4D Hough space.

[0116] The S6 specifically includes:

[0117] S61, in the 4D Hough space, for each original image of the original image set, the i max th 2D Hough space corresponding to the original image, the sum of the second edge point numbers associated with all the effective straight lines of all the original images is counted and taken as the initial effective second edge point number Np last ;

[0118] S62, the camera intrinsic parameters include distortion center (u c , v c ) and lens distortion parameters (k1, k2, k3), denoted as Dp(u c , v c , k1, k2, k3), and the initial value of the camera intrinsic parameters is set as w and h are the pixel width and height of the original image respectively;

[0119] S63, the objective function to be optimized is the weighted distance square sum of the second edge points of all original images in the 4D Hough space to the effective straight lines associated therewith, and the objective function is defined as:

[0120]

[0121] wherein, represents the second edge point corresponding to the first edge point k associated with the effective straight line m in the jth original image under the camera intrinsic parameters Dp, represents the distance of the second edge point to the effective straight line m associated therewith in the jth original image, β jm represents the weight of the effective straight line m in the jth original image, which can be determined according to the number of edge points associated with the effective straight line m, α j represents the weight of the jth image, which can be determined according to the number of effective straight lines of the jth image;

[0122] S64, in each optimization process, the effective straight line m remains unchanged, the first edge point k remains unchanged, and the second edge point changes according to the camera intrinsic parameters Dp, and the objective function is minimized by changing the camera intrinsic parameters Dp, and the optimal camera intrinsic parameters Dp opt of this optimization are output;

[0123] S65, according to the optimal camera intrinsic parameters of this optimization, the first edge information of each original image is transformed, the second edge information is regenerated, and the straight line detection is performed again according to the second edge information, the total number of effective straight lines Nl opt in the original image set and the sum of the second edge point numbers associated with all the effective straight lines Np opt are counted.

[0124] S66, when the total number of effective straight lines satisfies Nl opt greater than the first threshold value and Np opt >Np last , then this optimization is effective, and the next iteration will continue to optimize according to the regenerated effective straight line and the associated second edge point with Dp opt as the initial value, the initial effective second edge point number Np last is updated to Np opt ; otherwise, this optimization is ineffective, the iteration ends, and the camera parameter is selected as the optimal camera parameter Dp opt .

[0125] In this embodiment, the S61 can be executed synchronously with the S4.

[0126] In this embodiment, the lens distortion parameters k2 and k3 can always be 0 and do not participate in optimization.

[0127] In this embodiment, the model of the lens distortion parameter includes but is not limited to a polynomial model and a division model.

[0128] Preferably, the lens distortion parameter can be replaced by a normalized distortion parameter, and at this time, the camera parameter is denoted as Dp(u c , v c , p1, p2, p3).

[0129] Finally, step S7 is executed to update the camera parameter and perform image correction. The S7 specifically includes:

[0130] S71, using the optimal camera parameter Dp opt as the camera parameter;

[0131] S72, traversing each coordinate (x', y') of the corrected image, and obtaining the coordinate (x, y) of the coordinate (x', y') in the original image according to the distortion model;

[0132] S73, obtaining the pixel value at the coordinate (x', y') of the corrected image in an interpolation manner.

[0133] In the above step S72, the distance from each coordinate (x', y') of the corrected image to the distortion center is calculated, and the coordinate (x, y) of the coordinate (x', y') in the original image can be obtained according to the inverse transformation of formula (1).

[0134] In this embodiment, the S72 can be traversing each coordinate (x, y) of the original image, and obtaining the coordinate (x', y') in the corrected image according to the distortion model.

[0135] In the embodiment, the S73 interpolation method includes, but is not limited to, nearest neighbor interpolation, linear interpolation, bilinear interpolation, etc.

[0136] In conclusion, the application provides a camera intrinsic parameter automatic calibration method, which calibrates the principal point and lens distortion parameters of the camera by using a set of original images of general scenes, does not rely on a calibration tool such as a checkerboard, can simplify the camera intrinsic parameter calibration process, and has higher stability and precision compared with a single image.

[0137] In addition, it should be appreciated that, although the application has been disclosed above with reference to the preferred embodiments, the above embodiments are not intended to limit the application. Any person skilled in the art can make many possible changes and modifications to the technical solutions disclosed above, or modify equivalent embodiments, without departing from the scope of the technical solutions of the application. Therefore, any simple modification, equivalent change and modification made to the above embodiments according to the technical essence of the application, without departing from the content of the technical solutions of the application, shall still fall within the scope of protection of the technical solutions of the application.

Claims

1. A method for automatic calibration of camera intrinsic parameters, characterized in that, The method comprises the following steps: S1, acquire a raw image set of a general scene shot by a camera, the raw image set including N raw images, wherein ; S2, extracting first edge information of each original image in the original image set by an edge detection algorithm, the first edge information comprising image coordinates and gradient directions of first edge points, the first edge points being edge points of the original image; S3, performing straight line detection on the first edge information based on a 4D Hough space, and obtaining an initial estimated value of a lens distortion parameter; S4, counting a total number of valid straight lines in the original image set; S5, judging whether the total number of valid straight lines is greater than a given first threshold value, if yes, entering S6, otherwise, returning to S1; S6, iteratively optimizing camera intrinsic parameters based on the 4D Hough space using the valid straight lines; S7, updating the camera intrinsic parameters and performing image correction; The S6 specifically comprises: S61. In the 4D Hough space, for each original image in the original image set, the first... In a 2D Hough space, the sum of the number of second edge points associated with all valid lines in all the original images is used as the initial number of valid second edge points. ; S62, the camera intrinsic parameters comprise a distortion center and a lens distortion parameter , denoted as , an initial value of the camera intrinsic parameters is set as , and respectively are a pixel width and a height of the original image S63, an optimized objective function being a weighted distance square sum of second edge points of all original images in the 4D Hough space to their associated valid straight lines, the objective function being defined as: ； in, Indicates the first The second edge point corresponding to the first edge point k associated with the effective straight line m in the original image under the camera intrinsic parameters Dp. Indicates the first The distance from the second edge point in the original image to its associated valid line m, Indicates the first The weight of a valid line m in the original image can be determined based on the number of edge points associated with the valid line m. Indicates the first The weights of each image can be determined based on the number of valid straight lines in the j-th image; S64, in each optimization process, the effective straight line The first edge point k remains unchanged, the second edge point changes according to the camera intrinsic Dp, and the camera intrinsic Make the objective function minimum, output the optimal camera intrinsic of this optimization ; S65, transform the first edge information of each original image according to the optimized optimal camera intrinsic parameter, regenerate second edge information, and re-perform straight line detection according to the second edge information, and count the total number of effective straight lines in the original image set and the sum of the number of second edge points associated with all the effective straight lines ; S66、when the total number of effective straight lines satisfies the first threshold value and , this optimization is effective, and the next iteration will continue to optimize according to the regenerated effective straight line and the associated second edge point as the initial value, the initial number of effective second edge points is updated to ; otherwise, this optimization is ineffective, the iteration is ended, and the camera intrinsic parameter is selected as the optimal camera intrinsic parameter .

2. The camera intrinsic parameter automatic calibration method of claim 1, wherein, When extracting the first edge information of each original image in the original image set by the edge detection algorithm, the edge points of the original image are screened to determine the first edge points, specifically comprising: For each edge point p of the original image, the cosine sum of the gradient direction difference between the edge point p and all other edge points in the neighborhood is calculated, denoted as ; When The edge point p is rejected when 3. The camera intrinsic auto-calibration method of claim 2, wherein, The edge point p is taken as the center of a circle, and only the largest edge point in a neighborhood of a given radius is retained as the first edge point. The largest edge point, as the first edge point.

4. The automatic camera intrinsic parameter calibration method as described in claim 1, wherein the 4D Hough space is based on the traditional 2D Hough space with the addition of image dimension and distortion dimension, the minimum length of the image dimension is 1, and each original image corresponds to a 3D Hough space, characterized in that, The S3 specifically comprises: S31, initializing sampling values of a distance dimension, an angle dimension and a distortion dimension in a corresponding 3D Hough space for each original image, respectively, each distortion dimension corresponding to a 2D Hough space; S32. In the 4D Hough space, operations are performed from the image dimension and the distortion dimension, based on the first... The sampled values ​​of the distortion dimension are processed by the distortion model to the 1st distortion dimension. The first edge information of the original image is transformed to generate the second... The second edge information of the original image includes the image coordinates and gradient direction of the second edge point, and the second edge point is the edge point generated by the first edge point through the distortion model transformation. S33、in the first 2D Hough space corresponding to the first original image, a set of lines is voted using the second edge information and the first edge point and the second edge point are associated with the voted lines, and a score of the first 2D Hough space corresponding to the first original image is calculated; S34, calculate the total score of all original images corresponding to the 2D Hough space in the 4D Hough space, and select the 2D Hough space with the highest score as the initial estimation value of the lens distortion parameter. the sampling value of the distortion dimension corresponding to the 2D Hough space as the initial estimation value of the lens distortion parameter.​ 5. The camera intrinsic parameter automatic calibration method of claim 4, wherein, The S31 specifically comprises: The distance sampling values of the distance dimension and the angle sampling values of the angle dimension are generated in the initialization manner of a conventional 2D Hough space; said distortion dimension using a first radial distortion coefficient denotes, according to a given range of first radial distortion coefficients and a step size , the sampling value of the first distortion dimension is denoted as: ； wherein .

6. The camera intrinsic parameter automatic calibration method of claim 5, wherein, the distortion dimension uses a first normalized distortion coefficient representing; the first normalized distortion coefficient is expressed under a polynomial distortion model as: ； the first normalized distortion coefficient is expressed under the division distortion model as: ； wherein, denotes the farthest distance of the original image coordinates to the distortion center; According to the given first normalized distortion coefficient range and step size , the sampling value of the 4D Hough space under the first distortion dimension is expressed as: ； wherein .

7. The camera intrinsic parameter automatic calibration method of claim 5, wherein, The S4 specifically comprises: Given the minimum number threshold of the second edge points of the effective straight line association When the number of the second edge points of a straight line association is not less than , the straight line is determined as an effective straight line. In the 4D Hough space, for the first The corresponding image of the first In a 2D Hough space, the number of valid lines is counted and recorded. ; counting the total number of valid straight lines in the original image set 8. The camera intrinsic parameter automatic calibration method of claim 7, wherein, In the S5, the original images in which the valid straight lines exist in the original image set are reserved.

9. The camera intrinsic parameter automatic calibration method of claim 1, wherein, The S7 specifically comprises: S71, using the optimal camera intrinsic parameters as camera intrinsic parameters; S72, traversing each coordinate of the corrected image , obtaining the coordinate according to the distortion model the coordinate in the original image ; S73, get the corrected image coordinates by interpolation the pixel value at the center of the pixel.

Images