Lens distortion detection method and distortion detection device

Abstract

The invention discloses a lens distortion detection method and device, and the method comprises the steps: A, controlling a camera to shoot an image of a target, and obtaining a to-be-calibrated image; then executing the step B; step B, selecting a group of to-be-fitted angular points from the to-be-calibrated image, and performing fitting processing on the group of to-be-fitted angular points to obtain a distortion evaluation value; then executing the step C; and step C, detecting the distortion condition of the lens based on the distortion evaluation value so as to distinguish different lenses. In conclusion, the distortion detection precision of different lenses is improved.

Description

Technical Field

[0001] The present application relates to the technical field of sensor detection, and particularly to a lens distortion detection method and a distortion detection device. Background Art

[0002] A lens is an optical device composed of multiple groups of lenses. The thickness of the center and the edge of the lens is different. A convex lens is thick in the middle and thin at the edges, and a concave lens is thin in the middle and thick at the edges. Generally, light is orthogonal to the main axis and passes through the main axis without distortion. The farther the light is from the main axis, the greater the distortion. When trying to take close-up photos of some straight and tall buildings, in fact, the parallel lines do not appear parallel, and the edges become curved. This belongs to linear distortion. Lens distortion is actually based on the non-linear imaging characteristics of optical lenses. Usually, distortion correction is also required to obtain a more accurate image. For example, during the production of a lens, the magnitude of the edge distortion of the lens is proportional to the magnitude of the field of view angle. When using a lens with too small or too large distortion, calibration errors of the camera internal parameters often occur. Therefore, before calibrating the camera internal parameters, it is necessary to first detect the distortion of the lens in order to select a suitable calibration model for camera calibration. Existing technologies often calculate the distortion coefficient using the pixel points arranged in a matrix within the reference pixel region of multiple frames of images. Although the coverage area within the same frame of image is large (at least covering the target completely), it is easy to introduce various imaging deviations, affecting the distortion detection of different types of lenses. Summary of the Invention

[0003] The present application aims to provide a lens distortion detection method and a distortion detection device based on fitting processing. The specific technical solutions are as follows: A lens distortion detection method based on fitting processing includes: Step A, controlling a camera to capture an image of a target to obtain an image to be calibrated; then performing Step B; Step B, selecting a group of corner points to be fitted from the image to be calibrated, and then performing fitting processing on the group of corner points to be fitted to obtain a fitting result distortion evaluation value; then performing Step C; Step C, detecting the distortion of the lens based on the fitting result distortion evaluation value to distinguish different lenses. Compared with the existing technology, the present application only performs fitting processing on a group of corner points to be fitted to obtain a fitting result distortion evaluation value, without using the pixel points arranged in a matrix within the reference pixel region of multiple frames of images to calculate the distortion coefficient, thereby saving the amount of computation, and being able to select the corner points in the target that most reflect the distortion situation in the same frame of image for distortion detection, reducing the introduction of various imaging deviations, and improving the distortion detection accuracy of different lenses.

[0004] Further, the corner points to be fitted are the corner points of the column of cells or the row of cells that are closest to the boundary of the image to be calibrated within the target, so that a set of corner points to be fitted consists of all the corner points of the column of cells or the row of cells that are closest to the boundary of the image to be calibrated within the target; wherein the target is composed of a plurality of cells arranged row by row or column by column; there are at least three corner points to be fitted in a set of corner points to be fitted. Therefore, the present application selects a set of corner points that are closest to the boundary of the image to be calibrated but not located on the boundary of the target as a set of corner points to be fitted for fitting processing, so as to improve the accuracy of describing the distortion by the distortion evaluation value.

[0005] Further, in the step B, the method for obtaining the distortion evaluation value by fitting the set of corner points to be fitted includes: using the least squares method to perform linear fitting on the set of corner points to be fitted to obtain a target fitting line; calculating the distances from each corner point to be fitted in the set of corner points to be fitted to the target fitting line respectively; then averaging the distances from each corner point to be fitted to the target fitting line, and then setting the obtained average value as the distortion evaluation value of the fitting result. Thus, the average value of the distances from a set of corner points to be fitted to the target fitting line is used to describe the trend of the target fitting line, so as to determine the deviation formed by the connection line of the set of corner points mapped into the world coordinate system relative to the target fitting line, and further reflect the distortion situation of the lens.

[0006] Further, in the step B, the method for performing fitting processing on the set of corner points to be fitted to obtain a distortion evaluation value includes: performing curve fitting on the set of corner points to be fitted based on the least square method to form an Nth-degree polynomial equation, and determining the target fitting curve; where N is an integer greater than 2; the Nth-degree polynomial equation includes a polynomial composed of a first independent variable, coefficients of each term, and a second independent variable; setting the intersection point of the target fitting curve and the central axis of the image to be calibrated as the middle point; calculating the first-order partial derivative of the Nth-degree polynomial equation with respect to the first independent variable at the middle point to obtain a first first partial derivative value; calculating the first-order partial derivative of the Nth-degree polynomial equation with respect to the second independent variable at the middle point to obtain a first second partial derivative value; calculating the second-order partial derivative of the Nth-degree polynomial equation with respect to the first independent variable at the middle point to obtain a second first partial derivative value; calculating the second-order partial derivative of the Nth-degree polynomial equation with respect to the second independent variable at the middle point to obtain a second second partial derivative value; calculating the curvature of the middle point based on the first first partial derivative value, the first second partial derivative value, the second first partial derivative value, and the second second partial derivative value; and then setting the curvature of the middle point as the distortion evaluation value of the fitting result. Therefore, the distortion evaluation value can describe the maximum bending degree of the connection line (a straight line segment parallel to the boundary of the target) of a set of corner points to be fitted mapped into the world coordinate system in the image to be calibrated, serve as a basis for distinguishing different lenses, and further reflect the maximum distortion of the lens, serving as the best distortion parameter basis for distinguishing different lenses, thereby improving the accuracy of detecting the distortion of different lenses in step C.

[0007] Further, N is equal to the value 3 to form a cubic polynomial equation; the first independent variable included in the cubic polynomial equation is used to represent the abscissa of the pixel coordinate system, and the second independent variable included in the cubic polynomial equation is used to represent the ordinate of the pixel coordinate system, so that the first-order partial derivative of the cubic polynomial equation with respect to the first independent variable represents the tangent slope of the target fitting curve in the horizontal axis direction of the pixel coordinate system, and the first-order partial derivative of the cubic polynomial equation with respect to the second independent variable represents the tangent slope of the target fitting curve in the vertical axis direction of the pixel coordinate system. In summary, the present application can introduce a cubic polynomial for curvature calculation, quickly obtain the distortion evaluation value, reduce the time of fitting processing, and improve the efficiency of judging lens distortion.

[0008] Further, the method for calculating the curvature of the intermediate point based on the first partial derivative value, the second partial derivative value, the third partial derivative value, and the fourth partial derivative value includes: controlling the multiplication of the first partial derivative value and the fourth partial derivative value to obtain a first product; controlling the multiplication of the second partial derivative value and the third partial derivative value to obtain a second product; calculating the sum of the squares of the first partial derivative value and the second partial derivative value to obtain a first sum of squares; calculating the ratio between the absolute value of the difference between the first product and the second product and the square root of the cube of the first sum of squares to obtain the curvature of the intermediate point. Thus, the curvature of the pixel point with the maximum curvature in the target fitting curve or the curvature of the pixel point with the maximum curvature in the to-be-calibrated image is obtained; moreover, this application provides an evaluation numerical basis for judging and identifying the bending degree of the boundary of the target in the world coordinate system after being projected by the lens onto the pixel coordinate system.

[0009] Further, in step C, the method for detecting the distortion of the lens based on the distortion evaluation value of the fitting result includes: when the distortion evaluation value of the fitting result is greater than the preset threshold, it is determined that the distortion of the lens is the distortion of a fish-eye lens; when the distortion evaluation value of the fitting result is less than or equal to the preset threshold, it is determined that the distortion of the lens is the distortion of a pinhole lens; wherein, the greater the distortion evaluation value of the fitting result, the greater the distortion of the lens; the smaller the distortion evaluation value of the fitting result, the smaller the distortion of the lens. Therefore, it is necessary to set the preset threshold to distinguish camera models with different distortion degrees. Thus, step C realizes detecting the distortion of the lens according to the magnitude relationship between the distortion evaluation value and the preset threshold.

[0010] Further, when performing linear fitting on the set of corner points to be fitted to obtain the distortion evaluation value of the fitting result, the preset threshold used for detecting the distortion of the lens is the first preset threshold; when the distortion evaluation value of the fitting result is greater than the first preset threshold, it is determined that the distortion of the lens is greater than the preset distortion parameter; when the distortion evaluation value of the fitting result is less than or equal to the first preset threshold, it is determined that the distortion of the lens is less than or equal to the preset distortion parameter. Thus, the misjudgment of the distortion situation is reduced, and the accuracy of lens distortion detection is improved.

[0011] Further, when performing curve fitting on the set of corner points to be fitted, if the curvature of the middle point is set as the distortion evaluation value of the fitting result, the preset threshold used to detect the distortion of the lens is the second preset threshold; when the distortion evaluation value of the fitting result is greater than the second preset threshold, it is determined that the distortion of the lens is greater than the preset distortion parameter; when the distortion evaluation value of the fitting result is less than or equal to the second preset threshold, it is determined that the distortion of the lens is less than or equal to the preset distortion parameter; wherein, the second preset threshold is not equal to the first preset threshold; the third preset threshold is not equal to the first preset threshold; the third preset threshold is less than the second preset threshold. The middle point is the intersection of the target fitting curve and the central axis of the image to be calibrated; according to the image distortion characteristics, the pixel points on the central axis of the image to be calibrated are the pixel points with the maximum curvature in the image to be calibrated. Therefore, comparing the curvature of the middle point with the preset threshold can better judge the extreme curvature level of the surface / curve fitted from the image to be calibrated, that is, reflect the maximum distortion that the lens can produce. Therefore, comparing the curvature of the middle point with the second preset threshold reduces the misjudgment of the distortion situation and meets the calibration accuracy in the case of excessive lens distortion.

[0012] A distortion detection device, which includes a line laser module and a target. The line laser module includes a camera, and a lens for taking images is assembled in the camera; the target is within the shooting field of view of the camera; the line laser module is configured to execute the lens distortion detection method. By performing steps A to C, the distortion detection device only performs fitting processing on a set of corner points to be fitted to obtain the distortion evaluation value of the fitting result, without using the pixel points arranged in a matrix in the reference pixel area of multiple frames of images to calculate the distortion coefficient, thereby saving the amount of calculation, and being able to select the corner points in the target that most reflect the distortion situation in the same frame of image for distortion detection, reducing the introduction of multiple imaging deviations, improving the distortion detection accuracy of camera lenses of different models or different field of view angles, facilitating the subsequent selection of a reasonable camera calibration model to correct the lens distortion, and further improving the ranging accuracy of the line laser module. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 is a schematic flowchart of a lens distortion detection method disclosed in an embodiment of the present application.

[0014] Figure 2 is a schematic diagram showing the distribution of corner points to be fitted in the image to be calibrated in another embodiment of the present application; wherein, the target is shown as a distorted checkerboard in Figure 2 the rectangular image area of, Figure 2 the checkerboard in is composed of cells filled with diagonal lines arranged at intervals ( Figure 2 the cells filled with 3 right-leaning dashed line segments in) and cells filled with white, and the corner points are the intersections of the two boundaries of the cells. EMBODIMENTS

[0015] The following will combine the accompanying drawings in the embodiments of the present invention to describe the technical solutions in the embodiments of the present invention in detail. To further illustrate each embodiment, the present invention provides accompanying drawings. These accompanying drawings are a part of the disclosure content of the present invention, which are mainly used to illustrate the embodiments and can be combined with the relevant descriptions in the specification to explain the operating principles of the embodiments. With reference to these contents, those of ordinary skill in the art should be able to understand other possible implementation manners and the advantages of the present invention.

[0016] One of the main tasks of the lens manufacturer is to design and manufacture a lens as close as possible to the camera model. Thus, the gap between the imaging model and the imaging of the actual lens can be defined as the aforementioned distortion. Distortion can also be described by a certain mathematical function. It can be simply understood that distortion is to insert a function into the ideal camera imaging model, so that the finally obtained pixel coordinates are inconsistent with the coordinates obtained by the camera's standard projection relationship.

[0017] To detect lenses with different distortions, the present application discloses a lens distortion detection method; the execution subject of the lens distortion detection method is a control unit inside or outside the camera, which can be arranged inside the ranging module where the camera is located. The lens is assembled in the camera, and the camera is detachably installed in the ranging module. The camera disclosed in the present application is an electronic device with image acquisition capabilities. The ranging module can be a line laser module. Then the camera can use a fish-eye camera or a pinhole camera. The distortion generated by the lens of a fish-eye camera is generally greater than that generated by the lens of a pinhole camera.

[0018] Referring to Figure 1 it can be known that the lens distortion detection method disclosed in the present application includes: Step A: Control the camera to capture an image of the target to obtain a to-be-calibrated image; then execute Step B. In Step A, control the target to be within the field of view of the camera. The to-be-calibrated image shows the distorted target. The to-be-calibrated image is an image of the target; then select a group of multiple corner points that are located on the same straight line in the world coordinate system from the to-be-calibrated image, select at least three corner points to form a group of to-be-fitted corner points, and then accept fitting processing in Step B.

[0019] Step B: Select a group of to-be-fitted corner points from the to-be-calibrated image, and then perform fitting processing on the group of to-be-fitted corner points to obtain a distortion evaluation value; then execute Step C. In the present application, the obtained distortion evaluation value is used to describe the bending degree of the imaging line segment of a straight line in the physical world inside the lens or the deviation degree of a straight line in the physical world (world coordinate system) relative to the fitted straight line. Among them, a group of to-be-fitted corner points serves as multiple pixel points in the to-be-calibrated image that can best describe the bending degree of the imaging line segment. Therefore, the distortion evaluation value can reflect the distortion situation of the lens and is used as a reliable parameter for evaluating the size of the lens distortion in the present application.

[0020] In this application, the fitting method for a set of corner points to be fitted includes, but is not limited to, fitting a curve equation and solving for the curvature, or fitting a straight line and solving for the point-line distance, or other methods that can extract the value of the bending degree of the straight line in the actual physical environment (in the world coordinate system) imaged within the lens, etc.

[0021] Step C: Detect the distortion situation of the lens based on the fitting result distortion evaluation value to distinguish different lenses, so as to form a set of general recognition methods to detect cameras with different distortions. In step C, specifically detect the distortion magnitude of the lens according to the magnitude relationship between the fitting result distortion evaluation value and a preset threshold; among them, the greater the fitting result distortion evaluation value, the greater the distortion of the lens; the smaller the fitting result distortion evaluation value, the smaller the distortion of the lens. Generally, the imaging model of the lens is used to describe the projection relationship of the real camera lens, and there will be a gap with the reality. Therefore, this application distinguishes different lenses based on the fitting result distortion evaluation value, such as a pinhole lens and a fish-eye lens; among them, as long as the lens with a slightly larger imaging angle (which can be imagined as a fish-eye lens with a larger distortion), if the simple pinhole imaging model is still used to describe it, the camera calibration and ranging deviation will be very large. Therefore, before calibrating the camera internal parameters, it is necessary to first detect the distortion situation of the lens and distinguish different types of lenses, so as to select a suitable calibration model for camera calibration subsequently.

[0022] Compared with the prior art, this application only performs fitting processing on a set of corner points to be fitted to obtain the fitting result distortion evaluation value, without using the pixel points arranged in a matrix within the reference pixel area of multiple frames of images to calculate the distortion coefficient, thereby saving the amount of computation, and being able to select the corner points in the same frame of image that best reflect the distortion situation in the target for distortion detection, reducing the introduction of various imaging deviations, and improving the distortion detection accuracy of different lenses.

[0023] As an embodiment, the corner points to be fitted are the corner points of a column of cells or a row of cells that are closest to the boundary of the image to be calibrated inside the target, so that a set of corner points to be fitted consists of all the corner points of a column of cells or a row of cells that are closest to the boundary of the image to be calibrated inside the target. In this embodiment, multiple corner points to be fitted are selected from the image to be calibrated to represent the corner points distributed on a straight-line boundary of the target in the world coordinate system. For example, the corner points of the target distributed along the boundary of a row or a column of cell regions in the world coordinate system, due to distortion, the corner points distributed on the same straight line in the world coordinate system become distributed along a curve or discretely after being projected onto the pixel coordinate system, and fitting processing is required. Therefore, by performing step B, at least three corner points are selected from the corner points of a column of cells or a row of cells that are closest to the boundary of the image to be calibrated inside the target to form a set of corner points to be fitted to participate in the fitting processing.

[0024] The target is composed of multiple cells arranged row by row or column by column; there are at least three corner points to be fitted in a set of corner points to be fitted to represent the corner points distributed on a straight-line boundary of the target in the world coordinate system. It should be noted that the corner points belong to pixel points in the image to be calibrated. The target can be a checkerboard, and it is the checkerboard set on the surface of the calibration board in front of the camera; the target can also be other objects with straight boundary lines. The target is within the field of view of the camera; the distorted target is shown in the image to be calibrated.

[0025] Schematically, the image to be calibrated is Figure 2 a rectangular image in which multiple cells are distorted, and the image to be calibrated encloses Figure 2 all the cells in Figure 2 The distortion in

[0026] is the distortion generated when the target is projected onto the pixel plane of the camera. This rectangular image can be defined as the image of the target. For example, the rectangle in front of the fish-eye lens will generate distortion after focusing and imaging, including barrel distortion.

[0026] Preferably, the target is composed of multiple cells arranged row by row or column by column along one direction; so Figure 2 the cells with filled diagonal lines ( Figure 2 the cells with 3 right-tilted virtual line segments filled in Figure 2 ) and the cells filled with white that are arranged at intervals form the area covered by the target in the image to be calibrated (pixel coordinate plane UOV); the target in Figure 2 is similar to a black-and-white checkerboard, but has been distorted. In Figure 2 , the target can be regarded as composed of 48 cells arranged row by row along the V axis direction, where the number of cells arranged in each row is 8; the target can also be regarded as composed of 48 cells arranged column by column along the U axis direction, where the number of cells arranged in each column is 6.

[0027] In some embodiments, the corner points to be fitted are the corner points of the column of cells closest to the boundary of the image to be calibrated within the target; thus, the corner points to be fitted do not lie on the boundary of the target. Schematically, as Figure 2 shown, located within the target (a target area composed of cells in 6 rows and 8 columns), the corner points to be fitted lie on the boundary of the area composed of cells in 4 rows and 6 columns; specifically, within the target, the corner points to be fitted are the corner points of the column of cells closest to the right boundary of the image to be calibrated; correspondingly, the corner points to be fitted are Figure 2 the black dots that are not on the boundary of the target in the last column counted from left to right (along the positive direction of the U axis) in

[0028] In some embodiments, the corner points to be fitted are the corner points of the row of cells closest to the boundary of the image to be calibrated within the target; thus, the corner points to be fitted do not lie on the boundary of the target. Schematically, as Figure 2 shown, located within the target (a target area composed of cells in 6 rows and 8 columns), the corner points to be fitted lie on the boundary of the area composed of cells in 4 rows and 6 columns; specifically, within the target, the corner points to be fitted are the corner points of the row of cells closest to the upper boundary of the image to be calibrated; correspondingly, the corner points to be fitted are Figure 2 the black dots that are not on the boundary of the target in the last row counted from bottom to top (along the positive direction of the V axis) in

[0029] Combined with Figure 2 it can be seen that the cells with filled diagonal lines set at intervals ( Figure 2The target consists of cells filled with 3 right-leaning dashed lines and white-filled cells; the corner points are the intersection points of the edge lines of the cells filled with diagonal lines, or the intersection points of the edge lines of white cells, or the two endpoints of the common side of two adjacent cells. In Figure 2 it can be represented by black dots and can be regarded as being located at the two-dimensional coordinates of the pixel plane. Preferably, the Harris corner detection method is used to search for corner points within the complete target. Among them, the Harris corner detection method mainly searches for corner points based on the gradient of the change in the gray value of image pixels. Specifically, the area near the corner point of a cell is an area where the change in its pixel gray value is very large, and its gray value gradient is also very large; in other words, in the neighborhood of non-corner positions, the pixel values of each point change little, or are even almost equal, and its gradient is relatively small, which is used as the basis for searching for the corner points of the target.

[0030] As an embodiment, in step B, the method for fitting a set of corner points to be fitted to obtain a distortion evaluation value includes: using the least squares method to perform a linear fit on the set of corner points to be fitted to obtain a target fitting line; specifically, using a linear function (the target fitting line equation) of the least squares method to perform a linear fit, calculating the coefficients and constant terms of each order required for the linear equation, thereby constructing the target fitting line equation, and obtaining a target fitting line in the pixel coordinate system. This target fitting line is used as the best fitting line that approximates a pre-selected set of corner points to be fitted. Among them, the connection line of the set of corner points to be fitted is parallel to a boundary of the target in the target, and the connection line of the set of corner points mapped into the world coordinate system is a straight line segment.

[0031] After determining the target fitting line, calculate the distances from each corner point to be fitted within the set of corner points to the target fitting line respectively; specifically, use the coefficients and constant terms in the target fitting line equation to construct the point-to-line equation, and then substitute the coordinates of each corner point to be fitted into the point-to-line equation in turn to calculate the distances from each corner point to be fitted to the target fitting line. Then, find the average value of the distances from each corner point to be fitted to the target fitting line, and then set the obtained average value as the distortion evaluation value. Thus, use the average value of the distances from a set of corner points to be fitted to the target fitting line to describe the trend of the target fitting line, so as to determine the deviation formed by the connection line of the set of corner points mapped into the world coordinate system relative to the target fitting line, and further reflect the distortion situation of the lens.

[0032] As an embodiment, in step B, the method for fitting a set of corner points to be fitted to obtain a distortion evaluation value includes: Curve fitting is performed on the set of corner points to be fitted based on the least squares method to form an N-degree polynomial equation, and the target fitting curve is determined to form the fitting result of curve fitting within the image to be calibrated; where N is an integer greater than 2; the N-degree polynomial equation includes a polynomial composed of a first independent variable, coefficients of each term, and a second independent variable; the first independent variable can be represented by the abscissa U of Figure 2 and the second independent variable can be represented by the ordinate V of Figure 2 . The first independent variable and the second independent variable can both be combined with the coefficients of each term to form monomials of corresponding degrees in the N-degree polynomial equation.

[0033] It should be noted that at least 3 of the multiple corner points to be fitted preselected in this embodiment exist, and are arranged into a set of corner points to be fitted or a set of corner points; in this embodiment, curve fitting is performed based on a multiple function or a multiple polynomial (N-degree polynomial equation) of the least squares method, and the coefficients of each term and the constant term of the optimal multiple function are calculated, so that an N-degree polynomial equation can be constructed, which is also defined as an N-degree surface equation.

[0034] The intersection point of the target fitting curve and the central axis of the image to be calibrated is set as the middle point; specifically, the coordinates of the middle point in the pixel coordinate system UOV can be obtained by combining the equation of the central axis and the N-degree polynomial equation, and are denoted as the coordinates of the middle point; schematically, as Figure 2 shown, the central axis of the image to be calibrated can extend along the U-axis direction or along the V-axis direction. Among them, based on the image distortion characteristics, the curvature of the target fitting curve is the largest at the central axis or the center of the image to be calibrated, so that the curvature generated by the middle point relative to the pixel points or corner points distributed on both sides of the central axis within the target fitting curve is large.

[0035] After determining the N-degree polynomial equation and the coordinates of the middle point, the following partial derivative calculations are performed: Calculate the first-order partial derivative of the N-degree polynomial equation with respect to the first independent variable at the middle point to obtain the first partial derivative value; it can be understood that the first-order partial derivative of the first independent variable of the N-degree polynomial equation is obtained to obtain the first-order partial derivative formula, and then the coordinates of the middle point are substituted into the first-order partial derivative formula to calculate the first partial derivative value. Calculate the first-order partial derivative of the N-degree polynomial equation with respect to the second independent variable at the middle point to obtain the second partial derivative value; it can be understood that the first-order partial derivative of the second independent variable of the N-degree polynomial equation is obtained to obtain the first-order partial derivative formula, and then the coordinates of the middle point are substituted into the first-order partial derivative formula to calculate the second partial derivative value.

[0036] Calculate the second partial derivative of the N - th degree polynomial equation with respect to the first independent variable at the intermediate point to obtain the first second - partial derivative value. It can be understood as taking the second - order partial derivative of the N - th degree polynomial equation with respect to the first independent variable to obtain the second - order partial - derivative formula, and then substituting the coordinates of the intermediate point into the second - order partial - derivative formula to calculate the first second - partial derivative value. Calculate the second partial derivative of the N - th degree polynomial equation with respect to the second independent variable at the intermediate point to obtain the second second - partial derivative value. It can be understood as taking the second - order partial derivative of the N - th degree polynomial equation with respect to the second independent variable to obtain the second - order partial - derivative formula, and then substituting the coordinates of the intermediate point into the second - order partial - derivative formula to calculate the second second - partial derivative value.

[0037] Based on the first first - partial derivative value, the first second - partial derivative value, the first second - partial derivative value, and the second second - partial derivative value, calculate the curvature of the intermediate point; and then set the curvature of the intermediate point as the distortion evaluation value of the fitting result. The implementation process of calculating the curvature includes substituting the first first - partial derivative value, the first second - partial derivative value, the first second - partial derivative value, and the second second - partial derivative value into the curvature formula to convert the curvature of the intermediate point in the target fitting curve, which represents the curvature of the intermediate point on the curve fitted by the N - th degree polynomial equation. The curvature formula is pre - set to calculate the curvature of the N - th degree polynomial equation at a certain pixel point using the first - order partial derivative and the second - order partial derivative; and then set the curvature of the intermediate point as the distortion evaluation value. Generally, the curvature of the intermediate point is greater than the curvatures of the pixel points passed by the target fitting curve distributed on both sides of the intermediate point.

[0038] Therefore, the distortion evaluation value can describe the maximum bending degree of the connection line (a straight - line segment parallel to the boundary of the target) of a group of corner points to be fitted mapped into the world coordinate system within the image to be calibrated, and further reflect the maximum distortion of the lens, serving as the best distortion parameter basis for distinguishing different lenses and improving the accuracy of detecting the distortion of different lenses in step C.

[0039] On the basis of the foregoing embodiments, N is equal to the value 3 to form a cubic polynomial equation; and determine the fitted cubic polynomial surface. The general form of the cubic polynomial equation can be combined by those skilled in the art based on the coefficients of each term and the constant term calculated during the curve - fitting process to construct the N - th degree polynomial equation. Preferably, the coefficients of each term and the constant term all represent the fitting parameters. The coefficients of each term include the coefficient of the linear term, the coefficient of the quadratic term, the coefficient of the cubic term, and the coefficient of the cross - term, which belong to the coefficients of each term of the curve equation calculated based on the least - squares method. The fitting process using the curve - fitting method is to judge the distortion size of the lens as the basis for selecting the calibration method. Since there is a threshold setting for the curvature later, the type of the lens can be distinguished through a range.

[0040] The first independent variable included in the cubic polynomial equation is used to represent the abscissa of the pixel coordinate system, and the second independent variable included in the cubic polynomial equation is used to represent the ordinate of the pixel coordinate system, so that the first-order partial derivative of the cubic polynomial equation with respect to the first independent variable represents the tangent slope of the target fitting curve in the horizontal axis direction of the pixel coordinate system, and the first-order partial derivative of the cubic polynomial equation with respect to the second independent variable represents the tangent slope of the target fitting curve in the vertical axis direction of the pixel coordinate system. In summary, the present application can introduce a cubic polynomial for curvature calculation, quickly obtain the distortion evaluation value, reduce the time of fitting processing, and improve the efficiency of judging lens distortion.

[0041] Specifically, the method for calculating the curvature of the intermediate point based on the first first partial derivative value, the first second partial derivative value, the second first partial derivative value, and the second second partial derivative value includes: controlling the multiplication of the first first partial derivative value and the second second partial derivative value to obtain a first product; controlling the multiplication of the first second partial derivative value and the second first partial derivative value to obtain a second product; calculating the sum of the squares of the first first partial derivative value and the first second partial derivative value to obtain a first sum of squares; calculating the ratio between the absolute value of the difference between the first product and the second product and the square root of the cube of the first sum of squares to obtain the curvature of the intermediate point, which is equivalent to calculating the curvature of the intermediate point through the curvature formula. Thus, the curvature of the pixel point with the largest curvature in the target fitting curve or the largest curvature in the to-be-calibrated image is obtained; moreover, the present application provides an evaluation numerical basis for judging and identifying the degree of bending of the boundary of the target in the world coordinate system after being projected by the lens onto the pixel coordinate system.

[0042] As an embodiment, in step C, the method for detecting the distortion of the lens based on the fitting result distortion evaluation value includes: when the fitting result distortion evaluation value is greater than the preset threshold, determining that the distortion of the lens is the distortion of a fish-eye lens; when the fitting result distortion evaluation value is less than or equal to the preset threshold, determining that the distortion of the lens is the distortion of a pinhole lens; thereby distinguishing whether the lens assembled in the currently to-be-calibrated camera is a fish-eye lens or a pinhole lens. In this embodiment, the greater the fitting result distortion evaluation value, the greater the distortion of the lens. The smaller the fitting result distortion evaluation value, the smaller the distortion of the lens. Therefore, it is necessary to set the preset threshold to distinguish different camera models with different distortion degrees. Thus, step C realizes detecting the distortion of the lens according to the magnitude relationship between the distortion evaluation value and the preset threshold; when the distortion evaluation value is greater than the preset threshold, it is judged that the lens distortion is large; when the distortion evaluation value is less than the preset threshold, it is judged that the lens distortion is small.

[0043] Generally, the imaging model of a lens is used to describe the projection relationship of a real camera lens, and there will be a gap from reality. Therefore, this application uses different imaging models to describe the projection relationships of different lenses, such as the aforementioned pinhole imaging distortion calibration model and the fisheye lens distortion calibration model. In this application, for a lens with a slightly larger imaging angle (which can be imagined as a fisheye lens with a larger distortion), if the simple pinhole model is still used to describe it, the ranging deviation will be very large. Therefore, it is necessary to distinguish different camera models. The preset threshold is associated with the internal parameters of different cameras, specifically related to the stretching and deformation of the graphics in the world coordinate system by the lens of a fisheye lens (the lens of a fisheye camera) or a pinhole camera; the preset threshold can be expressed as the misalignment balance value between the corner points in the world coordinate environment and the projected corner points in the lens imaging plane.

[0044] Among them, as the assembly position of the target relative to the camera changes, the preset threshold can be adaptively changed; different lens models result in different lens distortions, and thus the settings of the preset threshold are also different; specifically, the preset threshold can be set according to actual test data.

[0045] As an embodiment, when performing a linear fit on the set of corner points to be fitted to obtain a fitting result distortion evaluation value, the preset threshold used to detect the distortion of the lens is the first preset threshold; when the fitting result distortion evaluation value is greater than the first preset threshold, it is determined that the distortion of the lens is greater than the preset distortion parameter; when the fitting result distortion evaluation value is less than or equal to the first preset threshold, it is determined that the distortion of the lens is less than or equal to the preset distortion parameter. Specifically, the method for detecting the distortion of the lens based on the fitting result distortion evaluation value specifically includes: when performing a linear fit on the set of corner points to be fitted to obtain a distortion evaluation value, the preset threshold used to detect the distortion of the lens is the first preset threshold; when the distortion evaluation value is greater than the first preset threshold, it is determined that the distortion of the lens is greater than the preset distortion parameter, indicating that the distortion of the lens is relatively large and can be classified as a fisheye lens; when the distortion evaluation value is less than or equal to the first preset threshold, it is determined that the distortion of the lens is less than or equal to the preset distortion parameter, indicating that the distortion of the lens is relatively small and can be classified as a pinhole lens. Among them, the preset distortion parameter is used as the distortion reference value for different lens models to describe the distortion degree of the lens and serves as the parameter basis for selecting and distinguishing different lenses. Thus, the misjudgment of the distortion situation is reduced, and the accuracy of lens distortion detection is improved.

[0046] As another embodiment, in the case of performing curve fitting on the set of corner points to be fitted, if the curvature of the middle point is set as the distortion evaluation value of the fitting result, the preset threshold used to detect the distortion of the lens is the second preset threshold; when the distortion evaluation value of the fitting result is greater than the second preset threshold, it is determined that the distortion of the lens is greater than the preset distortion parameter, indicating that the distortion of the lens is relatively large and can be classified as a fish-eye lens; when the distortion evaluation value of the fitting result is less than or equal to the second preset threshold, it is determined that the distortion of the lens is less than or equal to the preset distortion parameter and can be classified as a pinhole lens; wherein, the second preset threshold is not equal to the first preset threshold, and preferably, the second preset threshold is greater than the first preset threshold.

[0047] The preset distortion parameter is used as the distortion reference value for different types of lenses to describe the distortion degree of the lens and serves as the basis for the distortion parameter of selecting the matching calibration model.

[0048] The middle point is the intersection point of the target fitting curve and the central axis of the image to be calibrated; according to the image distortion characteristics, the pixel points on the central axis of the image to be calibrated are the pixel points with the largest curvature in the image to be calibrated. Therefore, comparing the curvature of the middle point with the preset threshold can better judge the extreme curvature level of the surface / curve fitted by the image to be calibrated, that is, reflect the maximum distortion that the lens can generate. Therefore, comparing the curvature of the middle point with the second preset threshold can reduce the misjudgment of the distortion situation and meet the calibration accuracy in the case of excessive distortion of the lens.

[0049] Both the straight line fitting and the curve fitting included in the fitting process disclosed in the foregoing embodiments can realize the judgment of the distortion size of the camera lens, but different thresholds need to be set so that the second preset threshold and the first preset threshold respectively correspond to the actual distortion situations of different types of lenses; for example, when the curvature of the pixel points on the imaging plane of all the lenses participating in the distortion detection is greater than the first preset threshold, the preset threshold used to detect the distortion of the lens selects the second preset threshold. Furthermore, if more numerical values of the preset threshold are set, more types of lenses can be distinguished by the preset threshold, that is, more types of lenses with different distortions can be identified, including more types of fish-eye lenses, so as to be compatible with the distortion detection accuracy of more cameras with different distortions.

[0050] Based on the foregoing embodiments, the present application also discloses a distortion detection device. The distortion detection device includes a line laser module and a target. The line laser module includes a camera, and a lens for taking images is assembled in the camera; the target is within the shooting field of view of the camera. The line laser module further includes a line laser sensor. The target can receive the laser line emitted by the line laser sensor and form a spot image, that is, light spots or light strips are displayed in the image of the target; the line laser module is configured to execute the lens distortion detection method disclosed in the foregoing embodiments. By executing steps A to C, the distortion detection device only performs fitting processing on a group of corner points to be fitted to obtain a fitting result distortion evaluation value, and does not use the pixel points arranged in a matrix within the reference pixel region of multiple frames of images to calculate the distortion coefficient, thereby saving the amount of computation. Moreover, it can select the corner points in the target that most reflect the distortion situation in the same frame of image for distortion detection, reduce the introduction of various imaging deviations, improve the distortion detection accuracy of camera lenses of different models or different field of view angles, and facilitate subsequent selection of a reasonable camera calibration model to correct the lens distortion, thereby improving the ranging accuracy of the line laser module.

[0051] The above embodiments are only for illustrating the technical concept and features of the present invention. The purpose is to enable those skilled in the art to understand the content of the present invention and implement it accordingly, and it should not be used to limit the protection scope of the present invention. Any equivalent transformation or modification made according to the spirit and essence of the present invention should be covered within the protection scope of the present invention.

Claims

1. Lens distortion detection method based on fitting processing, It is characterized in that include: Step A, controlling the camera to capture an image of the target to obtain an image to be calibrated; Then execute step B; Step B, selecting a group of corner points to be fitted from the image to be calibrated, and then performing fitting processing on the group of corner points to be fitted to obtain a distortion evaluation value; then executing step C; Step C: detecting the distortion of the lens based on the distortion evaluation value to distinguish different lenses.

2. The lens distortion detection method according to claim 1, It is characterized in that The corner points to be fitted are the corner points of a column of cells or a row of cells that are closest to the boundary of the image to be calibrated inside the target, so that a group of corner points to be fitted is composed of all corner points of a column of cells or a row of cells that are closest to the boundary of the image to be calibrated inside the target; The target is composed of a plurality of cells arranged in rows or columns; and a group of corner points to be fitted contains at least three corner points to be fitted.

3. The lens distortion detection method according to claim 2, It is characterized in that In the step B, the method of performing fitting processing on the set of corner points to be fitted to obtain the distortion evaluation value includes: Using the least square method to perform straight line fitting on the set of corner points to be fitted, to obtain a target fitting straight line; Calculating the distance between each corner point to be fitted in the set of corner points to be fitted and the target fitting line respectively; Then, the distances from each to-be-fitted corner point to the target fitting straight line are averaged, and the average value is set as the fitting result distortion evaluation value.

4. The lens distortion detection method according to claim 2, It is characterized in that In the step B, the method of performing fitting processing on the set of corner points to be fitted to obtain the distortion evaluation value includes: Based on the least square method, curve fitting is performed on the set of corner points to be fitted to form an N-order polynomial equation, and a target fitting curve is determined to be fitted; wherein N is an integer greater than 2; the N-order polynomial equation includes a polynomial consisting of a first independent variable, coefficients of each order term, and a second independent variable; The intersection point of the target fitting curve and the central axis of the image to be calibrated is set as the middle point; Calculate the first-order partial derivative of the N-order polynomial equation with respect to the first independent variable at the intermediate point to obtain the first partial derivative value; Calculate the first-order partial derivative of the N-order polynomial equation with respect to the second independent variable at the intermediate point to obtain the first and second partial derivative values; Calculate the second-order partial derivative of the N-order polynomial equation with respect to the first independent variable at the intermediate point to obtain the second-order partial derivative value; Calculate the second-order partial derivative of the N-order polynomial equation with respect to the second independent variable at the intermediate point to obtain the second partial derivative value; Based on the first partial derivative, the first second partial derivative, the second first partial derivative and the second second partial derivative, the curvature of the middle point is calculated; and then the curvature of the middle point is set as the fitting result distortion evaluation value.

5. The lens distortion detection method according to claim 3 or 4, It is characterized in that N is equal to the value 3 to form a cubic polynomial equation; The first independent variable included in the third-order polynomial equation is used to represent the horizontal coordinate of the pixel coordinate system, and the second independent variable included in the third-order polynomial equation is used to represent the vertical coordinate of the pixel coordinate system, so that the first-order partial derivative of the third-order polynomial equation with respect to the first independent variable represents the tangent slope of the target fitting curve in the direction of the horizontal coordinate axis of the pixel coordinate system, and the first-order partial derivative of the third-order polynomial equation with respect to the second independent variable represents the tangent slope of the target fitting curve in the direction of the vertical coordinate axis of the pixel coordinate system.

6. The lens distortion detection method according to claim 4, It is characterized in that The method of calculating the curvature of the midpoint based on the first partial derivative, the first second partial derivative, the second first partial derivative, and the second second partial derivative includes: Controlling the multiplication of the first-first partial derivative and the second-second partial derivative to obtain a first product; Control the multiplication of the first second partial derivative value and the second first partial derivative value to obtain a second product; Calculate the sum of squares of the first partial derivative and the first second partial derivative to obtain the first square sum; The curvature of the middle point is obtained by calculating the ratio of the absolute value of the difference between the first product and the second product and the square root of the cube of the first square sum.

7. The lens distortion detection method according to claim 3 or 4, It is characterized in that In step C, the method for detecting the distortion of the lens based on the distortion evaluation value of the fitting result includes: When the distortion evaluation value of the fitting result is greater than a preset threshold, it is determined that the distortion of the lens is the distortion of a fisheye lens; When the distortion evaluation value of the fitting result is less than or equal to a preset threshold, it is determined that the distortion of the lens is the distortion of a pinhole lens; Among them, when the distortion evaluation value of the fitting result is larger, the distortion of the lens is larger; when the distortion evaluation value of the fitting result is smaller, the distortion of the lens is smaller.

8. The lens distortion detection method according to claim 7, It is characterized in that When a straight line fitting is performed on the set of corner points to be fitted to obtain a distortion evaluation value of the fitting result, the preset threshold used for detecting the distortion of the lens is a first preset threshold; When the distortion evaluation value of the fitting result is greater than a first preset threshold, determining that the distortion of the lens is greater than a preset distortion parameter; When the distortion evaluation value of the fitting result is less than or equal to the first preset threshold, it is determined that the distortion of the lens is less than or equal to the preset distortion parameter.

9. The lens distortion detection method according to claim 8, It is characterized in that When curve fitting is performed on the set of corner points to be fitted, if the curvature of the middle point is set as the distortion evaluation value of the fitting result, the preset threshold used for detecting the distortion of the lens is the second preset threshold; When the distortion evaluation value of the fitting result is greater than a second preset threshold, determining that the distortion of the lens is greater than a preset distortion parameter; When the distortion evaluation value of the fitting result is less than or equal to a second preset threshold, determining that the distortion of the lens is less than or equal to a preset distortion parameter; Among them, the second preset threshold is not equal to the first preset threshold; the third preset threshold is not equal to the first preset threshold; the third preset threshold is less than the second preset threshold.

10. A distortion detection device, It is characterized in that The distortion detection device includes a line laser module and a target. The line laser module includes a camera in which a lens for capturing images is installed. The target is within the camera's field of view. The line laser module is configured to execute the lens distortion detection method described in any one of claims 1 to 9.

Images