Calibration method of fisheye camera and readable storage medium

Abstract

The application provides a fish-eye camera calibration method and a readable storage medium. The calibration method comprises the following steps: acquiring a calibration pattern; calibrating a first parameter in a manner of solving a vanishing point based on the calibration pattern; and calibrating a second parameter based on the calibration pattern and the first parameter. The first parameter comprises a focal length of the fish-eye camera and a translation parameter from an image plane coordinate system to an image coordinate system; and the second parameter comprises a distortion coefficient of the fish-eye camera. In this way, the parameters to be calibrated are solved in two steps, thereby avoiding the phenomenon that a large amount of data is required, a long calculation time is needed, a calibration result is unstable, and robustness is poor due to the calibration of a large number of parameters at one time. The calibration method can complete calibration only by using one calibration image under certain conditions, thereby solving the problem that the time, cost and robustness are not ideal in the prior art.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a calibration method for a fisheye camera and a readable storage medium. Background Technology

[0002] A fisheye camera is a camera equipped with a fisheye lens, which is a lens with an extremely short focal length and a field of view close to or exceeding 180°. It is an extreme wide-angle lens with wide applications in security, surveillance, autonomous driving, and other fields.

[0003] Due to the unique imaging characteristics of fisheye cameras, there is a significant difference between the raw image captured and the object being photographed. Therefore, specific algorithms are needed to convert the raw image into a properly proportioned image for easy recognition by the operator. This conversion process requires determining certain parameters of the fisheye camera, including its internal parameters and distortion coefficients.

[0004] The calibration process for the aforementioned parameters affects the quality of the final output image. Current techniques require multiple images of the 2D calibration object taken with a fisheye camera to obtain calibration images, followed by calibration using a corresponding algorithm. Reducing the number of images leads to poor robustness of the calibration results. Alternatively, 3D calibration can be performed using a fisheye camera, which places high demands on the manufacturing cost of the calibration object, its placement, and image quality. Therefore, considering time, cost, and robustness, none of the above methods are ideal. Summary of the Invention

[0005] The purpose of this invention is to provide a calibration method for fisheye cameras to solve the problems of insufficient overall performance in terms of time, cost and robustness in the prior art.

[0006] To address the aforementioned technical problems, the present invention provides a calibration method for a fisheye camera, the calibration method comprising the following steps: obtaining a calibration pattern; calibrating a first parameter based on the calibration pattern by solving for vanishing points; and calibrating a second parameter based on the calibration pattern and the first parameter.

[0007] The first parameter includes the focal length of the fisheye camera and the translation parameter from the image plane coordinate system to the image coordinate system; the second parameter includes the distortion coefficient of the fisheye camera.

[0008] Optionally, the calibration pattern is obtained by photographing a calibration board, which has a central checkerboard grid and multiple surrounding checkerboard grids. The central checkerboard grid is larger than the surrounding checkerboard grids, and the surrounding checkerboard grids are arranged around the central checkerboard grid.

[0009] Optionally, the number of surrounding chessboard squares is 4, and it satisfies one of the following characteristics:

[0010] The centers of the surrounding checkerboard squares are located at 0°, 90°, 180° and 270° from the center of the central checkerboard square, respectively, and the surrounding checkerboard squares are parallel to the central checkerboard square.

[0011] The y-axis of the surrounding chessboard squares points towards the four vertices of the central chessboard square in either a positive or negative direction, and the angle between the y-axis of the surrounding chessboard squares and the y-axis of the central chessboard square is 45°.

[0012] Optionally, the step of obtaining the calibration pattern is as follows: taking a picture of the calibration board only once, and obtaining the calibration pattern based on the picture result.

[0013] Optionally, the imaging steps of the fisheye camera include:

[0014] Calculate X camera =RX world +T, where X camera =[x camera y camera z camera ] T .

[0015] calculate and

[0016] calculate θ = arctan(r) and

[0017] calculate and

[0018] And, calculate u = a + c x and v = b + c y .

[0019] Among them, X camera The x-coordinate represents the coordinates of the feature point in the camera coordinate system. camera y camera z camera Representing X camera The x, y, and z components; X world Let R represent the coordinates of the feature point in the world coordinate system, R be the rotation transformation matrix from the world coordinate system to the camera coordinate system, and T be the translation vector from the world coordinate system to the camera coordinate system; f x Let f be the focal length of the fisheye camera in the x-direction. y Let i be the focal length of the fisheye camera in the y-direction; the value of i ranges from 1 to n, where n is a preset parameter, and k1 to k2 are also values. n c represents the distortion coefficient of the fisheye camera. xc is the translation parameter in the x-direction from the image plane coordinate system to the image coordinate system. y The translation parameter in the y-direction from the image plane coordinate system to the image coordinate system; u represents the x-coordinate of the feature point in the image coordinate system, and v represents the y-coordinate of the feature point in the image coordinate system.

[0020] The first parameter includes f x f y c x and c y The second parameter includes k1 to k n During the calibration of the second parameter, R and T also participate in the solution.

[0021] Optionally, the vanishing points include vanishing points in the x-direction and vanishing points in the y-direction; the step of calibrating the first parameter by solving for the vanishing points based on the calibration pattern includes:

[0022] Identify the corner points in the calibration pattern.

[0023] Based on the corner points, multiple x-axis elliptical segments with coincident major axes are fitted; wherein, the corner points located in the same x-axis straight line are fitted to the same x-axis elliptical segment, and the constraint conditions during fitting include: the endpoints of the multiple x-axis elliptical segments are common.

[0024] The two endpoints of the x-axis elliptical segment are set as the vanishing points in the x-axis direction.

[0025] Based on the corner points, multiple y-axis elliptical segments with coincident major axes are fitted; wherein, the corner points located in the same y-direction straight line are fitted to the same y-axis elliptical segment, and the constraint conditions during fitting include: the endpoints of the multiple y-axis elliptical segments are common.

[0026] The two endpoints of the y-direction elliptical segment are set as the vanishing points in the y-direction.

[0027] Furthermore, the first parameter is determined based on the vanishing point in the x-direction and the vanishing point in the y-direction.

[0028] Optionally, the step of determining the first parameter based on the vanishing point in the x-direction and the vanishing point in the y-direction includes:

[0029] calculate and Where dx is the distance to the vanishing point in the x-direction, and dy is the distance to the vanishing point in the y-direction.

[0030] Connect the vanishing points in the x-direction and the vanishing points in the y-direction to obtain the intersection point.

[0031] c is calculated based on the coordinates of the intersection point. x and cy .

[0032] Optionally, the step of calibrating the second parameter based on the calibration pattern and the first parameter includes: the calibration pattern and the first parameter participating in the imaging step of the fisheye camera, and using the original pattern of the calibration pattern as a comparison object, and obtaining the value of the second parameter based on an optimization algorithm.

[0033] To address the aforementioned technical problems, the present invention also provides a readable storage medium storing a program, which, when executed, performs the aforementioned calibration method.

[0034] Compared with existing technologies, the present invention provides a calibration method and readable storage medium for a fisheye camera. The calibration method includes the following steps: acquiring a calibration pattern; calibrating a first parameter based on the calibration pattern by solving for vanishing points; and calibrating a second parameter based on the calibration pattern and the first parameter. The first parameter includes the focal length of the fisheye camera and the translation parameter from the image plane coordinate system to the image coordinate system; the second parameter includes the distortion coefficient of the fisheye camera. This configuration solves the parameters to be calibrated in two steps, thereby avoiding the problems of large data requirements, long calculation time, unstable calibration results, and poor robustness caused by calibrating a large number of parameters at once. Under certain conditions, the above calibration method can complete the calibration with only one calibration image, solving the problems of unsatisfactory overall performance in terms of time, cost, and robustness in existing technologies. Attached Figure Description

[0035] Those skilled in the art will understand that the accompanying drawings are provided to better understand the invention and do not constitute any limitation on the scope of the invention. Wherein:

[0036] Figure 1 This is a schematic flowchart of a calibration method according to an embodiment of the present invention;

[0037] Figure 2 This is a schematic diagram of the projection of a physical point P in the world coordinate system onto the image plane.

[0038] Figure 3 This is a diagram of the vanishing point;

[0039] Figure 4 This is a schematic diagram of elliptical segment fitting according to an embodiment of the present invention;

[0040] Figure 5 This is a schematic diagram illustrating the calibration method for calculating the first parameter according to an embodiment of the present invention;

[0041] Figure 6a This is a schematic diagram of a calibration plate according to an embodiment of the present invention;

[0042] Figure 6b This is a schematic diagram of the pattern of a calibration plate according to another embodiment of the present invention;

[0043] Figure 7a These are the results of images taken with a fisheye camera calibrated using existing technology;

[0044] Figure 7b This is an embodiment of the calibration method of the present invention for shooting with a fisheye camera after calibration. Figure 7a The results of shooting the same scene.

[0045] In the attached image:

[0046] 1-Parallel lines; 2-Optical axis; 3-Fisheye lens; 4-Image plane; 5-Image of parallel lines; 6-Vanetage point; 7-Vanetage point in the x-direction; 8-Vanetage point in the y-direction; 9-Intersection point; 10-Y-axis of the surrounding checkerboard pattern. Detailed Implementation

[0047] To make the objectives, advantages, and features of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that the drawings are all in a very simplified form and are not drawn to scale, and are only used to facilitate and clarify the explanation of the embodiments of this invention. Furthermore, the structures shown in the drawings are often part of the actual structures. In particular, different figures may emphasize different aspects and may sometimes use different scales.

[0048] As used in this invention, the singular forms “a,” “an,” and “the” include plural objects; the term “or” is generally used to mean “and / or”; the term “a number” is generally used to mean “at least one”; and the term “at least two” is generally used to mean “two or more”. Furthermore, the terms “first,” “second,” and “third” are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as “first,” “second,” or “third” may explicitly or implicitly include one or at least two of that feature. “One end” and “the other end,” as well as “proximal end” and “distal end,” generally refer to two corresponding parts, including not only endpoints. The terms “installed,” “connected,” and “joined” should be interpreted broadly, for example, as a fixed connection, a detachable connection, or an integral part; a mechanical connection or an electrical connection; a direct connection or an indirect connection through an intermediate medium; or a connection within two elements or an interaction between two elements. Furthermore, as used in this invention, the phrase "one element is disposed on another element" generally only indicates that there is a connection, coupling, cooperation, or transmission relationship between the two elements, and the connection, coupling, cooperation, or transmission between the two elements can be direct or indirect through an intermediate element. It should not be construed as indicating or implying a spatial positional relationship between the two elements, i.e., one element can be located arbitrarily inside, outside, above, below, or to one side of the other element, unless otherwise explicitly stated. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0049] The core idea of ​​this invention is to provide a calibration method for fisheye cameras to solve the problems of insufficient overall performance in terms of time, cost and robustness in the existing technology.

[0050] The following description refers to the accompanying drawings.

[0051] Please refer to Figure 1 The present invention provides a calibration method for a fisheye camera, the calibration method comprising the following steps:

[0052] S10: Obtain the calibration pattern.

[0053] S20: Based on the calibration pattern, calibrate the first parameter by solving for the vanishing point.

[0054] And, S30: Based on the calibration pattern and the first parameter, calibrate the second parameter.

[0055] The first parameter includes the focal length of the fisheye camera and the translation parameter from the image plane coordinate system to the image coordinate system; the second parameter includes the distortion coefficient of the fisheye camera.

[0056] To explain the specific meaning of the above parameters, the imaging steps of the fisheye camera involved in this embodiment are described below.

[0057] Please refer to Figure 2 , Figure 2 In this diagram, the coordinate system formed by the Xc, Yc, and Zc axes is the camera coordinate system. This camera coordinate system uses the camera optical axis as the z-axis and the lens plane as the xOy plane, with its origin on the camera optical axis. The coordinate system formed by the x and y axes is the image plane coordinate system. This image plane coordinate system uses the sensor plane as the xOy plane, with its origin also on the optical axis. The distance between the origins of the camera coordinate system and the image plane coordinate system is the focal length f.

[0058] Outside the camera, there exists a physical point P. If the fisheye camera is a pinhole camera, then the image point of point P is... Figure 2 However, since the fisheye camera is not a pinhole imaging model, the light path bends when projected through the lens, and therefore, it will eventually be projected onto point p instead of point p'.

[0059] However, there is a connection between points p and p'. First, points p, p', and the origin of the image plane coordinate system are collinear. Second, there is a functional relationship between the distances from point p to the origin and the distances from point p' to the origin, which can be called d(θ). Here, θ is the angle between the line connecting point P to the camera coordinate system and the Zc axis. Figure 2 In This represents the angle between the line connecting point p and the origin and the x-axis, and is unrelated to the main content of this embodiment.

[0060] Although the expressions for d(θ) differ based on different theories, they can all be approximated by a Taylor expansion, and the Taylor expansions are identical in form. Among them, k' j Let k'1 be the coefficient to be determined. We can divide each term in the expression by k'1, remove higher-order infinitesimals, and take the first n+1 terms to obtain... Considering computational cost and accuracy, n can be 3 or 4, etc.

[0061] By combining the world coordinate system and the image coordinate system, the complete imaging process can be obtained. The world coordinate system is set according to actual needs, generally selecting the upper left corner of the chessboard being photographed as the origin, the chessboard plane as the xOy plane, and the direction perpendicular to the chessboard as the z-axis. The image coordinate system uses the sensor plane as the xOy plane, generally with the upper left corner as the origin, but other origins may be set for special needs.

[0062] The imaging steps of the fisheye camera include:

[0063] Calculate X camera =RX world +T, where X camera =[x camera y camera z camera ] T .

[0064] calculate and

[0065] calculate θ = arctan(r) and

[0066] calculate and

[0067] And, calculate u = a + c x and v = b + c y .

[0068] Among them, X camera The x-coordinate represents the coordinates of the feature point in the camera coordinate system. camera y camera z camera Representing X camera The x, y, and z components; X world X represents the coordinates of the feature point in the world coordinate system, R is the rotation transformation matrix from the world coordinate system to the camera coordinate system, and T is the translation vector from the world coordinate system to the camera coordinate system. world In some computational analyses, R and T need to be refined to each element of a vector or matrix. However, for the purposes of this embodiment, these three parameters are understood as a whole, without delving into their specific elements. x Let f be the focal length of the fisheye camera in the x-direction. y Let be the focal length of the fisheye camera in the y-direction; the value of i ranges from 1 to n, where n is a preset parameter, i.e., the number of terms in the Taylor expansion minus 1. k1~k n Also referred to as the distortion coefficient of the fisheye camera; c x c is the translation parameter in the x-direction from the image plane coordinate system to the image coordinate system. y The translation parameter in the y-direction from the image plane coordinate system to the image coordinate system; u represents the x-coordinate of the feature point in the image coordinate system, and v represents the y-coordinate of the feature point in the image coordinate system.

[0069] Based on the above description, it can be seen that the first parameter includes f x f y cx and c y The second parameter includes k1 to k n Furthermore, during the calibration of the second parameter, R and T also participate in the solution process. When both the first and second parameters are known, the fisheye camera can output an image with a normal aspect ratio that is easy for the operator to recognize. When the first and second parameters are unclear, calibration is required using a specific method.

[0070] In existing technologies, the above parameters are calibrated simultaneously. Therefore, the calibration processes of each parameter affect each other, making it difficult to achieve a good calibration result. To improve the calibration effect, it is often necessary to take multiple shots or set up special calibration subjects (such as 3D objects).

[0071] In this embodiment, a portion of the parameters are obtained by solving for the vanishing point, and then another portion of the parameters are solved. This reduces the number of parameters that need to be calibrated in each step, thereby improving robustness and achieving the beneficial effect of reducing the number of shots.

[0072] Please refer to Figure 3 , Figure 3 A schematic diagram of the vanishing point is shown. When a set of parallel lines 1 are imaged on the image plane 4 along the optical axis 2 through the fisheye lens 3, the images 5 of all the parallel lines intersect at two points, which are the vanishing points 6. Therefore, some parameters, namely the first parameter, can be inferred from the vanishing points.

[0073] Specifically, the vanishing points include the x-axis vanishing point and the y-axis vanishing point. Here, "x-axis" and "y-axis" mean "directions toward the x / y axes of the image coordinate system," but they are not strictly parallel. Understandably, since it's impossible to achieve perfect alignment during shooting, and subsequent imaging and fitting cannot be perfectly horizontal or vertical, the "x-axis" and "y-axis" are not entirely equivalent to the "direction of the x-axis" and "direction of the y-axis," and include a range within ±5°.

[0074] The step of calibrating the first parameter by solving for vanishing points based on the calibration pattern includes:

[0075] Identify the corner points in the calibration pattern. Corner points refer to the four vertices of each small rectangle in the checkerboard grid.

[0076] Based on the corner points, multiple x-axis elliptical segments with coincident major axes are fitted; wherein, the corner points located on straight lines in the same x-axis are fitted to the same x-axis elliptical segment. Here, "straight lines" refers to the straight lines in the photographed pattern, not the straight lines after imaging; subsequent y-axis straight lines are understood in the same way. An "elliptical segment" is defined as a curve that is part of an ellipse, but not a complete ellipse. In practice, an "elliptical segment" is often slightly less than half the length of an ellipse. The fitting constraints include: the endpoints of multiple x-axis elliptical segments must be concurrent. Concurrent endpoints here refer to the left and right endpoints of any two endpoints being concurrent. Specific fitting algorithms can be set according to actual needs and will not be described in detail here.

[0077] The two endpoints of all the x-axis elliptical segments are set as the vanishing points of the x-axis.

[0078] Based on the corner points, multiple y-axis elliptical segments with coincident major axes are fitted; wherein, the corner points located in the same y-direction straight line are fitted to the same y-axis elliptical segment, and the constraint conditions during fitting include: the endpoints of the multiple y-axis elliptical segments are common.

[0079] The two endpoints of the y-direction elliptical segment are configured as the vanishing points in the y-direction.

[0080] Furthermore, the first parameter is determined based on the vanishing point in the x-direction and the vanishing point in the y-direction.

[0081] The above process can be referred to Figure 4 To understand. Figure 4 The figure shows the elliptical segment obtained after fitting, the vanishing point 7 in the x-direction, and the vanishing point 8 in the y-direction.

[0082] Further, the step of determining the first parameter based on the vanishing point in the x-direction and the vanishing point in the y-direction includes:

[0083] calculate and Where dx is the distance to the vanishing point in the x-direction, and dy is the distance to the vanishing point in the y-direction. Figure 5 The diagram shows dx and dy.

[0084] Connect the vanishing point in the x-direction and the vanishing point in the y-direction to obtain intersection point 9.

[0085] c is calculated based on the difference between the actual and theoretical positions of the intersection point 9. x and c y .

[0086] The calculation process can be referenced. Figure 5To understand this, we calculate cx and cy based on the coordinates of intersection point 9. As mentioned earlier, for ease of solution, the intersection of the vanishing points is used as the origin of the coordinate axes. The coordinates of this point are also a parameter to be solved in the optimization. After solving for it, we know the offset relative to the upper left corner of the image, which is c. x and c y .

[0087] Based on the above, the calibration method for the fisheye camera has been optimized in terms of process. In a preferred embodiment, the calibration board and the shooting process have also been optimized.

[0088] In existing technologies, at least 20 independent checkerboard patterns are generally required to complete the calibration process. Since this embodiment optimizes the calibration process, the number of checkerboard patterns is reduced. For example, five checkerboard patterns can be placed on one calibration board, obtaining five sets of data in a single shot, thus reducing the number of shots. It should be understood that placing five checkerboard patterns on one calibration board is not a conventional technique, as existing technologies require at least 20 images for calibration to achieve good results, and 20 images (due to resolution and other issues) obviously cannot be simultaneously placed on a single calibration board.

[0089] Three-dimensional calibration objects and two-dimensional calibration boards cannot be treated the same. For three-dimensional calibration objects to achieve good calibration results, strict requirements are placed on the angles between their planes, making replication more difficult. Two-dimensional calibration boards, on the other hand, can actually be replicated by printing and pasting the paper onto a rigid board; however, the replication costs differ.

[0090] Furthermore, the inventors considered the impact of the relative positions of the five checkerboard squares on the final effect and adopted various settings. This embodiment demonstrates two calibration board patterns, such as... Figure 6a and Figure 6b As shown. Both share the characteristic of having a central checkerboard grid and multiple surrounding checkerboard grids on the calibration plate. The central checkerboard grid is larger than the surrounding checkerboard grids; here, "size" refers to the dimensions of the small rectangles that make up the checkerboard grid, as well as the overall size of the checkerboard grid. The surrounding checkerboard grids are arranged around the central checkerboard grid.

[0091] Specifically, Figure 6a The proposed scheme is as follows: there are four surrounding chessboard squares, with their centers located at 0°, 90°, 180°, and 270° from the center of the central chessboard square, respectively. The surrounding chessboard squares are parallel to the central chessboard square. "Parallel" should be understood as the chessboard squares being parallel along their x-axis (or y-axis).

[0092] Figure 6bThe proposed solution is as follows: there are four surrounding chessboard squares, and the y-axis 10 of the surrounding chessboard squares are directed towards the four vertices of the central chessboard square in either a forward or reverse direction. The angle between the y-axis 10 of the surrounding chessboard squares and the y-axis of the central chessboard square is 45°.

[0093] Figure 6b The diagram shows the y-axis 10 of the surrounding checkerboard. It should be understood that the arrow symbol representing the y-axis 10 of the surrounding checkerboard is not itself part of the pattern on the calibration board. In this embodiment, the y-axis refers to the direction perpendicular to the length of the checkerboard. There are two possible directions, therefore, they are described as "forward or reverse" to clarify their meaning. "Orientation" means that the vertex of the central checkerboard is within an angular range in that direction, for example, within ±5°, or within a range of distance from the y-axis 10 of the surrounding checkerboard, for example, ±1cm; rather than strictly referring to passing through it. A checkerboard refers to a large matrix composed of multiple closely arranged small matrices of alternating light and dark colors, and the specific colors of the light and dark areas are not limited to white and black.

[0094] Understandable. Figure 6a and Figure 6b This is merely an exemplary embodiment. In other embodiments, the number of surrounding chessboard squares is not limited to four, and the positions of the surrounding chessboard squares are not limited to a precise location; they only need to be distributed around the central chessboard square.

[0095] Based on the aforementioned calibration board, this method can be further optimized as follows: the specific steps for obtaining the calibration pattern are: taking only one photograph based on the calibration board, and obtaining the calibration pattern based on the photographing result. When calibrating the first parameter, only the pattern of the central checkerboard is used; when calibrating other parameters, the pattern is split into 5 independent checkerboard patterns, and then optimized and solved.

[0096] This method can be used in conjunction with the aforementioned calibration board to achieve the beneficial effect of a single shot; it can also be used with a standard two-dimensional checkerboard to reduce the number of shots (but not just one shot). It should not be understood that this method can only work with the aforementioned calibration board.

[0097] The step of calibrating the second parameter based on the calibration pattern and the first parameter includes: the calibration pattern and the first parameter participating in the imaging step of the fisheye camera, and using the original pattern of the calibration pattern as a comparison object, obtaining the value of the second parameter based on an optimization algorithm. The original pattern refers to the pattern from which the calibration pattern is captured, such as a set of parallel lines, one or more checkerboard patterns, etc.

[0098] An optimization algorithm is a method that selects a research scheme to achieve the optimal goal under certain constraints. In this specification, it refers to a method that uses the difference between the output result and the original pattern as the objective, and optimizes the second parameter, R, and T as variable factors, with the objective being to minimize the difference. Least squares is one type of optimization algorithm.

[0099] Specifically, the objective function of the optimization algorithm can be set according to actual needs, such as the difference in pixel position, the difference in straight line curvature, or the difference in content coverage between the actual output image and the original image; it can also be a comprehensive index of the above evaluation indicators.

[0100] Figure 7a The images show the results captured by a fisheye camera after calibration using existing technology (20 images). Figure 7b This paper demonstrates the image capture results of a fisheye camera calibrated using a calibration method according to an embodiment of the present invention. Both images captured the same scene. The image is a composite of three perspectives, with the central image representing the main field of view, where objects and lines conform to human subjective perception. The two outer images represent the larger field of view areas; in terms of content completeness, the results are similar. The calibration effect of this embodiment is comparable to traditional methods. However, the calibration process is significantly faster than traditional methods.

[0101] This embodiment also provides a readable storage medium storing a program, which, when executed, performs the aforementioned calibration method. Because the readable storage medium can run a program, it also possesses the advantages of short calibration time, low cost, and good robustness.

[0102] In summary, this embodiment provides a calibration method for a fisheye camera and a readable storage medium. The calibration method includes the following steps: acquiring a calibration pattern; calibrating a first parameter based on the calibration pattern by solving for vanishing points; and calibrating a second parameter based on the calibration pattern and the first parameter. The first parameter includes the focal length of the fisheye camera and the translation parameters from the image plane coordinate system to the image coordinate system; the second parameter includes the distortion coefficients of the fisheye camera. This configuration solves for the parameters to be calibrated in two steps, thus avoiding the problems of large data requirements, long computation time, unstable calibration results, and poor robustness caused by calibrating a large number of parameters at once. Under certain conditions, the above calibration method can complete the calibration with only one calibration image, solving the problems of insufficient overall performance in terms of time, cost, and robustness in the prior art.

[0103] The above description is only a description of preferred embodiments of the present invention and is not intended to limit the scope of the present invention in any way. Any changes or modifications made by those skilled in the art based on the above disclosure shall fall within the protection scope of the present invention.

Claims

1. A method for calibrating a fisheye camera, characterized in that, The calibration method comprises the following steps: obtaining a calibration pattern; calibrating a first parameter in a manner of solving vanishing points based on the calibration pattern; and calibrating a second parameter based on the calibration pattern and the first parameter; wherein the first parameter comprises a focal length of the fisheye camera and a translation parameter from an image plane coordinate system to an image coordinate system; and the second parameter comprises a distortion coefficient of the fisheye camera; the imaging step of the fisheye camera comprises: Compute X camera = RX world + T, where X camera = [x camera y camera z camera ] T ; Computing and ; Computing , and ; Computing and ; and, Computing and ; wherein X camera represents the coordinates of the feature point in the camera coordinate system, x camera , y camera , and z camera represent the x component, the y component, and the z component of X camera , respectively; X world represents the coordinates of the feature point in the world coordinate system, R is a rotation transformation matrix of the world coordinate system to the camera coordinate system, and T is a translation vector of the world coordinate system to the camera coordinate system; f x is the focal length of the fisheye camera in the x direction, f y is the focal length of the fisheye camera in the y direction; the value range of i is 1-n, wherein n is a preset parameter, and k1-k n are distortion coefficients of the fisheye camera; c x is a translation parameter of the x direction of the image plane coordinate system to the image coordinate system, and c y is a translation parameter of the y direction of the image plane coordinate system to the image coordinate system; u represents the x coordinate of the feature point in the image coordinate system, and v represents the y coordinate of the feature point in the image coordinate system. The first parameters include f x , f y , c x , and c y , and the second parameters include k1~k n ; in the process of calibrating the second parameters, R and T are solved together. the vanishing points comprise x-direction vanishing points and y-direction vanishing points; the step of calibrating the first parameter in a manner of solving vanishing points based on the calibration pattern comprises: identifying corner points in the calibration pattern; fitting a plurality of x-direction elliptical segments based on the corner points; wherein the corner points located in the same x-direction straight line are fitted to the same x-direction elliptical segment, and the constraint condition in the fitting comprises that end points of the plurality of x-direction elliptical segments are co-pointed; two end points of the x-direction elliptical segment are set as the x-direction vanishing points; fitting a plurality of y-direction elliptical segments based on the corner points; wherein the corner points located in the same y-direction straight line are fitted to the same y-direction elliptical segment, and the constraint condition in the fitting comprises that end points of the plurality of y-direction elliptical segments are co-pointed; two end points of the y-direction elliptical segment are set as the y-direction vanishing points; and determining the first parameter based on the x-direction vanishing points and the y-direction vanishing points comprises: Computing and where dx is the distance of the vanishing point in the x direction, dy is the distance of the vanishing point in the y direction; connecting the vanishing points in the x direction, connecting the vanishing points in the y direction, obtaining the intersection point; calculating c x and c y based on the coordinates of the intersection point.

2. The calibration method of claim 1, wherein the calibration pattern is obtained based on shooting a calibration board, and the calibration board is provided with a center checkerboard and a plurality of surrounding checkerboards, the size of the center checkerboard is greater than that of the surrounding checkerboards, and the surrounding checkerboards are arranged around the center checkerboard.

3. The calibration method of claim 2, wherein, The number of the surrounding checkerboards is four, and one of the following features is satisfied: the centers of the surrounding checkerboards are respectively located at 0°, 90°, 180° and 270° of the center of the center checkerboard, and the surrounding checkerboards are parallel to the center checkerboard; y-axes of the surrounding checkerboards are respectively positively or negatively oriented to four vertices of the center checkerboard, and the angle between the y-axes of the surrounding checkerboards and the y-axis of the center checkerboard is 45°.

4. The calibration method of claim 2, wherein The step of obtaining the calibration pattern specifically comprises: shooting the calibration board only once, and obtaining the calibration pattern based on the shooting result.

5. The calibration method of claim 1, wherein The step of calibrating the second parameter based on the calibration pattern and the first parameter comprises: the calibration pattern and the first parameter participate in the imaging step of the fisheye camera, the original pattern of the calibration pattern is taken as a comparison object, and the value of the second parameter is obtained based on an optimization algorithm.

6. A readable storage medium characterized by, The readable storage medium stores a program, and the program runs to execute the calibration method in any one of claims 1-5.

Images