A linear transformation-based equidistance cylindrical fisheye image unwrapping method

Abstract

The application discloses an equidistance cylindrical fish-eye image unfolding method based on linear transformation, and the method comprises the following steps: obtaining the internal parameter and distortion parameter of a fish-eye camera through camera calibration; obtaining an equidistance cylindrical fish-eye image unfolding image by using the projection principle of equidistance cylindrical surface; realizing linear transformation of the equidistance cylindrical fish-eye image unfolding image by changing the corresponding parameters of the field of view; and finally obtaining the equidistance cylindrical fish-eye image unfolding image, so that the method is suitable for a fish-eye lens with a larger field of view, can effectively remove distortion, and significantly improves the visual effect and practicability of the unfolded image.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, and more specifically, relates to a method for unfolding equidistant cylindrical fisheye images based on linear transformation. Background Technology

[0002] Fisheye cameras, with their wide field of view and panoramic imaging capabilities, are widely used in virtual reality, robot vision, autonomous driving, and environmental monitoring. However, the extreme distortion of fisheye images presents numerous challenges in subsequent image analysis and processing. Fisheye image unfolding is the process of converting images captured by fisheye lenses into images with a conventional viewpoint. Therefore, how to efficiently and accurately unfold fisheye images has become an important research topic in the field of image processing.

[0003] When unfolding a fisheye image, the image pixels need to be resampled and pixel coordinates transformed to ensure that the shape of objects in the unfolded image is both intuitive and recognizable. However, this process often results in geometric distortion and photometric distortion.

[0004] Traditional fisheye image unfolding methods are mainly divided into geometric transformation methods and physical modeling methods. Geometric transformation methods establish a mathematical model between the fisheye image and the target projection, performing image mapping transformations to remove distortions in the fisheye image. Common examples include fast unfolding and 30-degree rotation unfolding. However, these algorithms require extensive interpolation calculations, resulting in significant computational overhead and poor real-time performance. Physical modeling methods are based on the physical properties of optical lenses, constructing a projection model of the fisheye camera for image unfolding. Common examples include cylindrical unfolding and linear projection. These methods are effective in addressing some distortions, but for fisheye lenses with large fields of view, they still face significant image distortion problems. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of the prior art and provide an equidistant cylindrical fisheye image unfolding method based on linear transformation. Based on the unit spherical projection model of the fisheye image, the equidistant cylindrical unfolding diagram is generated by performing inverse projection of the fisheye image into spherical space and combining it with a horizontal mapping method.

[0006] To achieve the above-mentioned objective, the present invention provides a method for unfolding equidistant cylindrical fisheye images based on linear transformation, characterized by comprising the following steps:

[0007] (1) Perform parameter calibration on the fisheye camera;

[0008] Multiple images of the calibration plate were captured from different angles using a fisheye camera to detect corner points, and the camera intrinsic parameters and distortion parameters were calibrated using Zhang Zhengyou's calibration method.

[0009] (2) Construct a template image with all pixel values ​​of 0. The size of the template image is equal to the equidistant cylindrical unfolded diagram of the fisheye image.

[0010] (3) Take a fisheye image using a calibrated fisheye camera, and then unfold the fisheye image into an equidistant cylindrical surface.

[0011] The objective of this invention is achieved as follows:

[0012] This invention relates to an equidistant cylindrical fisheye image unfolding method based on linear transformation. It obtains the intrinsic parameters and distortion parameters of the fisheye camera through camera calibration, then uses the projection principle of equidistant cylinders to obtain an equidistant cylindrical unfolded image of the fisheye image. By changing the corresponding parameters of the field of view angle, a linear transformation of the equidistant cylindrical unfolded image is achieved, ultimately yielding an equidistant cylindrical unfolded image of the fisheye image. This method is suitable for fisheye lenses with a large viewing angle, effectively removing distortion and significantly improving the visual effect and practicality of the unfolded image.

[0013] Meanwhile, the equidistant cylindrical fisheye image unfolding method based on linear transformation of the present invention also has the following beneficial effects:

[0014] (1) By directly establishing the mapping relationship between the fisheye image and the unfolded image, this method has the advantage of low computational overhead. At the same time, this method is applicable to fisheye lenses with a large field of view, can effectively remove distortion, and significantly improve the visual effect and practicality of the unfolded image.

[0015] (2) This invention obtains the intrinsic parameters and distortion parameters of the fisheye camera through camera calibration, and uses the projection principle of equidistant cylindrical surfaces to obtain the equidistant cylindrical unfolded image of the fisheye image. By changing the corresponding parameters of the field of view angle, a linear transformation of the pixels in the unfolded image is achieved, which can directly obtain the mapping relationship between the fisheye image and the unfolded image. This method can not only remove fisheye distortion and maintain the true proportion of the image structure, but also cover all pixel information of the original fisheye image, avoiding data loss, and has good practicality.

[0016] (3) The present invention has low computational overhead. For fisheye lenses with a large field of view, it can retain all pixel information of the image while removing distortion to the greatest extent and avoiding image distortion and jaggedness, thus achieving good visual effects. Attached Figure Description

[0017] Figure 1 This is a flowchart of the method for unfolding equidistant cylindrical fisheye images based on linear transformation according to the present invention;

[0018] Figure 2 This is a schematic diagram of the principle of unfolding equidistant cylindrical surfaces;

[0019] Figure 3 It is a camera coordinate system mapping diagram;

[0020] Figure 4 This is a diagram showing the unfolded effect of equidistant cylindrical surfaces;

[0021] Figure 5 It is a linear transformation effect diagram of the equidistant cylindrical surface development diagram. Detailed Implementation

[0022] The specific embodiments of the present invention will now be described with reference to the accompanying drawings to enable those skilled in the art to better understand the invention. It should be particularly noted that in the following description, detailed descriptions of known functions and designs that might obscure the main content of the invention will be omitted here.

[0023] Example

[0024] Figure 1 This is a flowchart of the method for unfolding equidistant cylindrical fisheye images based on linear transformation according to the present invention.

[0025] In this embodiment, as Figure 1 As shown, the present invention provides a method for unfolding equidistant cylindrical fisheye images based on linear transformation, comprising the following steps:

[0026] S1. Perform parameter calibration on the fisheye camera;

[0027] Multiple calibration board images were captured from different perspectives using a fisheye camera. Then, corner points in each calibration board image were detected, and the camera intrinsic parameters and distortion parameters were calibrated using the Zhang Zhengyou calibration method.

[0028] S1.1, Checkerboard corner detection;

[0029] S1.1.1 Select a checkerboard calibration board with known size and spacing, place the checkerboard calibration board at different positions within the field of view of the fisheye camera, try to cover the wide-angle view of the fisheye lens, and then take multiple images of the calibration board from different angles.

[0030] S1.1.2 For each calibration board image, calculate the local gradient change of each pixel in the calibration board image, and the horizontal gradient I of each pixel. x and vertical gradient I y They are respectively:

[0031]

[0032] Among them, I (x,y) This represents the pixel value at coordinates (x, y) in the calibration plate image, where x and y are the horizontal and vertical coordinates of the pixel.

[0033] S1.1.3 Calculate the gradient matrix M of each pixel in the 3×3 neighborhood of each calibration board image. j ;

[0034]

[0035] Among them, M j This represents the gradient matrix of the j-th pixel in its 3×3 neighborhood.

[0036] S1.1.4 Calculate the response value R of each pixel in each calibration plate image. j :

[0037] R j =det(M j )-K(trace(M j ) 2 )

[0038] Among them, det(M j )=λ1λ1 represents matrix M j The determinant, trace(M) j )=λ1+λ2 represents matrix M j The traces, λ1, λ2 are matrices M j The eigenvalues, K is the weight used to balance the determinant and the trace;

[0039] In this embodiment, the parameter K is typically between 0.04 and 0.06, used to balance the weights of the determinant and the trace.

[0040] S1.1.5 When the response value is positive and large, it indicates that the pixel has significant corner characteristics. Therefore, we can filter corners by setting a threshold, and then traverse each pixel in each calibration board image to find the response value R. j Pixels larger than a threshold are marked as corner points; for fisheye lens calibration, these corner points can be extracted as calibration feature points within the fisheye view, which can effectively extract the intersection points of the checkerboard pattern.

[0041] S1.2, Perform sub-pixel precision on the corner points of each calibration board image;

[0042] Iterate through each corner point in each calibration board image, select pixels within a 1×3 neighborhood centered on the corner point, and perform secondary interpolation to obtain the sub-pixel x-coordinate value of the corner point. sub ;

[0043]

[0044] Where x0 represents the x-coordinate value of the corner point in the calibration plate image, This represents the pixel values ​​of the two pixels to the left and right of the corner point. Represents the pixel value of a corner point;

[0045] After calculating a more accurate corner position through interpolation, the original corner position is updated, achieving sub-pixel level accuracy.

[0046] S1.3 Fisheye camera parameter calibration;

[0047] S1.3.1. Fix the world coordinate system on the chessboard calibration board, then the world coordinates of the corner points on the chessboard calibration board are denoted as P(X). w ,Y w Z w ), where X w ,Y w Z w This represents the values ​​of the X, Y, and Z axes, with Z... w =0;

[0048] S1.3.2, Using the rotation matrix R and translation vector T, the world coordinates P(X) of the corner point are changed. w ,Y w Z w Convert to camera coordinates P c (X c ,Y c Z c );

[0049]

[0050] S1.3.3. Based on the equidistant projection model, point P in the camera coordinate system... c (X c ,Y c Z c Convert the coordinates of u and v to point p(u,v) in pixel coordinates.

[0051]

[0052] Among them, (c x ,c y ) is the pixel coordinate of the center point of the calibration board image, f is the focal length of the fisheye camera, r is the radius of the projection point on the calibration board image plane, and θ is the incident angle;

[0053] S1.3.4. Obtain the corresponding distortion coordinates p(u) in the pixel coordinate system using the Kannala-Brandt model. d ,v d );

[0054] r d =θ+k1θ 3 +k2θ 5 +k3θ 7 +k4θ 9

[0055]

[0056] Where, r d The distorted radial distance, (u d ,v d ) represents the distorted pixel coordinates, and k1 to k4 represent the distortion coefficients;

[0057] S1.3.5. Acquire corner points from multiple calibration board images and record the pixel coordinates (u) of each corner point. i ,v i ) and distorted pixel coordinates (u d,i ,v d,i );

[0058] S1.3.6. The reprojection error E is minimized by nonlinear optimization method, thereby determining the intrinsic parameter matrix of the fisheye camera: rotation matrix R and translation vector T, as well as the distortion coefficients: k1~k4.

[0059]

[0060] Where N is the total number of corner points detected;

[0061] In this embodiment, the mean of the reprojection error is used to evaluate the calibration accuracy to ensure that the model can accurately describe the imaging characteristics of the fisheye camera.

[0062] S2. Construct a template image with width W, length H, and all pixel values ​​of 0. The pixel coordinates of each pixel in the template image are represented as p(i,j), where W = 2 × H.

[0063] In this embodiment, the equidistant cylindrical unfolded diagram is a horizontal unfolding of a unit sphere; therefore, the horizontal field of view is (-π, π), and the vertical field of view is... Therefore, we first construct a template image of the same size as the unfolded diagram of the equidistant cylindrical surface, with its width and height satisfying W = 2H;

[0064] S3, fisheye image equidistant cylindrical surface unfolding;

[0065] S3.1. Take a fisheye image using the calibrated fisheye camera, such as... Figure 4 As shown in (a);

[0066] S3.2. Establish a right-handed spatial coordinate system O-XYZ with the center O of the unit spherical model as the origin;

[0067] S3.3. Use the horizontal mapping method to find the polar coordinates of each pixel point p(i,j) in the right-handed coordinate system O-XYZ.

[0068]

[0069] in, δ is the angle between the line connecting the spatial coordinate point P(X,Y,Z) corresponding to pixel point p(i,j) and the center of the sphere O and the XOZ plane, and δ is the angle between the line connecting the projection point P′(X,0,Z) of the spatial coordinate point P(X,Y,Z) on the XOZ plane and the center of the sphere O and the positive half-axis of the Z-axis.

[0070] In this embodiment, the equidistant cylindrical unfolding algorithm designed by the present invention is essentially based on the unit spherical projection model of the fisheye image, and uses the horizontal mapping method to construct an inverse mapping between the equidistant cylindrical unfolding diagram and the fisheye image.

[0071] Fisheye camera unit spherical model, such as Figure 2 As shown in (a), its equidistant cylindrical surface development diagram is as follows: Figure 2 As shown in (b), on the unit spherical model, l1 is the line connecting the spatial coordinate point P(X,Y,Z) to the center O of the sphere, and l2 is the line connecting the projection point P′(X,0,Z) of the spatial coordinate point P(X,Y,Z) onto the XOZ plane to the center O of the sphere. Let... Let l1 be the angle between l1 and the XOZ plane, and δ be the angle between l2 and the positive Z-axis. Then, for any point p(i,j), the angle can be obtained using the horizontal mapping method. Regarding the angle δ, since the equidistant cylindrical development is a horizontal mapping of a unit sphere, therefore the angle... The range is The range of δ is (-π, π).

[0072] S3.4. Using the transformation relationship between polar coordinates and rectangular coordinates, obtain the polar coordinates. The world coordinates P(X,Y,Z) in the right-handed coordinate system O-XYZ;

[0073]

[0074] S3.5. Normalize the world coordinates P(X,Y,Z) to obtain the normalized world coordinates Q′(X′,Y′,Z′). The normalized coordinates can be regarded as coordinates in the camera coordinate system, such as... Figure 3 As shown;

[0075] X′=X / Z

[0076] Y′=Y / Z

[0077] Z' = Z / Z = 1

[0078] S3.6. Using the normalized world coordinates Q′(X′,Y′,Z′), calculate the distortion coordinates p(i,j) corresponding to pixel p(i,j) using the Kannala-Brandt model of the fisheye camera. d ,jd );

[0079] r d =θ+k1θ 3 +k2θ 5 +k3θ 7 +k4θ 9

[0080] i d =f x *X'*scale+c x

[0081] j d =f y *Y'*scale+c y

[0082] Among them, f x ,f y Let be the focal lengths of the fisheye camera on the X and Y axes, respectively; and r be the radial distance of the world coordinates Q′(X′,Y′,Z′) projected onto the image plane, satisfying the following conditions: scale is a scaling factor, and satisfies scale = r d / r;c x ,c y Here are the pixel coordinates of the center point of the captured fisheye image; the incident angle θ = atan(r);

[0083] S3.7, Traverse each distorted coordinate p(i) d ,j d ), determine if each coordinate value is an integer, if the coordinate value of each point i is an integer. d ,j d If all values ​​are integers, then the pixel value at position p(i,j) in the expanded image is directly replaced with the pixel value at position p(i,j) in the distorted coordinates p(id,jd) of the fisheye image; otherwise, proceed to step S3.8.

[0084] S3.8, Distortion coordinates p(i) for non-integer values d ,j d Perform bilinear interpolation resampling;

[0085]

[0086] in, p(i) d ,j d The coordinate representation after bilinear interpolation resampling, (a,b) represents (i d ,j d The integer part of (di) d ,dj d ) represents (i d,j d The fractional part of ) represents the image value of the fisheye image at the integer coordinates (a,b);

[0087] Using distortion coordinates in fisheye images The pixel value at position p(i,j) in the unfolded image is replaced by the pixel value at position p(i,j) to obtain an initial equidistant cylindrical unfolded image. By using bilinear interpolation, the pixel points in the unfolded image can be smoothly transitioned, and image distortion and jaggedness can be avoided, resulting in a good visual effect.

[0088] S3.9 Perform a linear projection transformation on the initial equidistant cylindrical surface development;

[0089] In this embodiment, the initial equidistant cylindrical surface unfolded diagram is as follows: Figure 4 As shown in (b), a large number of shadows are not filled, which is not conducive to visual observation. If the expansion is directly performed, a large area of ​​pixel accumulation will occur, which cannot achieve the ideal visual effect. Therefore, this invention proposes an expansion method based on linear transformation to achieve uniform expansion of pixel information in the unfolded image. The specific process is as follows:

[0090] S3.9.1 Set the pixel information linear expansion variable n, such that H′=nH, W′=nW, n>1, and in this embodiment, n is 2.88;

[0091] S3.9.2. Traverse each pixel point p(i,j) in the initial equidistant cylindrical unfolded diagram, and calculate the polar coordinates of p(i,j) in the right-handed coordinate system O-XYZ under linear expansion of pixel information.

[0092]

[0093] S3.9.3, Order Given δ′=δ, find the correspondence between i′, j′ and i, j;

[0094]

[0095] S3.9.4 Utilizing the distortion coordinates in fisheye images Replace the pixel value at position p(i,j) in the unfolded image with the pixel value at position p(i,j) to obtain the equidistant cylindrical unfolded image of the fisheye image.

[0096] Instance verification

[0097] (1) Selecting experimental data

[0098] This experiment used a self-designed engineering fisheye camera for data acquisition. The camera resolution was 2880*1620, the frame rate was 20 frames per second, the camera target size was 7.2mm*5.4mm, and the physical focal length after focusing was approximately 1.8mm. In this experiment, the camera was fixed to a vehicle suspension to capture video from the vehicle, and the video data was acquired using an RTSP bitrate. The video was then processed into frames for subsequent experiments.

[0099] (2) Parameter determination

[0100] During camera calibration in this experiment, an 8*12 checkerboard calibration board was selected, with each checkerboard square measuring 25mm*25mm. Nine different angles within the fisheye camera's field of view were selected, and 17 images of the calibration board were taken for camera calibration. The calibration reprojection error threshold was set to 0.5 pixels.

[0101] When performing equidistant cylindrical projection, it is necessary to set the resolution of the unfolded diagram, with the width W set to 3000 pixels and the height H set to 1500 pixels. When performing linear expansion, setting the expansion parameter n to 2.88 will achieve the best expansion effect.

[0102] (3) Experimental Results

[0103] The fisheye camera underwent checkerboard corner detection and calibration, achieving a reprojection error of 0.35 pixels. The calibration results are shown in Table 1. Fisheye images were captured using the calibrated fisheye camera, such as... Figure 4 As shown in (a), then an equidistant cylindrical projection is performed, and the unfolded diagram of the equidistant cylindrical surface after projection is shown in Figure 1. Figure 4 As shown in (b), a linear transformation is performed on the equidistant cylindrical surface development diagram, and the transformed equidistant cylindrical surface development diagram is as follows. Figure 5 As shown, the single-frame conversion time is 25ms / frame, which has good real-time performance.

[0104] Table 1 Camera calibration results

[0105]

[0106] Table 1

[0107] Although the illustrative specific embodiments of the present invention have been described above to enable those skilled in the art to understand the invention, it should be understood that the invention is not limited to the scope of the specific embodiments. For those skilled in the art, various changes are obvious as long as they are within the spirit and scope of the invention as defined and determined by the appended claims, and all inventions utilizing the concept of the present invention are protected.

Claims

1. A linear transformation based equidistant cylindrical fisheye image unwrapping method, characterized in that, Includes the following steps: (1) Perform parameter calibration on the fisheye camera; Multiple images of the calibration plate were captured from different angles using a fisheye camera to detect corner points, and the camera intrinsic parameters and distortion parameters were calibrated using Zhang Zhengyou's calibration method. (2) Construct a template image with all pixel values ​​of 0. The size of the template image is equal to the equidistant cylindrical unfolded diagram of the fisheye image. (3) Take a fisheye image using a calibrated fisheye camera, and then use the projection principle of equidistant cylindrical surfaces to unfold the fisheye image into equidistant cylindrical surfaces. (3.1) Take a fisheye image using a calibrated fisheye camera; (3.2) Establish a right-handed spatial coordinate system O-XYZ with the center O of the unit spherical model as the origin; (3.3) The polar coordinates of each pixel point p(i, j) in the template image in the spatial right-hand coordinate system O-XYZ are calculated by using the horizontal mapping method Wherein, W, H are width and length of the template image respectively, and satisfy W=2xH; is the angle between the line connecting the spatial coordinate point P(X, Y, Z) corresponding to the pixel point p(i, j) and the ball center O and the XOZ plane, and δ is the angle between the line connecting the projection point P'(X, 0, Z) of the spatial coordinate point P(X, Y, Z) on the XOZ plane and the ball center O and the positive half-axis of the Z axis. (3.4) Using the transformation relationship between polar coordinates and rectangular coordinates, obtain the polar coordinates. The world coordinates P(X,Y,Z) in the right-handed coordinate system O-XYZ; (3.5) Normalize the world coordinates P(X,Y,Z) to obtain the normalized world coordinates Q′(X′,Y′,Z′); X′=X / Z Y′=Y / Z Z' = Z / Z = 1 (3.6), using the normalized world coordinates Q'(X', Y', Z'), the distortion coordinates p(i, j) corresponding to the pixel point p(i, j) are calculated by the Kannala-Brandt model of the fisheye camera. d d )​ r d = θ + k1θ 3 + k2θ 5 + k3θ 7 + k4θ 9 i d = f x *X'*scale+c x j d = f y * Y'* scale + c y Among them, f x ,f y Let be the focal lengths of the fisheye camera on the X and Y axes, respectively; and r be the radial distance of the world coordinates Q′(X′,Y′,Z′) projected onto the image plane, satisfying the following conditions: scale is a scaling factor, and satisfies scale = r d / r;c x ,c y Here are the pixel coordinates of the center point of the captured fisheye image; the incident angle θ = atan(r); (3.7), traverse each distortion coordinate p(i d ,j d ), determine whether each coordinate value is an integer, if each coordinate point coordinate value i d ,j d is an integer, then directly use the pixel value at the position of the fisheye image distortion coordinate p(i d ,j d ) to replace the pixel value at the position of the p(i,j) in the unwrapping image, to obtain an initial equirectangular unwrapping image; otherwise, go to step (3.8); (3.8) bilinearly interpolating and resampling the non-integer distorted coordinates p(i d ,j d ) in, p(i) d ,j d The coordinate representation after bilinear interpolation resampling, (a,b) represents (i d ,j d The integer part of (di) d ,dj d ) represents (i d ,j d The fractional part of ) represents the image value of the fisheye image at the integer coordinates (a,b); Using distortion coordinates in fisheye images Replace the pixel value at position p(i,j) in the unfolded image with the pixel value at position p(i,j) to obtain an initial equidistant cylindrical unfolded image; (3.9) Perform a linear projection transformation on the initial equidistant cylindrical surface development; (3.9.1) Set the pixel information linear expansion variable n such that H′=nH, W′=nW, n>1; (3.9.2) Traverse each pixel point p(i,j) in the initial equidistant cylindrical unfolded diagram, and calculate the polar coordinates of p(i,j) in the right-handed coordinate system O-XYZ under the linear expansion of pixel information. (3.9.3), Order Given δ′=δ, find the correspondence between i′, j′ and i, j; (3.9.4) Replace the pixel value at p(i,j) in the initial equidistant cylindrical unfolded image with the pixel value at pixel point p(i,j) in the initial equidistant cylindrical unfolded image to obtain the equidistant cylindrical unfolded image of the fisheye image.

2. The method for unfolding equidistant cylindrical fisheye images based on linear transformation according to claim 1, characterized in that, The specific method for calibrating the fisheye camera parameters is as follows: (2.1) Detection of corner points on the chessboard; (2.1.1) Select a checkerboard calibration board with known size and spacing, place the checkerboard calibration board at different positions within the field of view of the fisheye camera, and then take multiple images of the calibration board from different angles. (2.1.2), calculating the horizontal gradient I of each pixel point in each calibration board image x and the vertical gradient I y ; wherein I (x,y) represents the pixel value of the pixel point at coordinate (x, y) in the calibration board image, x and y are the horizontal and vertical coordinates of the pixel point; (2.1.3), calculating the gradient matrix M of each pixel in each calibration board image in a 3x3 neighborhood j ; where M j represents the gradient matrix of the jth pixel point in the 3x3 neighborhood; (2.1.4), calculating the response value R of each pixel point in each calibration plate image j : R j = det(M j )-K(trace(M j ) 2 ) where det(M j ) = λ1λ2represents the determinant of the matrix M j , trace(M j ) = λ1+ λ2represents the trace of the matrix M j , λ1, λ2are eigenvalues of the matrix M j , and K is a weight used to balance the determinant and the trace. (2.1.5), traversing each pixel point in each calibration plate image, finding out the response value R j pixel points greater than the threshold value, and marking them as corner points; (2.2) Perform sub-pixel precision on the corner points of each calibration board image; Each corner point in each calibration board image is traversed, and a 1x3 neighborhood of pixels centered on the corner point is selected for quadratic interpolation to obtain the sub-pixel horizontal coordinate value x of the corner point sub ; Where x0 represents the x-coordinate value of the corner point in the calibration plate image, This represents the pixel values ​​of the two pixels to the left and right of the corner point. Represents the pixel value of a corner point; (2.3) Fisheye camera parameter calibration; (2.3.1), fix the world coordinate system on the checkerboard calibration board, and the world coordinates of the corner points on the checkerboard calibration board are marked as P(X w ,Y w ,Z w ), wherein X w ,Y w ,Z w represents the value of X, Y, Z three-axis, and Z w = 0. (2.3.2) converting the world coordinates P(X w ,Y w ,Z w ) of the corner point into camera coordinates P c (X c ,Y c ,Z c ) by a rotation matrix R and a translation vector T; (2.3.3) converting the point P in the camera coordinate system into a point p(u,v) in the pixel coordinate system according to the equirectangular projection model c (X c ,Y c ,Z c ) wherein (c x ,c y ) is the pixel coordinate of the center point of the calibration board image, f is the focal length of the fisheye camera, r is the radius of the projected point on the image plane of the calibration board, and θ is the incident angle. (2.3.4) using the corresponding distorted coordinates p(u, v) in the pixel coordinate system obtained by Kannala-Brandt's model d ,v d ); r d = θ + k1θ 3 + k2θ 5 + k3θ 7 + k4θ 9 wherein r d is the distorted radial distance, (u d ,v d ) is the distorted pixel coordinate, and k1-k4 are distortion coefficients. (2.3.5), collect the corner points on the multiple images of the calibration board, and record the pixel coordinates (u i ,v i ) and the distortion pixel coordinates (u d,i ,v d,i ) corresponding to each corner point; (2.3.6) The reprojection error E is minimized by nonlinear optimization method, thereby determining the intrinsic parameters of the fisheye camera: rotation matrix R and translation vector T, and distortion coefficients: k1~k4; Where N is the total number of corner points detected.

3. The method for unfolding equidistant cylindrical fisheye images based on linear transformation according to claim 1, characterized in that, The polar coordinates The range of values ​​for satisfies: The range is The range of values ​​for δ is (-π, π).

Images