Equidistant cylindrical fisheye image expansion method based on linear transformation

Abstract

The invention discloses a linear transformation-based isometric cylindrical fisheye image expansion method, which comprises the following steps of: acquiring internal parameters and distortion parameters of a fisheye camera through camera calibration, acquiring an isometric cylindrical expansion image of a fisheye image by utilizing a projection principle of an isometric cylindrical surface, and realizing linear transformation of the isometric cylindrical expansion image by changing corresponding parameters of a field angle. Therefore, the method is suitable for a fisheye lens with a large visual angle, distortion can be effectively removed, and the visual effect and practicability of the expanded image are remarkably improved.

Description

Technical Field

[0001] The invention belongs to the technical field of image processing, and more specifically, relates to an equidistant cylindrical fisheye image expansion method based on linear transformation. Background Art

[0002] Fisheye cameras are widely used in virtual reality, robot vision, autonomous driving, environmental monitoring and other fields due to their large field of view and panoramic imaging. However, fisheye images pose many challenges in subsequent image analysis and processing due to their extreme distortion. Fisheye image expansion is the process of converting images taken with a fisheye lens into images with a regular viewing angle. Therefore, how to efficiently and accurately expand fisheye images has become an important research topic in the field of image processing.

[0003] When expanding a fisheye image, it is necessary to resample the image pixels and transform the pixel coordinates to ensure that the shape of the objects in the expanded image is both intuitive and recognizable. However, geometric distortion and photometric distortion often occur in this process.

[0004] Traditional fisheye image expansion methods are mainly divided into geometric transformation method and physical modeling method. The geometric transformation method establishes a mathematical model between the fisheye image and the target projection, performs image mapping transformation, and thus removes the distortion in the fisheye image. Common methods include the fast expansion method and the 30-degree rotation expansion method. However, since such algorithms require a large number of interpolation calculations, they have a large computational overhead and poor real-time performance. The physical modeling method is based on the physical properties of the optical lens and constructs a projection model of the fisheye camera to expand the image. Common methods include the cylindrical expansion method and the linear projection method. Such methods have certain effects in solving some distortions, but for fisheye lenses with larger field angles, they still face more significant image distortion problems. Summary of the invention

[0005] The purpose of the present invention is to overcome the shortcomings of the prior art and provide an equidistant cylindrical fisheye image expansion method based on linear transformation. Based on the unit sphere projection model of the fisheye image, the equidistant cylindrical expansion image is generated by reversely projecting the fisheye image into the spherical space and combining the horizontal mapping method.

[0006] To achieve the above-mentioned object of the invention, the present invention provides an equidistant cylindrical fisheye image expansion method based on linear transformation, characterized in that it comprises the following steps:

[0007] (1) Calibrate the parameters of the fisheye camera;

[0008] Use a fisheye camera to take multiple calibration plate images at different viewing angles, detect corner points, and use Zhang Zhengyou calibration method to calibrate the camera intrinsic parameters and distortion parameters;

[0009] (2) Construct a template image with all pixel values ​​​​as 0, and the size of the template image is equal to the equidistant cylindrical expansion image of the fisheye image;

[0010] (3) Use the calibrated fisheye camera to take a fisheye image, and then expand the fisheye image equidistantly.

[0011] The object of the invention of the present invention is achieved in this way:

[0012] The invention discloses an equidistant cylindrical fisheye image expansion method based on linear transformation, which obtains the internal parameters and distortion parameters of a fisheye camera by camera calibration, obtains the equidistant cylindrical expansion diagram of the fisheye image by utilizing the projection principle of the equidistant cylinder, realizes the linear transformation of the equidistant cylindrical expansion diagram by changing the parameters corresponding to the field of view angle, and finally obtains the equidistant cylindrical expansion diagram of the fisheye image, so that the equidistant cylindrical expansion diagram of the fisheye image is suitable for fisheye lenses with larger viewing angles, can effectively remove distortion, and significantly improve the visual effect and practicality of the expanded image.

[0013] At the same time, the equidistant cylindrical fisheye image expansion method based on linear transformation of the present invention also has the following beneficial effects:

[0014] (1) By directly establishing the mapping relationship between the fisheye image and the unfolded image, the method has the advantage of low computational overhead. At the same time, the method is suitable for fisheye lenses with larger viewing angles, can effectively remove distortion, and significantly improve the visual effect and practicality of the unfolded image.

[0015] (2) The present invention obtains the internal parameters and distortion parameters of the fisheye camera through camera calibration, obtains the equidistant cylinder expansion diagram of the fisheye image by using the projection principle of the equidistant cylinder, and realizes the linear transformation of the pixel points of the expansion diagram by changing the corresponding parameters of the field of view angle, and can directly obtain the mapping relationship between the fisheye image and the expansion diagram. This method can not only remove the fisheye distortion and maintain the true proportion of the image structure, but also cover all the pixel information of the original fisheye image, avoiding data loss, and has good practicality.

[0016] (3) The present invention has low computational overhead. For a fisheye lens with a larger viewing angle, it can also remove distortion to the greatest extent while retaining all pixel information of the image, and avoid image distortion and aliasing, thereby achieving good visual effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a flow chart of the equidistant cylindrical fisheye image expansion method based on linear transformation of the present invention;

[0018] Figure 2 It is the principle diagram of equidistant cylinder development;

[0019] Figure 3 is the camera coordinate system mapping;

[0020] Figure 4 This is the effect diagram of the equidistant cylinder expansion;

[0021] Figure 5 This is the linear transformation effect diagram of the equidistant cylinder development diagram. DETAILED DESCRIPTION

[0022] The specific implementation of the present invention is described below in conjunction with the accompanying drawings so that those skilled in the art can better understand the present invention. It should be noted that in the following description, when the detailed description of known functions and designs may dilute the main content of the present invention, these descriptions will be omitted here.

[0023] Example

[0024] Figure 1 It is a flow chart of the equidistant cylindrical fisheye image expansion method based on linear transformation of the present invention.

[0025] In this embodiment, if Figure 1 As shown, the present invention provides an equidistant cylindrical fisheye image expansion method based on linear transformation, comprising the following steps:

[0026] S1. Calibrate the parameters of the fisheye camera;

[0027] Use a fisheye camera to take multiple calibration plate images at different viewing angles, then detect the corner points in each calibration plate image, and use Zhang Zhengyou calibration method to calibrate the camera intrinsic parameters and distortion parameters;

[0028] S1.1, checkerboard corner detection;

[0029] S1.1.1. Select a checkerboard calibration plate with known size and spacing, place the checkerboard calibration plate at different positions within the field of view of the fisheye camera to cover the wide-angle view of the fisheye lens as much as possible, and then take multiple images of the calibration plate at different viewing angles;

[0030] S1.1.2. For each calibration plate image, calculate the local gradient change of each pixel in the calibration plate image, and the horizontal gradient I of each pixel x and vertical gradient I y They are:

[0031]

[0032] Among them, I (x,y) Represents the pixel value of the pixel at the coordinate (x, y) in the calibration plate image, where x and y are the horizontal and vertical coordinates of the pixel;

[0033] S1.1.3. Calculate the gradient matrix M of each pixel in the 3×3 neighborhood of each calibration plate image. j ;

[0034]

[0035] Among them, M j Represents the gradient matrix of the j-th pixel in a 3×3 neighborhood;

[0036] S1.1.4. Calculate the response value R of each pixel in each calibration plate image j :

[0037] R j = det(M j )-K(trace(M j ) 2 )

[0038] Among them, det(M j )=λ1λ1 represents the matrix M j The determinant of trace(M j )=λ1+λ2 represents the matrix M j The trace of λ1,λ2 is the matrix M j The eigenvalue of , K is the weight used to balance the determinant and the trace;

[0039] In this embodiment, the parameter K is usually set between 0.04 and 0.06, and is used to balance the weights of the determinant and the trace.

[0040] S1.1.5. When the response value is positive and large, it indicates that the pixel has significant corner characteristics. Therefore, we can filter corners by setting a threshold, and then traverse each pixel in each calibration plate image to find the response value R. j Pixels larger than the threshold are marked as corner points. For the calibration of fisheye lenses, these corner points can be extracted from the fisheye field of view as calibration feature points, which can effectively extract the intersection points of the chessboard.

[0041] S1.2, perform sub-pixel precision on the corner points in each calibration plate image;

[0042] Traverse each corner point in each calibration plate image, select pixels in a 1×3 neighborhood with the corner point as the center, and perform secondary interpolation processing to obtain the sub-pixel horizontal coordinate value x of the corner point. sub ;

[0043]

[0044] Among them, x0 represents the horizontal coordinate value of the corner point in the calibration plate image, Represents the pixel values ​​of the two pixels on the left and right of the corner point, Represents the pixel value of the corner point;

[0045] After obtaining a more accurate corner point position through interpolation calculation, the original corner point position is updated to achieve sub-pixel accuracy.

[0046] S1.3, fisheye camera parameter calibration;

[0047] S1.3.1. Fix the world coordinate system on the checkerboard calibration plate. The world coordinates of the corner points on the checkerboard calibration plate are labeled P(X w ,Y w ,Z w ), where X w ,Y w ,Z w Indicates the values ​​of the X, Y, and Z axes. w =0;

[0048] S1.3.2, the world coordinates P(X w ,Y w ,Z w ) is converted into camera coordinates P c (X c ,Y c ,Z c );

[0049]

[0050] S1.3.3, according to the isometric projection model, point P in the camera coordinate system c (X c ,Y c ,Z c ) is converted to point p(u,v) in the pixel coordinate system;

[0051]

[0052] Among them, (c x ,c y ) is the pixel coordinate of the center point of the calibration plate image, f is the focal length of the fisheye camera, r is the radius of the projection point on the calibration plate image plane, and θ is the incident angle;

[0053] S1.3.4, use the Kannala-Brandt model to obtain the corresponding distortion coordinates p(u d ,v d );

[0054] r d =θ+k1θ 3 +k2θ 5 +k3θ 7 +k4θ 9

[0055]

[0056] Among them, r d is the radial distance after distortion, (u d ,v d ) are the distorted pixel coordinates, k1~k4 are the distortion coefficients;

[0057] S1.3.5. Collect the corner points on multiple calibration plate images and record the pixel coordinates (u i ,v i ) and distorted pixel coordinates (u d,i ,v d,i );

[0058] S1.3.6. By using a nonlinear optimization method to minimize the reprojection error E, the internal parameter matrix of the fisheye camera is determined: the rotation matrix R and the translation vector T, as well as the distortion coefficients: k1~k4;

[0059]

[0060] Where N is the total number of corner points detected;

[0061] In this embodiment, the calibration accuracy is evaluated using the mean of the reprojection error to ensure that the model can accurately describe the imaging characteristics of the fisheye camera.

[0062] S2. Construct a template image with a width of W, a length of H, and a pixel value of all 0. The pixel coordinates of each pixel in the template image are represented by p(i, j), where W = 2×H;

[0063] In this embodiment, the equidistant cylindrical expansion image is horizontally expanded from the unit sphere, so the horizontal field of view angle is (-π, π) and the vertical field of view angle is Therefore, we first construct a template image of the same size as the equidistant cylindrical expansion image, whose width and height satisfy W = 2H;

[0064] S3, equidistant cylindrical expansion of fisheye image;

[0065] S3.1. Use the calibrated fisheye camera to take a fisheye image, such as Figure 4 (a)

[0066] S3.2. Take the center O of the unit sphere model as the coordinate origin and establish a spatial right-handed coordinate system O-XYZ;

[0067] S3.3. Use the horizontal mapping method to find the polar coordinates of each pixel point p(i,j) in the right-handed coordinate system O-XYZ in space

[0068]

[0069] in, is the angle between the line connecting the spatial coordinate point P(X,Y,Z) corresponding to the pixel point p(i,j) and the sphere center O and the XOZ plane, and δ is the angle between the line connecting the projection point P′(X,0,Z) of the spatial coordinate point P(X,Y,Z) on the XOZ plane and the sphere center O and the positive half axis of the Z axis;

[0070] In this embodiment, the equidistant cylinder expansion algorithm designed by the present invention is essentially based on the unit sphere projection model of the fisheye image and uses the horizontal mapping method to construct the reverse mapping between the equidistant cylinder expansion image and the fisheye image.

[0071] The unit sphere model of a fisheye camera is as follows Figure 2 (a) shows that its equidistant cylindrical expansion diagram is as follows Figure 2 As shown in (b), on the unit sphere model, l1 is the line connecting the spatial coordinate point P(X,Y,Z) and the sphere center O, and l2 is the line connecting the projection point P′(X,0,Z) of the spatial coordinate point P(X,Y,Z) on the XOZ plane and the sphere center O. Assume is the angle between l1 and the XOZ plane, δ is the angle between l2 and the positive half axis of the Z axis, then for any point p(i,j), the angle can be obtained by using the horizontal mapping method. and angle δ, since the equidistant cylinder development diagram is horizontally mapped from the unit sphere, the angle The range is The range of δ is (-π,π).

[0072] S3.4. Use the conversion relationship between polar coordinates and rectangular coordinates to obtain polar coordinates The world coordinates P(X,Y,Z) in the right-handed coordinate system O-XYZ in space;

[0073]

[0074] S3.5. Normalize the world coordinates P(X, Y, Z) to obtain the normalized world coordinates Q′(X′, Y′, Z′). The normalized coordinates can be regarded as the coordinates in the camera coordinate system, such as Figure 3 As shown;

[0075] X′=X / Z

[0076] Y′=Y / Z

[0077] Z'=Z / Z=1

[0078] S3.6, using the normalized world coordinates Q′(X′, Y′, Z′), the Kannala-Brandt model of the fisheye camera is used to calculate the distorted coordinates p(i, j) corresponding to the pixel p(i, j). d ,jd );

[0079] r d =θ+k1θ 3 +k2θ 5 +k3θ 7 +k4θ 9

[0080] i d =f x *X'*scale+c x

[0081] j d =f y *Y'*scale+c y

[0082] Among them, f x ,f y are the focal lengths of the fisheye camera on the X and Y axes respectively; r is the radial distance of the projection of the world coordinate Q′(X′, Y′, Z′) on the image plane, and satisfies scale is the scaling factor, and scale = r d / r;c x ,c y is the pixel coordinate of the center point of the captured fisheye image; the incident angle θ = atan (r);

[0083] S3.7, traverse each distortion coordinate p(i d ,j d ), determine whether each coordinate value is an integer. If the coordinate value of each coordinate point i d ,j d If all are integers, the pixel value at the position p(id, jd) of the distorted coordinate in the fisheye image is directly used to replace the pixel value at the position p(i, j) in the expanded image; otherwise, proceed to step S3.8;

[0084] S3.8. For non-integer distortion coordinates p(i d ,j d ) performs bilinear interpolation resampling;

[0085]

[0086] in, It means p(i d ,j d ) is the coordinate representation after bilinear interpolation resampling, (a,b) represents (i d ,j d ), (di d ,dj d ) means (i d,j d ), I(a,b) represents the image value of the fisheye image at the integer coordinates (a,b);

[0087] Using the distortion coordinates in fisheye images The pixel value at position p(i,j) in the expanded image is replaced by the pixel value at position p(i,j) to obtain an initial equidistant cylindrical expanded image. The bilinear interpolation method can realize smooth transition of pixels in the expanded image, avoid image distortion and jagged phenomenon, and have a good visual effect.

[0088] S3.9, perform a projective linear transformation on the initial equidistant cylindrical development diagram;

[0089] In this embodiment, the initial equidistant cylindrical surface development diagram is as follows: Figure 4 As shown in (b), a large number of black shadows are not filled, which is not conducive to visual observation. If the image is expanded directly, large-area pixel accumulation will occur, which cannot achieve the ideal visual effect. Therefore, the present invention proposes an expansion method based on linear transformation to achieve uniform expansion of pixel information of the expanded image. The specific process is as follows:

[0090] S3.9.1. Set the pixel information linear expansion variable n so that H′=nH, W′=nW, n>1. In this embodiment, n is 2.88;

[0091] S3.9.2. Traverse each pixel point p(i,j) in the initial equidistant cylindrical expansion image, and calculate the polar coordinates of p(i,j) in the right-handed coordinate system O-XYZ in space under the linear expansion of pixel information

[0092]

[0093] S3.9.3, Order δ′=δ, find the corresponding relationship between i′, j′ and i, j;

[0094]

[0095] S3.9.4. Using the Distorted Coordinates in Fisheye Images The pixel value at position p(i,j) in the expanded image is replaced by the pixel value at position p(i,j) to obtain the equidistant cylindrical expanded image of the fisheye image.

[0096] Example verification

[0097] (1) Select experimental data

[0098] This experiment uses a self-designed engineering fisheye camera for data collection. The camera has a resolution of 2880*1620, a frame rate of 20 frames per second, a camera target size of 7.2mm*5.4mm, and a physical focal length of about 1.8mm after the camera is focused. In this experiment, the camera is fixed by the vehicle suspension, and the vehicle video is captured. The video data is collected using the rtsp code stream. The video is framed for subsequent experiments.

[0099] (2) Parameter determination

[0100] In this experiment, an 8*12 chessboard calibrated board was selected, with each chessboard grid size of 25mm*25mm. Nine different angles in the fisheye camera field of view were selected, and 17 calibration board images were taken for camera calibration. The calibration reprojection error threshold was set to 0.5 pixels.

[0101] When performing equidistant cylindrical projection, you need to set the expanded image resolution, set the width W to 3000 pixels, and the height H to 1500 pixels. When performing linear expansion, set the expansion parameter n to 2.88 to achieve the best expansion effect.

[0102] (3) Experimental results

[0103] The fisheye camera is subjected to chessboard corner detection and camera calibration, and the calibration reprojection error reaches 0.35 pixels. The calibration results are shown in Table 1. The fisheye image is taken by the calibrated fisheye camera, as shown in Figure 4 (a) is shown, and then an equidistant cylindrical projection is performed. The equidistant cylindrical expansion diagram after projection is shown as Figure 4 As shown in (b), the equidistant cylindrical expansion diagram is linearly transformed, and the transformed equidistant cylindrical expansion diagram is as follows Figure 5 As shown, the single frame conversion time is 25ms / frame, which has good real-time performance.

[0104] Table 1 Camera calibration results

[0105]

[0106] Table 1

[0107] Although the above describes the illustrative specific embodiments of the present invention to facilitate those skilled in the art to understand the present invention, it should be clear that the present invention is not limited to the scope of the specific embodiments. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations using the concept of the present invention are protected.

Claims

1. A method for expanding an equidistant cylindrical fisheye image based on linear transformation, characterized in that: The following steps are involved: (1) Calibrate the parameters of the fisheye camera; Use a fisheye camera to take multiple calibration plate images at different viewing angles, detect corner points, and use Zhang Zhengyou calibration method to calibrate the camera intrinsic parameters and distortion parameters; (2) Construct a template image with all pixel values ​​​​as 0, and the size of the template image is equal to the equidistant cylindrical expansion image of the fisheye image; (3) Use the calibrated fisheye camera to take a fisheye image, and then use the equidistant cylinder projection principle to expand the fisheye image equidistant cylinder.

2. The linear transformation-based equidistant cylindrical fisheye image expansion method according to claim 1, characterized in that: The specific method for calibrating the fisheye camera parameters is as follows: (2.1), checkerboard corner detection; (2.1.1) Select a checkerboard calibration plate with known size and spacing, place the checkerboard calibration plate at different positions within the field of view of the fisheye camera, and then take multiple images of the calibration plate at different viewing angles; (2.1.2) Calculate the horizontal gradient I of each pixel in each calibration plate image x and vertical gradient I y ; Among them, I (x,y) Represents the pixel value of the pixel at the coordinate (x, y) in the calibration plate image, where x and y are the horizontal and vertical coordinates of the pixel; (2.1.3) Calculate the gradient matrix M of each pixel in the 3×3 neighborhood of each calibration plate image j ; Among them, M j Represents the gradient matrix of the j-th pixel in a 3×3 neighborhood; (2.1.4) Calculate the response value R of each pixel in each calibration plate image j : R j =det(M j )-K(trace(M j ) 2 ) Among them, det(M j )=λ1λ1 represents the matrix M j The determinant of trace(M j )=λ1+λ2 represents the matrix M j The trace of λ1,λ2 is the matrix M j The eigenvalue of , K is the weight used to balance the determinant and the trace; (2.1.5) Traverse each pixel in each calibration plate image and find the response value R j Pixels larger than the threshold are marked as corner points; (2.2) Perform sub-pixel precision on the corner points in each calibration plate image; Traverse each corner point in each calibration plate image, select pixels in a 1×3 neighborhood with the corner point as the center, and perform secondary interpolation processing to obtain the sub-pixel horizontal coordinate value x of the corner point. sub ; Among them, x0 represents the horizontal coordinate value of the corner point in the calibration plate image, Represents the pixel values ​​of the two pixels on the left and right of the corner point, The pixel value representing the corner point; (2.3) Fisheye camera parameter calibration; (2.3.1) Fix the world coordinate system on the checkerboard calibration plate, and the world coordinates of the corner points on the checkerboard calibration plate are marked as P(X w ,Y w ,Z w ), where X w ,Y w ,Z w Indicates the values ​​of the X, Y, and Z axes. w =0; (2.3.2) The world coordinates P(X w ,Y w ,Z w ) is converted into camera coordinates P c (X c ,Y c ,Z c ); (2.3.3) According to the isometric projection model, point P in the camera coordinate system c (X c ,Y c ,Z c ) is converted to point p(u,v) in the pixel coordinate system; Among them, (c x ,c y ) is the pixel coordinate of the center point of the calibration plate image, f is the focal length of the fisheye camera, r is the radius of the projection point on the calibration plate image plane, and θ is the incident angle; (2.3.4) Use the Kannala-Brandt model to obtain the corresponding distortion coordinates p(u d ,v d ); r d =θ+k1θ 3 +k2θ 5 +k3θ 7 +k4θ 9 Among them, r d is the radial distance after distortion, (u d ,v d ) are the distorted pixel coordinates, k1~k4 are the distortion coefficients; (2.3.5) Collect the corner points on multiple calibration plate images and record the pixel coordinates corresponding to each corner point (u i ,v i ) and distorted pixel coordinates (u d,i ,v d,i ); (2.3.6) By solving the minimization of the reprojection error E through nonlinear optimization method, the internal parameter matrix of the fisheye camera is determined: the rotation matrix R and the translation vector T, as well as the distortion coefficients: k1~k4; Where N is the total number of detected corner points.

3. The linear transformation-based equidistant cylindrical fisheye image expansion method according to claim 1, characterized in that: The method for equidistant cylindrical expansion of the fisheye image is as follows: (3.1) Use the calibrated fisheye camera to take a fisheye image; (3.2) Taking the center O of the unit sphere model as the origin, establish a spatial right-handed coordinate system O-XYZ; (3.3) Use the horizontal mapping method to find the polar coordinates of each pixel p(i,j) in the template image in the right-handed coordinate system O-XYZ Where W and H are the width and length of the template image, respectively, and W = 2 × H; is the angle between the line connecting the spatial coordinate point P(X,Y,Z) corresponding to the pixel point p(i,j) and the sphere center O and the XOZ plane, and δ is the angle between the line connecting the projection point P′(X,0,Z) of the spatial coordinate point P(X,Y,Z) on the XOZ plane and the sphere center O and the positive half axis of the Z axis; (3.4) Using the conversion relationship between polar coordinates and rectangular coordinates, we can obtain the polar coordinates. The world coordinates P(X,Y,Z) in the right-handed coordinate system O-XYZ in space; (3.5) Normalize the world coordinates P(X, Y, Z) to obtain the normalized world coordinates Q′(X′, Y′, Z′); X′=X / Z Y′=Y / Z Z'=Z / Z=1 (3.6), using the normalized world coordinates Q′(X′, Y′, Z′), the distorted coordinates p(i) corresponding to the pixel p(i, j) are obtained through the Kannala-Brandt model of the fisheye camera. d ,j d ); r d =θ+k1θ 3 +k2θ 5 +k3θ 7 +k4θ 9 i d =f x *X'*scale+c x j d =f y *Y'*scale+c y Among them, f x ,f y are the focal lengths of the fisheye camera on the X and Y axes respectively; r is the radial distance of the projection of the world coordinate Q′(X′, Y′, Z′) on the image plane, and satisfies scale is the scaling factor, and scale = r d / r;c x ,c y is the pixel coordinate of the center point of the captured fisheye image; the incident angle θ = atan (r); (3.7), traverse each distortion coordinate p(i d ,j d ), determine whether each coordinate value is an integer. If the coordinate value of each coordinate point i d ,j d If they are all integers, the pixel value at the position p(id,jd) of the distorted coordinate in the fisheye image is directly used to replace the pixel value at the position p(i,j) in the expanded image to obtain an initial equidistant cylindrical expanded image; otherwise, proceed to step (3.8); (3.8), for non-integer distortion coordinates p(i d ,j d ) performs bilinear interpolation resampling; in, It means p(i d ,j d ) is the coordinate representation after bilinear interpolation resampling, (a,b) represents (i d ,j d ), (di d ,dj d ) means (i d ,j d ), I(a,b) represents the image value of the fisheye image at the integer coordinates (a,b); Using the distortion coordinates in fisheye images The pixel value at position p(i,j) in the expanded image is replaced by the pixel value at position p(i,j) to obtain an initial equidistant cylindrical expanded image; (3.9) Perform a projective linear transformation on the initial equidistant cylindrical expansion diagram; (3.9.1) Set the pixel information linear expansion variable n so that H′=nH, W′=nW, n>1; (3.9.2) Traverse each pixel point p(i,j) in the initial equidistant cylindrical expansion image, and calculate the polar coordinates of p(i,j) in the right-handed coordinate system O-XYZ in the space under the linear expansion of pixel information (3.9.3), δ′=δ, find the corresponding relationship between i′, j′ and i, j; (3.9.4) Use the pixel value at pixel point p(i,j) in the initial equidistant cylindrical expansion image to replace the pixel value at p(i',j') in the initial equidistant cylindrical expansion image to obtain the equidistant cylindrical expansion image of the fisheye image.

4. The linear transformation-based equidistant cylindrical fisheye image expansion method according to claim 3, characterized in that: The polar coordinates The value range of satisfies: The range is The value range of δ is (-π,π).

Images