A vehicle-mounted surround view splicing method and system based on deformable registration

Abstract

This invention relates to a method and system for vehicle-mounted surround view stitching based on deformable registration. The method includes: S1: acquiring non-planar views from four fisheye vehicle cameras, correcting the non-planar views to obtain planar views; S2: optimizing the planar views using minimum disparity tolerance depth to obtain optimized planar views, and using the ORB algorithm to obtain a global alignment transformation matrix H; S3: representing the optimized planar views with a mesh, using H to reposition the corresponding mesh vertices fitted to the optimized planar views to align matching points, and defining a constraint energy function to suppress mesh deformation, obtaining a mesh-registered planar view; S4: fusing the mesh-registered planar views using a linear weighting method and a gain compensation method to obtain a stitched four-channel fisheye vehicle-mounted surround view. The method provided by this invention can achieve panoramic surround view stitching from four fisheye cameras without relying on camera parameters, and effectively optimizes artifacts using image fusion methods.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and autonomous driving, specifically to an in-vehicle surround view stitching method and system based on deformable registration. Background Technology

[0002] Bird's-eye view panoramic images of vehicles provide omnidirectional perception of the surrounding environment, making it a crucial and challenging aspect of autonomous driving. Compared to high-cost LiDAR solutions, using inexpensive fisheye cameras offers a significant cost advantage. However, issues such as distortion, boundary irregularities, and computational costs that may arise when stitching non-planar views from fisheye cameras place higher demands on the performance of surround-view stitching algorithms.

[0003] By using fisheye cameras installed in four orthogonal viewing directions on the vehicle, each with a field of view greater than 180°, images with a large overlap area are simultaneously acquired. These images collectively cover the entire 360° viewing range of the surrounding environment. Using a surround view stitching algorithm, these images can be stitched together into a panoramic image in the form of a bird's-eye view.

[0004] Most current surround-view stitching methods use rigid image alignment based on geometric transformations, which requires hardware parameter information such as lens aperture, focal length, and assembly of the four fisheye cameras. However, this information is usually proprietary to the camera vendor or requires additional measurement procedures. These surround-view stitching methods, which rely on specific camera parameters, limited camera selection, and closed commercial software, restrict the development of general-purpose in-vehicle panoramic applications. Therefore, how to achieve surround-view stitching of four-channel fisheye images without relying on camera parameters, while mitigating inconsistencies caused by inaccurate pixel alignment and differences in camera illumination, has become an urgent problem to be solved. Summary of the Invention

[0005] To address the aforementioned technical problems, this invention provides a vehicle surround view stitching method and system based on deformable registration.

[0006] The technical solution of this invention is: a vehicle surround view stitching method based on deformable registration, comprising:

[0007] Step S1: Obtain a non-planar view using a four-channel fisheye vehicle camera, and correct the non-planar view to obtain a planar view;

[0008] Step S2: Optimize the planar view using the minimum disparity tolerance depth to obtain the optimized planar view, and use the ORB algorithm to obtain the global alignment transformation matrix H;

[0009] Step S3: Represent the optimized planar view using a mesh, reposition the corresponding mesh vertices fitted to the optimized planar view using H to align the matching points, and suppress mesh deformation by defining constraint energy functions, wherein the constraint energy functions include: feature alignment constraint energy function, local structure preservation constraint energy function, and global similarity constraint energy function; to obtain the mesh-registered planar view;

[0010] Step S4: The planar view after mesh registration is fused using a linear weighting method and a gain compensation method to obtain a spliced ​​four-way fisheye vehicle surround view.

[0011] Compared with the prior art, the present invention has the following advantages:

[0012] 1. Existing fisheye camera stitching methods require measuring a series of hardware parameters of four fisheye cameras, such as lens aperture, focal length, and assembly errors. However, these parameters are usually proprietary to the camera supplier or require additional closed commercial software for measurement and calibration. This limits the application of fisheye camera panoramic surround-view stitching in a wider range of scenarios, such as the availability of a limited number of camera models. This invention discloses a vehicle-mounted surround-view stitching method based on deformable registration, which can achieve panoramic surround-view stitching of four fisheye cameras without relying on camera parameters. This method is more suitable for developing applications for general scenarios, giving the technology a better prospect for widespread adoption.

[0013] 2. Due to the different installation positions of the four cameras, their optical centers are not aligned, resulting in parallax in the four images and causing artifacts in the merged area. Furthermore, since the four images are installed and processed independently, the different lighting conditions often lead to inconsistent photometric characteristics among the four images. Finally, corresponding points in the overlapping areas of images captured by adjacent cameras (such as the front and left cameras) may have varying degrees of fisheye distortion. This invention utilizes an image fusion method to effectively optimize artifacts, eliminate visual inconsistencies in the stitching results, and thus improve the visual effect. Attached Figure Description

[0014] Figure 1 This is a flowchart of a vehicle surround view stitching method based on deformable registration in an embodiment of the present invention;

[0015] Figure 2A This is a schematic diagram showing the position of point P in the original fisheye image in an embodiment of the present invention;

[0016] Figure 2B This is a schematic diagram illustrating the mapping of point P to S on a 3D unit sphere in an embodiment of the present invention;

[0017] Figure 2C This is a schematic diagram of projecting S onto a two-dimensional orthogonal coordinate system O in an embodiment of the present invention;

[0018] Figure 3A This is a schematic diagram of the minimum parallax tolerance depth d of the fisheye camera in an embodiment of the present invention;

[0019] Figure 3B This is a schematic diagram of the minimum visible depth d1 of the fisheye camera in an embodiment of the present invention;

[0020] Figure 4A This is a schematic diagram of images captured by front, left, rear, and right fisheye cameras in an embodiment of the present invention;

[0021] Figure 4B This is a schematic diagram of the stitched image in an embodiment of the present invention;

[0022] Figure 5 This is a structural block diagram of a vehicle surround view stitching system based on deformable registration, as described in an embodiment of the present invention. Detailed Implementation

[0023] This invention provides a vehicle-mounted surround view stitching method based on deformable registration, which can realize panoramic surround view stitching of four fisheye cameras without relying on camera parameters. Furthermore, it utilizes image fusion methods to effectively optimize artifacts and eliminate visual inconsistencies in the stitching results, thereby improving the visual effect.

[0024] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below through specific implementations and in conjunction with the accompanying drawings.

[0025] Example 1

[0026] like Figure 1 As shown in the figure, an embodiment of the present invention provides a vehicle surround view stitching method based on deformable registration, which includes the following steps:

[0027] Step S1: Obtain a non-planar view from a four-channel fisheye vehicle camera, correct the non-planar view, and obtain a planar view;

[0028] Step S2: Optimize the planar view using the minimum disparity tolerance depth to obtain the optimized planar view, and use the ORB algorithm to obtain the global alignment transformation matrix H;

[0029] Step S3: Represent the optimized planar view with a mesh, and use H to reposition the corresponding mesh vertices fitted to the optimized planar view to align the matching points. Then, define constraint energy functions to suppress mesh deformation. The constraint energy functions include: feature alignment constraint energy function, local structure preservation constraint energy function, and global similarity constraint energy function; to obtain the planar view after mesh registration.

[0030] Step S4: The planar views after mesh registration are fused using a linear weighting method and a gain compensation method to obtain a stitched four-way fisheye vehicle surround view.

[0031] Based on the optical imaging principle of fisheye cameras, the initially captured fisheye image is a non-planar view with significant distortion around its edges. Therefore, fisheye image correction must be performed before image stitching to eliminate this distortion. First, the center of the circular content region of the fisheye image needs to be determined, which typically coincides with the image's focal point. The circular content region can be extracted by calculating the smallest circumcircle with the center of the circle as the focal point, and then bounded by a fitted square to crop the original fisheye image.

[0032] In one embodiment, step S1 above: acquiring a non-planar view from a four-channel fisheye vehicle camera, correcting the non-planar view to obtain a planar view, specifically includes:

[0033] Step S11: Crop the non-planar view to an edge length of L, and use each image point P(x, y) of the non-planar view as spherical coordinates. Mapped onto a 3D unit sphere; where θ is the azimuth angle; It's the angle of elevation;

[0034] Figure 2A The image shows the position of point P in the original fisheye image;

[0035] Figure 2B As shown, point P is mapped to S on the 3D unit sphere;

[0036] However, in practice, due to design or manufacturing deviations, the centerline of a fisheye lens inevitably has a pitch angle ρ≠0 with the horizontal plane of the camera; fisheye image correction may be affected by ignoring this pitch angle ρ and cause severe distortion along the horizontal centerline of the image. Therefore, it is necessary to calculate the Cartesian coordinates of S and project them onto a 2D orthogonal coordinate system.

[0037] Step S12: Calculate the Cartesian coordinates S′(x′, y′, z′) of S, as shown in formulas (1) to (3):

[0038]

[0039]

[0040]

[0041] Where ρ is the pitch angle between the centerline of the fisheye lens and the horizontal plane of the camera, which can be calibrated by yourself or obtained from the camera supplier;

[0042] Step S13: Project point S′ onto a 2D orthogonal coordinate system using spherical projection O(u, v), as shown in formulas (4) to (5):

[0043]

[0044]

[0045] Where γ represents the field of view of each camera lens, and β = tan -1 (z′ / x′), where L is the length of the non-planar view.

[0046] like Figure 2C As shown, S is projected onto a two-dimensional orthogonal coordinate system O.

[0047] In practical applications, only one calibration of a given fisheye camera is needed to obtain the parameters for fisheye camera image correction. These pre-calculated parameters can then be reused for each image stitching from a lookup table (LUT), avoiding redundant calculations.

[0048] In one embodiment, step S2 above: optimizing the planar view using the minimum disparity tolerance depth to obtain the optimized planar view, and using the ORB algorithm to obtain the global alignment transformation matrix H, specifically includes:

[0049] Step S21: Optimize the planar view by increasing the minimum parallax tolerance depth d to obtain the optimized planar view, as shown in formula (6):

[0050]

[0051] Where r is the radius of the camera lens, γ represents the field of view of each fisheye camera lens, φ = (Y - π) / 2; O1 and O2 represent the lens vertex and the object position point, respectively, F represents the farthest point of view of the lens, and d1 represents the minimum visible depth of the camera.

[0052] The purpose of image stitching in this invention is to convert views with inter-lens parallax from fisheye cameras into a visually consistent stitched result. It's important to note that objects closer to the camera exhibit greater parallax between images captured by adjacent cameras. In such cases, it's impossible to correctly establish the corresponding features between the images to be matched, potentially leading to severe artifacts in the stitched image. Therefore, this step optimizes inter-lens parallax by increasing the minimum parallax tolerance depth d. Specifically, this involves cropping the fisheye-corrected image to increase the horizontal field of view, which increases the minimum visible depth and effectively reduces artifacts in the stitched image.

[0053] like Figure 3A and Figure 3BAs shown in the figure, the shaded area represents the camera's minimum visible depth d1. When an object is at this position, it cannot be seen by the camera. Therefore, the minimum parallax tolerance depth d( Figure 3A The gray shaded area needs to be greater than d1;

[0054] Step S22: Let I i and I j Represents two adjacent optimized planar views; I is calculated using the ORB algorithm. i and I j The matching points between them are then used to calculate the global alignment transformation matrix.

[0055] To optimize two adjacent planar views I i and I j To perform a match, it is necessary to identify I. i and I j The matching points between the pairs are then used for image alignment. This embodiment of the invention uses the ORB algorithm to calculate I. i and I j Given the k-th pair of matching points p = [x, y]T∈I i and p′=[x′,y′] T ∈I j The correspondence between homogeneous coordinates p and p′ represents the projective homography of a plane. The following equation can be estimated and solved using the DLT algorithm:

[0056]

[0057] Where N is I i and I j The number of matching pairs between points, let h be h = [h1h2…h9] T And |h|=1, while Defined as

[0058] Global alignment transformation matrix By Rewritten as a 3×3 matrix to obtain, using H to obtain image I i The pixel at position p is aligned to the image I. j The upper position is p′.

[0059] In one embodiment, step S3 above involves: representing the optimized planar view using a mesh; repositioning the corresponding mesh vertices fitted to the optimized planar view using H to align matching points; and suppressing mesh deformation by defining constraint energy functions, wherein the constraint energy functions include: feature alignment constraint energy function, local structure preservation constraint energy function, and global similarity constraint energy function; and obtaining the mesh-registered planar view, specifically including:

[0060] Step S31: Represent the optimized plan view using a mesh, and through mesh deformation, use H to adjust I. i and I j The corresponding grid vertices in the overlapping areas are repositioned to achieve grid alignment;

[0061] Define a constraint energy function to suppress mesh deformation, and define a constraint energy function for feature alignment, as shown in Equation (7):

[0062]

[0063] Among them, M ij From image I i and Image I j It consists of all k pairs of matching vertices between them; Represents image I i All adjacent images with overlapping regions; α ik and α jk The weighting factor is represented by the mesh vertices w after mesh deformation. ik or w jk In target image I i or I j The position within the mesh element before deformation is determined;

[0064] Solve the least squares solution of formula (7) to ensure that the distance between each paired matching point in the overlapping image region is as short as possible;

[0065] Step S32: Define the constraint energy function for local structure preservation to limit the degree of mesh deformation and prevent excessive mesh stretching, as shown in Equation (8):

[0066]

[0067] Among them, M ef Image I i The set of opposite edges of the mesh before and after deformation, where T is the similarity transformation matrix; E = [e i1 e i2 …e in ] represents image I i Before deformation, the mesh edge is F = [f i1 fi2 …f in ] respectively represent image I i The edge of the deformed mesh, n is the image I i The number of grid edges;

[0068] Step S33: Define the energy function for global similarity constraints, which is used to prevent the source image from tilting during stitching, as shown in formulas (9) to (10):

[0069]

[0070] F(T) = |T x -s i cos θ i | 2 +|T y -s i sin θ i | 2 (10)

[0071] Among them, S i θ is the scaling factor for each source image. i T is the rotation factor; x and T y These are the components of T along the x and y axes; α g (e k The weighting coefficients can be used to smooth the deformable mesh transformation between overlapping and non-overlapping regions of the undeformed image;

[0072] Because fisheye images are non-planar views, barrel distortion is unavoidable: the closer the imaged object is to the center of the lens, the more natural its structure appears; conversely, the farther the imaged object is from the center of the lens, the more severe the distortion. Therefore, embodiments of the present invention construct global similarity constraints to maintain the visual naturalness of panoramic surround-view stitched images.

[0073] This invention employs a grid-based method to guide image alignment: two adjacent fisheye-corrected images to be stitched are each fitted with a grid network. Each grid cell contains sub-image patches, and H is estimated only at the grid vertices. Then, H is used to align I. i and I j The corresponding grid vertices fitted in the model are repositioned to align with pre-identified matching points. Thus, I... i and I jThe meshes will align as the two corresponding meshes in their overlapping areas deform. Mesh-based methods may produce overstretched artifacts and introduce mesh distortion into the stitching result. To eliminate overstretching of the image structure and restore its naturalness, this invention constructs three constraint functions: a constraint energy function for feature alignment, a constraint energy function for local structure preservation, and a constraint energy function for global similarity, to suppress the uncertainty of mesh deformation.

[0074] Figure 4A This indicates images taken by the front, left, rear, and right fisheye cameras. Figure 4B This represents the stitched image.

[0075] In one embodiment, step S4 above, which involves fusing the mesh-registered planar view using a linear weighting method and a gain compensation method to obtain a stitched four-way fisheye vehicle surround view, specifically includes:

[0076] Step S41: The linear weighting method uses a ramp function to merge the overlapping areas of the two mesh-registered planar views. The merged pixel B(r, c) in row r and column c of the overlapping area of ​​size r×n is shown in formula (11):

[0077] B(r,c)=α1*L(r,c)+α2*R(r,c) (11)

[0078] Where L(r,c) and L(r,c) are the pixels taken from the left and right images at position (r,c), respectively;

[0079] When mixing the overlapping regions on the right, α1 = c / n, α2 = (n-c+1) / n, where n is the width of the mixing region;

[0080] When the overlapping regions on the left are mixed, α2 = c / n, α1 = (n-c+1) / n;

[0081] Step S42: Use a gain compensation method to reduce color or brightness differences between images captured by different camera lenses, wherein image I i with I j The compensation factor g between ij As shown in formula (12):

[0082]

[0083] Where, N ij Image I i With Image I j The number of pixels in the overlapping region; and Representing image I i With Image I jAverage pixel intensity of the overlapping region; σ g and σ N These represent the gain and standard deviation of the pixel intensity error, respectively.

[0084] To better stitch together fisheye camera images from non-planar scenes, this embodiment of the invention employs an image fusion method to reduce inconsistencies caused by inaccurate pixel alignment. A linear weighting method is used to fuse adjacent fisheye corrected images after grid registration, and a gain compensation method is used to compensate for the strong exposure differences between the four fisheye camera lenses.

[0085] This invention discloses a vehicle-mounted surround view stitching method based on deformable registration, which can realize panoramic surround view stitching of four fisheye cameras without relying on camera parameters. It is more suitable for developing applications in general scenarios, bringing better prospects for the promotion of this technology. At the same time, this invention can effectively optimize artifacts and eliminate visual inconsistencies in the stitching results by using image fusion methods, thereby improving the visual effect.

[0086] Example 2

[0087] like Figure 5 As shown, this embodiment of the invention provides a vehicle surround view stitching system based on deformable registration, comprising the following modules:

[0088] The planar view acquisition module 51 is used to acquire non-planar views from four fisheye vehicle cameras, correct the non-planar views, and obtain planar views.

[0089] The global alignment transformation matrix module 52 is used to optimize the planar view through the minimum disparity tolerance depth to obtain the optimized planar view, and to obtain the global alignment transformation matrix H using the ORB algorithm.

[0090] The mesh registration planar view module 53 is used to represent the optimized planar view with a mesh, reposition the corresponding mesh vertices fitted to the optimized planar view using H to align the matching points, and suppress mesh deformation by defining constraint energy functions, which include: feature alignment constraint energy function, local structure preservation constraint energy function, and global similarity constraint energy function; thus obtaining the mesh-registered planar view.

[0091] The planar view stitching module 54 is used to fuse the planar views after grid registration using a linear weighting method and a gain compensation method to obtain a stitched four-way fisheye vehicle surround view.

[0092] The above embodiments are provided merely for the purpose of describing the present invention and are not intended to limit the scope of the invention. The scope of the invention is defined by the appended claims. Various equivalent substitutions and modifications made without departing from the spirit and principles of the invention should be covered within the scope of the invention.

Claims

1. A vehicle-mounted surround view stitching method based on deformable registration, characterized in that, include: Step S1: Obtain a non-planar view using a four-channel fisheye vehicle camera, and correct the non-planar view to obtain a planar view; Step S2: Optimize the planar view using the minimum disparity tolerance depth to obtain the optimized planar view, and use the ORB algorithm to obtain the global alignment transformation matrix. Specifically, it includes: Step S21: For the planar view, increase the minimum parallax tolerance depth. After optimization, the optimized plan view is obtained, as shown in formula (6): (6) in, The radius of the camera lens. This represents the field of view of each fisheye camera lens. ; These represent the lens vertex and the object's position, respectively, with F representing the lens's farthest viewing point. Indicates the minimum visible depth of the camera; Step S22: Let and Represent two adjacent optimized planar views; calculated using the ORB algorithm. and The matching points between them are then used to calculate the global alignment transformation matrix. Step S3: Represent the optimized plan view using a grid, and utilize... The corresponding mesh vertices fitted to the optimized planar view are repositioned to align matching points, and constraint energy functions are defined to suppress mesh deformation. These constraint energy functions include: feature alignment constraint energy function, local structure preservation constraint energy function, and global similarity constraint energy function. The resulting planar view after mesh registration includes: Step S31: Represent the optimized planar view using a grid, and utilize the grid deformation to achieve the desired effect. right and The corresponding grid vertices in the overlapping areas are repositioned to achieve grid alignment; Define a constraint energy function to suppress mesh deformation, and define a constraint energy function for feature alignment, as shown in Equation (7): (7) in, From the image and images All between Composition of matched vertices; Representation and Image All adjacent images with overlapping regions; and Represents the weighting factor, determined by the vertices of the deformed mesh. or In the target image or The position within the mesh element before deformation is determined; Solve the least squares solution of formula (7) to ensure that the distance between each paired matching point in the overlapping image region is as short as possible; Step S32: Define the constraint energy function for preserving the local structure, as shown in Equation (8): (8) in, It is an image The set of opposite edges of the mesh before and after deformation, where T is the similarity transformation matrix; Representing an image The edge of the mesh before deformation, Representing images respectively The edge of the deformed mesh, n is the image. The number of grid edges; Step S33: Define the energy function for global similarity constraints, as shown in formulas (9) to (10): (9) (10) in, The scaling factor for each source image. The rotation factor; and These are the components of T on the x-axis and y-axis; These are weighting coefficients; Step S4: The planar view after mesh registration is fused using a linear weighting method and a gain compensation method to obtain a spliced ​​four-way fisheye vehicle surround view.

2. The vehicle surround view stitching method based on deformable registration according to claim 1, characterized in that, Step S1: Acquire a non-planar view using a four-channel fisheye vehicle camera, correct the non-planar view to obtain a planar view, specifically including: Step S11: Crop the non-planar view to an edge length of L, where each image point of the non-planar view... As spherical coordinates Mapped onto a 3D unit sphere; where, It is the azimuth angle; It's the angle of elevation; Step S12: Calculation Cartesian coordinates As shown in formulas (1) to (3): (1) (2) (3) in, The pitch angle between the centerline of the fisheye lens and the horizontal plane of the camera can be calibrated by yourself or obtained from the camera supplier; Step S13: Use spherical projection Point Projected onto a 2D orthogonal coordinate system, as shown in formulas (4) to (5): (4) (5) in, This represents the field of view for each camera lens. L is the length of the non-planar view.

3. The vehicle surround view stitching method based on deformable registration according to claim 2, characterized in that, Step S4: The mesh-registered planar view is fused using a linear weighting method and a gain compensation method to obtain a stitched four-way fisheye vehicle surround view, specifically including: Step S41: The linear weighting method uses a ramp function to fuse the overlapping regions of two images: in a region of size... The fused pixels in the r-row and c-column overlapping region As shown in formula (11): (11) in, and It is a location The pixels above are taken from the left and right images respectively; When the overlapping area on the right is mixed , , It is the width of the mixing region; When the overlapping area on the left is mixed, , ; Step S42: Use a gain compensation method to reduce color or brightness differences between images captured by different camera lenses, wherein the image and Compensation factors between As shown in formula (12): (12) in, Representing an image With images The number of pixels in the overlapping region; and Representing images respectively With images Average pixel intensity of the overlapping region; and These represent the gain and standard deviation of the pixel intensity error, respectively.

4. A vehicle-mounted surround view stitching system based on deformable registration, characterized in that, Includes the following modules: The planar view acquisition module is used to acquire non-planar views from four fisheye vehicle cameras, correct the non-planar views, and obtain a planar view. The module for obtaining the global alignment transformation matrix is ​​used to optimize the planar view using the minimum disparity tolerance depth to obtain the optimized planar view, and then uses the ORB algorithm to obtain the global alignment transformation matrix. Specifically, it includes: Step S21: For the planar view, increase the minimum parallax tolerance depth. After optimization, the optimized plan view is obtained, as shown in formula (6): (6) in, The radius of the camera lens. This represents the field of view of each fisheye camera lens. ; These represent the lens vertex and the object's position, respectively, with F representing the lens's farthest viewing point. Indicates the minimum visible depth of the camera; Step S22: Let and Represent two adjacent optimized planar views; calculated using the ORB algorithm. and The matching points between them are then used to calculate the global alignment transformation matrix. ; The grid registration planar view module is used to represent the optimized planar view using a grid, utilizing... The corresponding mesh vertices fitted to the optimized planar view are repositioned to align matching points, and constraint energy functions are defined to suppress mesh deformation. These constraint energy functions include: feature alignment constraint energy function, local structure preservation constraint energy function, and global similarity constraint energy function. The resulting planar view after mesh registration includes: Step S31: Represent the optimized planar view using a grid, and utilize the grid deformation to achieve the desired effect. right and The corresponding grid vertices in the overlapping areas are repositioned to achieve grid alignment; Define a constraint energy function to suppress mesh deformation, and define a constraint energy function for feature alignment, as shown in Equation (7): (7) in, From the image and images All between Composition of matched vertices; Representation and Image All adjacent images with overlapping regions; and Represents the weighting factor, determined by the vertices of the deformed mesh. or In the target image or The position within the mesh element before deformation is determined; Solve the least squares solution of formula (7) to ensure that the distance between each paired matching point in the overlapping image region is as short as possible; Step S32: Define the constraint energy function for preserving the local structure, as shown in Equation (8): (8) in, It is an image The set of opposite edges of the mesh before and after deformation, where T is the similarity transformation matrix; Representing an image The edge of the mesh before deformation, Representing images respectively The edge of the deformed mesh, n is the image. The number of grid edges; Step S33: Define the energy function for global similarity constraints, as shown in formulas (9) to (10): (9) (10) in, The scaling factor for each source image. The rotation factor; and These are the components of T on the x-axis and y-axis; These are weighting coefficients; The planar view stitching module is used to fuse the planar views after grid registration using a linear weighting method and a gain compensation method to obtain a stitched four-way fisheye vehicle surround view.

Images