A fish-eye multi-camera calibration and GPU single-pass fusion rendering method based on 3D bowl model

Abstract

The application relates to a fish-eye multi-camera calibration method based on a 3D bowl model and a GPU single-pass fusion method, and the method comprises the following steps: S1, a 3D bowl model grid model is constructed, a bowl-shaped 3D grid model with a vehicle center as an origin; S2, vertex coordinate transformation, for each vertex P of the bowl model local , the following coordinate transformations are sequentially performed; S3, Kannala-Brandt fish-eye projection calculation, for each vertex, KB fish-eye projection is performed in the coordinate system of each camera; S4, camera coordinate system fusion weight calculation, for each vertex, the fusion weight of the camera is calculated in the coordinate system of each camera. Compared with the traditional 2D AVM, the distortion of distant objects is significantly reduced, and the division by zero problem at the projection center is avoided through threshold judgment, so that the calibration numerical stability is ensured.

Description

Technical Field

[0001] This invention relates to the field of vehicle-mounted holographic technology, and in particular to a method for fisheye multi-camera calibration and GPU single-pass fusion based on a 3D bowl model. Background Technology

[0002] Traditional in-vehicle AVM systems employ the inverse perspective transformation (IPM) method. This involves correcting distortion in images from four fisheye cameras, projecting them onto a 2D ground plane using a homography matrix, and then performing alpha blending and stitching on the overlapping areas to generate a bird's-eye view. A typical solution, such as the AVM system on the TI TDAx platform, uses a geometrically aligned LUT (Lookup Table) and a photometrically aligned LUT to handle geometric mapping and color correction, respectively. Projection and blending are performed on the CPU / GPU for each frame using a lookup table.

[0003] For 3D surround view, Socionext's 360° Wrap-Around View System projects panoramic images onto a bowl-shaped 3D mesh based on OpenGLES, providing both 2D top-down and 3D surround view modes. This solution uses a LUT to pre-compute the projection mapping table, and performs texture sampling at runtime by looking up the table in the OpenGLES rendering pipeline.

[0004] 2D planar projection assumes the ground is flat, which cannot handle the correct projection of 3D structures such as columns and walls, resulting in severe distortion of distant objects. Although 3D bowl-shaped solutions (such as Socionext) solve the planar assumption problem, their fusion weights are still calculated in the vehicle coordinate system. When there is a pitch angle installation deviation of the camera, the azimuth angle in the vehicle coordinate system does not correspond to the actual field of view of the camera, resulting in weight misalignment at the fusion seam.

[0005] Traditional fusion methods render each camera feed separately to a texture and then overlay it using alpha blending, requiring multiple rendering passes and GPU state switching. On automotive embedded platforms (such as the Qualcomm SA8295P), the GPU's computing power is limited, and the overhead of multiple passes limits the frame rate, making it difficult to achieve 60fps real-time rendering on high-resolution models.

[0006] Existing solutions perform projection transformation and weight calculation frame by frame during runtime, with CPU / GPU overhead being positively correlated with the bowl model resolution. When the number of bowl model vertices increases (e.g., to the order of 100,000) to improve surround view accuracy, the real-time performance of frame-by-frame calculation becomes difficult to guarantee.

[0007] Some AVM schemes employ feature point matching methods for image stitching: feature points (such as ORB and SIFT) are extracted in the overlapping areas of adjacent cameras, and the homography matrix is ​​calculated through feature matching to achieve image alignment and stitching. Fusion strategies employ methods such as Laplacian pyramid fusion and fade-in / fade-out weighted fusion to eliminate stitching seams.

[0008] Feature point matching methods are not robust to scenes lacking texture (such as white ground or sparse parking lot markings), and matching failures can lead to misalignment in the stitching.

[0009] Laplace pyramid fusion requires multi-scale decomposition and reconstruction, which incurs high computational overhead and is not suitable for real-time processing on embedded platforms.

[0010] Feature point matching methods are difficult to directly extend to 3D bowl model projection because the projection relationship on the bowl model is determined by the geometric model and does not require estimation of transformation at the image level. Summary of the Invention

[0011] In view of the shortcomings of the prior art, the present invention provides a method for fisheye multi-camera calibration and GPU single-pass fusion based on 3D bowl model. Compared with traditional 2D AVM, it not only significantly reduces the distortion of distant objects, but also avoids the division by zero problem at the projection center by threshold judgment, thus ensuring the stability of calibration values.

[0012] To achieve the above and other related objectives, the present invention provides the following technical solution: A method for fisheye multi-camera calibration and GPU single-pass fusion rendering based on a 3D bowl model, the method comprising: S1. Construct a 3D bowl-shaped mesh model, with the center of the vehicle as the origin; S2. Vertex coordinate transformation, for each vertex P of the bowl mold local Perform the following coordinate transformations in sequence; S3. Kannala-Brandt fisheye projection calculation: Perform KB fisheye projection for each vertex in the coordinate system of each camera; S4. Camera coordinate system fusion weight calculation: For each vertex, calculate the camera fusion weight in each camera coordinate system; S5 GPU single-pass fusion rendering uses a custom shader in the rendering stage to complete the sampling and weighted fusion of four camera textures through a single rendering pass.

[0013] Furthermore, in step S1, the bowl-shaped 3D mesh model is a dome structure with an open top, and the vertices are distributed on curved surfaces at different heights from the ground, covering the ground around the vehicle and the near-vertical area. The number of vertices in the bowl model is in the hundreds of thousands.

[0014] Furthermore, in step S2, the process of processing each vertex P of the bowl mold... local The following coordinate transformations are performed sequentially: S21. Local coordinate system → World coordinate system P world =MeshTransform.TransformPoint(P local ); S22. Subtract the root node offset and the rear axis offset. P shifted =P world -rootOffset-vehicleOffset; S23. Unity Coordinate System → Vehicle RFU Coordinate System P vehicle =(P shifted .x, P shifted .z,P shifted .y), Where vehicleOffset is the offset from the vehicle center to the rear axle, and MeshTransform is the value of the form. TransformPoint is the coordinate transformation function, P local Let P be the coordinates of the vertex of the bowl mold. world Let P be the coordinates of each vertex of the bowl model in the world coordinate system, rootOffset be the offset compensation amount from the vehicle center to the rear axle, and P be the offset compensation amount from the vehicle center to the rear axle. shifted To transform the coordinates, P shifted .x is the x-coordinate, P shifted .y is the ordinate, P shifted .z is the oblique coordinate, P vehicle These are the RFU coordinates of the vehicle.

[0015] Furthermore, in step S3, the Kannala-Brandt fisheye projection calculation includes: S31. First, transform the vehicle RFU coordinates to the camera OpenCV coordinate system using the extrinsic parameter matrix, then perform projection, and perform camera coordinate transformation P. cam , P cam =extrinsic matrix ×P vehicle , Among them, extrinsic matrix Let P be the extrinsic parameter matrix. vehicle The RFU coordinates of the vehicle; S32. Calculate the incident angle θ. r=sqrt(Xc 2 +Yc 2 ), θ = atan2(r, Zc), Where (X,Y,Z) is P cam The coordinates of P, where c is the coordinate of P. cam The distance to the origin, where r is the radius; S33.KB Distortion Model θ d , θ d =θ×(1+k1·θ 2 +k2·θ 4 +k3·θ 6 +k4·θ 8 ), Where k1, k2, k3, and k4 are distortion coefficients; S34. Scaling factor (scale) scale=θ d / r, when r>ε, ε=1e-8 scale=1, when r ≤ ε, the center point of the optical axis; S35. Project to pixel coordinates (u img v img ), u img =fx×scale×Xc+cx, v img =fy×scale×Yc+cy, Where (x,y) are the projection coordinates, and fx and fy are the focal length parameters; S36. Normalize to [0,1] and store in the UV channel (u norm v norm ), u norm =u img / width, v norm =v img / height, Where width is the width and height is the height.

[0016] Furthermore, the Kannala-Brandt fisheye projection calculation also includes: S37. Validity judgment, invalid conditions: Zc≤0, the vertex is behind the camera, or θ>π / 2, the incident angle is too large.

[0017] Furthermore, in step S4, calculating the camera fusion weights in each camera coordinate system includes: S41. Transform the vertices to the camera coordinate system (Xc, Yc, Zc); S42. After the judgment, if Zc≤0, the weight of the camera is 0; S43. Calculate the horizontal deflection angle δ, δ=atan2(Xc,Zc); S44. Asymmetric FOV selection: If δ≥0, halfFov=fovRight If δ < 0, halfFov = fovLeft. Among them, fovRight and fovLeft are independently set parameters; S45. Calculation of cosine attenuation weight w If d = |δ|, and d ≤ halfFov, w=bias×cos((d / halfFov)×π / 2), Otherwise, w=0.

[0018] Furthermore, the calculation of camera fusion weights in each camera coordinate system also includes normalizing the numerical value w. i w i =w i / (w front +w back +w left + w right ), where w front w back w left and w right These are the weights for the front, back, left, and right sides of the camera, respectively.

[0019] Furthermore, in step S5, the GPU single-pass fusion rendering includes: S51. Texture Packaging: Four fisheye camera images are packaged into two DisplayTextures; S52. Vertex Shader: Passes the four-way projected coordinates and vertex color RGBA to the fragment shader; S53. Fragment Shader: For each camera i, i=0-3, According to UV i The Y component is sampled in the upper half (Y < 0.5) or lower half (Y ≥ 0.5). A Y offset correction is applied: upperOffset = YFlip × 0.5. The corresponding DisplayTexture is then sampled to obtain color C. i If UV i C is outside the range [0,1]. i =InvalidColor; S54. Weighted fusion: Color=R×C front +G×C back +B×C left+A×C right , R, G, B, and A are the normalized weights in the vertex colors.

[0020] Furthermore, the two DisplayTextures include: DisplayTexture0: The upper half stores the data from the front camera, and the lower half stores the data from the rear camera. DisplayTexture1: The upper half stores the left camera image, and the lower half stores the right camera image.

[0021] Furthermore, the 3D mesh model is created using 3D modeling tools or generated through parametric surfaces.

[0022] The present invention has the following positive effects: 1. The 3D bowl model of this invention eliminates the limitations of planar projection: the vertices of the bowl model are distributed on a 3D curved surface, with near vertices close to the ground and far vertices raised, naturally restoring the perspective relationship of near objects appearing larger and far objects smaller, without the planar assumption of inverse perspective transformation (IPM). Compared with traditional 2D AVM, the distortion of distant objects is significantly reduced.

[0023] 2. The camera coordinate system weighting of this invention eliminates pitch error: The weighting is calculated in the camera coordinate system using atan2(Xc, Zc) to calculate the horizontal deflection angle, which is naturally aligned with the camera's optical axis. When there is a pitch angle installation deviation in the camera, the horizontal deflection angle in the camera coordinate system still correctly reflects the actual field of view of the camera; while the azimuth angle of the traditional vehicle coordinate system will produce a systematic deviation, resulting in weight misalignment at the fusion seam. This invention eliminates this source of error in principle.

[0024] 3. The asymmetric FOV adaptation installation deviation of the present invention: It supports independent setting of the left and right half field of view of the same camera (fovLeft ≠ fovRight), adapting to the actual scene of the camera being installed with the orientation offset to the side. Traditional methods usually assume a symmetrical FOV.

[0025] 4. The pre-computation + single-pass implementation of this invention achieves zero intra-frame overhead: Projected UVs and blending weights are pre-computed and encoded into the mesh vertex attributes during the calibration phase. At runtime, the GPU only needs to sample 4-way textures and perform weighted summation in a single pass, without performing projection transformations frame by frame or multiple rendering blending operations. CPU load and frame rate are completely decoupled, and a 100,000-vertex bowl model can run stably at 60fps on an embedded platform.

[0026] 5. Robust handling of optical axis singularities in this invention: By using threshold judgment (r > ε), the division-to-zero problem at the projection center is avoided, ensuring the stability of the calibration values. Attached Figure Description

[0027] Figure 1This is a flowchart illustrating the overall calibration and rendering process of this invention. Figure 2 This is the single-vertex projection calculation process of the present invention; Figure 3 This is the GPU single-pass fusion rendering pipeline of the present invention. Detailed Implementation

[0028] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0029] Example: Figure 1 or Figure 2 As shown, a method for fisheye multi-camera calibration and GPU single-pass fusion rendering based on a 3D bowl model is described, the method comprising: S1. Construct a 3D bowl-shaped mesh model, with the center of the vehicle as the origin; S2. Vertex coordinate transformation, for each vertex P of the bowl mold local Perform the following coordinate transformations in sequence; S3. Kannala-Brandt fisheye projection calculation: Perform KB fisheye projection for each vertex in the coordinate system of each camera; S4. Camera coordinate system fusion weight calculation: For each vertex, calculate the camera fusion weight in each camera coordinate system; S5 GPU single-pass fusion rendering uses a custom shader in the rendering stage to complete the sampling and weighted fusion of four camera textures through a single rendering pass.

[0030] In this embodiment, in step S1, the bowl-shaped 3D mesh model is a dome structure with an open top, and the vertices are distributed on curved surfaces at different heights from the ground, covering the ground around the vehicle and the near-vertical area. The number of vertices of the bowl model is in the hundreds of thousands.

[0031] In this embodiment, in step S2, the process of processing each vertex P of the bowl mold... local The following coordinate transformations are performed sequentially: S21. Local coordinate system → World coordinate system P world =MeshTransform.TransformPoint(P local ); S22. Subtract the root node offset and the rear axis offset. P shifted =P world -rootOffset-vehicleOffset; S23. Unity Coordinate System → Vehicle RFU Coordinate System P vehicle =(P shifted .x, P shifted .z,P shifted .y), Where vehicleOffset is the offset from the vehicle center to the rear axle, and MeshTransform is the value of the form. TransformPoint is the coordinate transformation function, P local Let P be the coordinates of the vertex of the bowl mold. world Let P be the coordinates of each vertex of the bowl model in the world coordinate system, rootOffset be the offset compensation amount from the vehicle center to the rear axle, and P be the offset compensation amount from the vehicle center to the rear axle. shifted To transform the coordinates, P shifted .x is the x-coordinate, P shifted .y is the ordinate, P shifted .z is the oblique coordinate, P vehicle These are the RFU coordinates of the vehicle.

[0032] In this embodiment, step S3, the Kannala-Brandt fisheye projection calculation includes: S31. First, transform the vehicle RFU coordinates to the camera OpenCV coordinate system using the extrinsic parameter matrix, then perform projection, and perform camera coordinate transformation P. cam , P cam =extrinsic matrix ×P vehicle , Among them, extrinsic matrix Let P be the extrinsic parameter matrix. vehicle The RFU coordinates of the vehicle; S32. Calculate the incident angle θ. r=sqrt(Xc 2 +Yc 2 ), θ = atan2(r, Zc), Where (X,Y,Z) is P cam The coordinates of P, where c is the coordinate of P. cam The distance to the origin, where r is the radius; S33.KB Distortion Model θ d , θ d =θ×(1+k1·θ 2+k2·θ 4 +k3·θ 6 +k4·θ 8 ), Where k1, k2, k3, and k4 are distortion coefficients; S34. Scaling factor (scale) scale=θ d / r, when r>ε, ε=1e-8 scale=1, when r ≤ ε, the center point of the optical axis; S35. Project to pixel coordinates (u img v img ), u img =fx×scale×Xc+cx, v img =fy×scale×Yc+cy, Where (x,y) are the projection coordinates, and fx and fy are the focal length parameters; S36. Normalize to [0,1] and store in the UV channel (u norm u norm ), u norm =u img / width, v norm =v img / height, Where width is the width and height is the height.

[0033] In this implementation, the Kannala-Brandt fisheye projection calculation also includes: S37. Validity judgment, invalid conditions: Zc≤0, the vertex is behind the camera, or θ>π / 2, the incident angle is too large.

[0034] In this embodiment, step S4, calculating the camera fusion weights in each camera coordinate system, includes: S41. Transform the vertices to the camera coordinate system (Xc, Yc, Zc); S42. After the judgment, if Zc≤0, the weight of the camera is 0; S43. Calculate the horizontal deflection angle δ, δ=atan2(Xc,Zc); S44. Asymmetric FOV selection: If δ≥0, halfFov=fovRight If δ < 0, halfFov = fovLeft. Among them, fovRight and fovLeft are independently set parameters; S45. Calculation of cosine attenuation weight w If d = |δ|, and d ≤ halfFov, w=bias×cos((d / halfFov)×π / 2), Otherwise, w=0.

[0035] In this embodiment, calculating the camera fusion weights in each camera coordinate system further includes normalizing the value w. i w i =w i / (w front +w back +w left + w right ), where w front w back w left and w right These are the weights for the front, back, left, and right sides of the camera, respectively.

[0036] In this embodiment, as Figure 3 As shown, in step S5, the GPU single-pass fusion rendering includes: S51. Texture Packaging: Four fisheye camera images are packaged into two DisplayTextures; S52. Vertex Shader: Passes the four-way projected coordinates and vertex color RGBA to the fragment shader; S53. Fragment Shader: For each camera i, i=0-3, According to UV i The Y component is sampled in the upper half (Y < 0.5) or lower half (Y ≥ 0.5). A Y offset correction is applied: upperOffset = YFlip × 0.5. The corresponding DisplayTexture is then sampled to obtain color C. i If UV i C is outside the range [0,1]. i =InvalidColor; S54. Weighted fusion: Color=R×C front +G×C back +B×C left +A×C right , R, G, B, and A are the normalized weights in the vertex colors.

[0037] In this embodiment, the two DisplayTextures include: DisplayTexture0: The upper half stores the data from the front camera, and the lower half stores the data from the rear camera. DisplayTexture1: The upper half stores the left camera image, and the lower half stores the right camera image.

[0038] In this embodiment, the 3D mesh model is created using 3D modeling tools or generated through parametric surfaces.

[0039] The present invention provides a fisheye multi-camera calibration and GPU single-pass fusion rendering system based on a 3D bowl model, including a computer device that is programmed or configured to perform the steps of the fisheye multi-camera calibration and GPU single-pass fusion rendering method based on a 3D bowl model.

[0040] The present invention also provides a computer-readable storage medium storing a computer program programmed or configured to perform the described fisheye multi-camera calibration and GPU single-pass fusion rendering method based on a 3D bowl model.

[0041] Any references to memory, storage, database, or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0042] In summary, compared with traditional 2D AVM, this invention not only significantly reduces the distortion of distant objects, but also avoids the division-by-zero problem at the projection center by using threshold judgment, thus ensuring the stability of calibration values.

[0043] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A method for fisheye multi-camera calibration and GPU single-pass fusion rendering based on a 3D bowl model, characterized in that, The method includes: S1. Construct a 3D bowl-shaped mesh model, with the center of the vehicle as the origin; S2. Vertex coordinate transformation, for each vertex P of the bowl mold local Perform the following coordinate transformations in sequence; S3. Kannala-Brandt fisheye projection calculation: Perform KB fisheye projection for each vertex in the coordinate system of each camera; S4. Camera coordinate system fusion weight calculation: For each vertex, calculate the camera fusion weight in each camera coordinate system; S5 GPU single-pass fusion rendering uses a custom shader in the rendering stage to complete the sampling and weighted fusion of four camera textures through a single rendering pass.

2. The fisheye multi-camera calibration and GPU single-pass fusion rendering method based on a 3D bowl model according to claim 1, characterized in that, In step S1, the bowl-shaped 3D mesh model is a dome structure with an open top. The vertices are distributed on curved surfaces at different heights from the ground, covering the ground around the vehicle and the nearby facade area. The number of vertices in the bowl model is in the hundreds of thousands.

3. The fisheye multi-camera calibration and GPU single-pass fusion rendering method based on a 3D bowl model according to claim 1, characterized in that, In step S2, the process of processing each vertex P of the bowl mold... local The following coordinate transformations are performed sequentially: S21. Local coordinate system → World coordinate system P world =MeshTransform.TransformPoint(P local ); S22. Subtract the root node offset and the rear axis offset. P shifted =P world -rootOffset-vehicleOffset; S23. Unity Coordinate System → Vehicle RFU Coordinate System P vehicle =(P shifted .x, P shifted .z,P shifted .y), Where vehicleOffset is the offset from the vehicle center to the rear axle, and MeshTransform is the value of the form. TransformPoint is the coordinate transformation function, P local Let P be the coordinates of the vertex of the bowl mold. world Let P be the coordinates of each vertex of the bowl model in the world coordinate system, rootOffset be the offset compensation amount from the vehicle center to the rear axle, and P be the offset compensation amount from the vehicle center to the rear axle. shifted To transform the coordinates, P shifted .x is the x-coordinate, P shifted .y is the ordinate, P shifted .z is the oblique coordinate, P vehicle These are the RFU coordinates of the vehicle.

4. The fisheye multi-camera calibration and GPU single-pass fusion rendering method based on a 3D bowl model according to claim 3, characterized in that, In step S3, the Kannala-Brandt fisheye projection calculation includes: S31. First, transform the vehicle RFU coordinates to the camera OpenCV coordinate system using the extrinsic parameter matrix, then perform projection, and perform camera coordinate transformation P. cam , P cam =extrinsic matrix ×P vehicle , Among them, extrinsic matrix Let P be the extrinsic parameter matrix. vehicle The RFU coordinates of the vehicle; S32. Calculate the incident angle θ. r=sqrt(Xc 2 +Yc 2 ), θ = atan2(r, Zc), Where (X,Y,Z) is P cam The coordinates of P, where c is the coordinate of P. cam The distance to the origin, where r is the radius; S33.KB Distortion Model θ d , i d =θ×(1+k1·θ 2 +k2·θ 4 +k3·θ 6 +k4·θ 8 ), Where k1, k2, k3, and k4 are distortion coefficients; S34. Scaling factor (scale) scale=θ d / r, when r>ε,ε=1e-8, scale=1, when r ≤ ε, the center point of the optical axis; S35. Project to pixel coordinates (u img v img ), u img =fx×scale×Xc+cx, v img =fy×scale×Yc+cy, Where (x,y) are the projection coordinates, and fx and fy are the focal length parameters; S36. Normalize to [0,1] and store in the UV channel (u norm v norm ), u norm =u img / width, v norm =v img / height, Where width is the width and height is the height.

5. The fisheye multi-camera calibration and GPU single-pass fusion rendering method based on a 3D bowl model according to claim 4, characterized in that, The Kannala-Brandt fisheye projection calculation also includes: S37. Validity judgment, invalid conditions: Zc≤0, the vertex is behind the camera, or θ>π / 2, the incident angle is too large.

6. The fisheye multi-camera calibration and GPU single-pass fusion rendering method based on a 3D bowl model according to claim 1, characterized in that, In step S4, calculating the camera fusion weights in each camera coordinate system includes: S41. Transform the vertices to the camera coordinate system (Xc, Yc, Zc); S42. After the judgment, if Zc≤0, the weight of the camera is 0; S43. Calculate the horizontal deflection angle δ, δ=atan2(Xc,Zc); S44. Asymmetric FOV selection: If δ≥0, halfFov=fovRight If δ < 0, halfFov = fovLeft. Among them, fovRight and fovLeft are independently set parameters; S45. Calculation of cosine attenuation weight w If d = |δ|, and d ≤ halfFov, w=bias×cos((d / halfFov)×π / 2), Otherwise, w=0.

7. The fisheye multi-camera calibration and GPU single-pass fusion rendering method based on a 3D bowl model according to claim 6, characterized in that, The calculation of camera fusion weights in each camera coordinate system also includes normalizing the value w. i w i =w i / (w front +w back +w left + w right ), where w front w back w left and w right These are the weights for the front, back, left, and right sides of the camera, respectively.

8. The fisheye multi-camera calibration and GPU single-pass fusion rendering method based on a 3D bowl model according to claim 1, characterized in that, In step S5, the GPU single-pass fusion rendering includes: S51. Texture Packaging: Four fisheye camera images are packaged into two DisplayTextures; S52. Vertex Shader: Passes the four-way projected coordinates and vertex color RGBA to the fragment shader; S53. Fragment Shader: For each camera i, i=0-3, According to UV i The Y component is sampled in the upper half (Y < 0.5) or lower half (Y ≥ 0.5). A Y offset correction is applied: upperOffset = YFlip × 0.

5. The corresponding DisplayTexture is then sampled to obtain color C. i If UV i C is outside the range [0,1]. i =InvalidColor; S54. Weighted fusion: Color=R×C front +G×C back +B×C left +A×C right , R, G, B, and A are the normalized weights in the vertex colors.

9. The fisheye multi-camera calibration and GPU single-pass fusion rendering method based on a 3D bowl model according to claim 8, characterized in that: The two DisplayTextures include: DisplayTexture0: the upper half stores the front camera image, and the lower half stores the rear camera image. DisplayTexture1: The upper half stores the left camera image, and the lower half stores the right camera image.

10. The fisheye multi-camera calibration and GPU single-pass fusion rendering method based on a 3D bowl model according to claim 1, characterized in that: The 3D mesh model is created using 3D modeling tools or generated through parametric surfaces.

Images