Image calibration and splicing method based on sub-pixel reconstruction and elliptic coefficient weighting

Abstract

According to the image calibration and splicing method based on sub-pixel reconstruction and elliptic coefficient weighting, a super-resolution reconstruction module is introduced before image array splicing, the splicing precision is improved through sub-pixel-level image registration, and the splicing precision better than that of a single pixel unit can be achieved. Aiming at the problems of edge dislocation and stiff transition existing in multi-camera array system view field splicing, the invention provides a weighted splicing algorithm based on an elliptic coefficient, and the algorithm constructs an elliptic gradient weight model, realizes smooth transition of an image boundary region, and remarkably improves the overall visual effect of a panoramic image.

Description

Technical Field

[0001] This invention relates to the calibration, correction, and stitching technology of image arrays captured by camera arrays, and in particular to an image calibration and stitching method based on sub-pixel reconstruction and elliptic coefficient weighting. Background Technology

[0002] Traditional single-camera-based imaging systems, limited by hardware constraints and the laws of optical physics, suffer from an irreconcilable contradiction between the field of view and resolution of the generated images, leading to a bottleneck in image quality improvement. Camera array systems, however, effectively overcome this limitation through a multi-camera collaborative imaging architecture combined with advanced image processing algorithms. This system employs multiple distributed sub-cameras to simultaneously acquire scene information. Utilizing precise image calibration and stitching techniques, it not only achieves panoramic coverage with a large field of view but also significantly improves the spatial resolution of the images, resulting in a qualitative leap in imaging performance. The key to achieving a perfect fusion of high resolution and a large field of view lies in overcoming the technical bottlenecks of traditional image calibration and stitching.

[0003] As a core technological support for multi-camera collaborative imaging systems, image calibration and stitching algorithms directly determine the quality and accuracy of the final image. This technology can handle problems such as field-of-view distortion and color differences that occur when multiple camera systems acquire images, accurately stitching together local images captured by each sub-camera into a complete large-field-of-view image. Camera calibration, as a crucial step to ensure seamless image stitching, can be further subdivided into two key steps: calibration and correction. The calibration process precisely measures the internal geometric characteristics of each camera, including intrinsic parameters such as focal length and principal point coordinates, as well as extrinsic parameters such as spatial pose, providing basic data for subsequent processing. The correction process performs geometric correction and color difference processing on the image based on the calibration results. After a rigorous calibration process, the images from each sub-camera achieve a high degree of uniformity in geometry and color, laying the foundation for subsequent high-precision image stitching. Currently, the mainstream image stitching methods are mainly divided into direct averaging fusion, fixed-weighted fusion, and variable-weighted fusion.

[0004] However, existing image calibration and stitching algorithms still face many technical bottlenecks. Traditional processing methods are limited by pixel-level registration accuracy, with a stitching error as small as one pixel, making it difficult to meet the needs of high-precision application scenarios. In terms of stitching overlapping areas, fixed-weight fusion strategies cannot adapt to complex and ever-changing actual shooting environments, and are prone to producing obvious stitching marks. Summary of the Invention

[0005] This invention proposes an image calibration and stitching method based on sub-pixel reconstruction and elliptic coefficient weighting. The implementation process of the method is shown in the attached figure. Figure 1As shown. Compared to traditional techniques, this method introduces a super-resolution reconstruction module before image array stitching, improving stitching accuracy through sub-pixel-level image registration, achieving stitching accuracy superior to that of a single pixel. Addressing the edge misalignment and abrupt transitions in multi-camera array system field-of-view stitching, this method proposes a weighted stitching algorithm based on elliptic coefficients. This algorithm constructs an elliptical gradient weight model, achieving smooth transitions in image boundary regions and significantly improving the overall visual effect of panoramic images.

[0006] The method includes the following steps:

[0007] Step 1: Use a calibration board to accurately calibrate each sub-camera in the N rows and M columns camera array;

[0008] Step 2: Use a camera array to capture images of the target scene from multiple perspectives with overlapping fields of view;

[0009] Step 3: Preprocess the acquired images based on the calibration parameters, including geometric distortion correction, color calibration, and super-resolution reconstruction. The flowchart is attached. Figure 2 As shown;

[0010] Step 4: Seamlessly stitch the image array using the proposed weighted stitching algorithm based on elliptic coefficients.

[0011] Further, step 1 includes the following sub-steps:

[0012] Sub-step 1.1: Each sub-camera separately acquires several chessboard images in different poses, ensuring that all areas of the camera's field of view are covered, including the center, edges, and tilt direction, and that the calibration board is fully visible in each image;

[0013] Sub-step 1.2: Based on the images acquired in sub-step 1.1, calculate the intrinsic parameter matrix for each sub-camera;

[0014] Sub-step 1.3: For each row of cameras, calibrate the adjacent sub-cameras from left to right: In the common field of view of each group of adjacent sub-cameras, acquire multiple sets of chessboard images with different poses. The chessboard images must appear in both cameras at the same time to ensure that the calibration board pose covers the overlapping area of ​​the field of view of the adjacent cameras, and calculate the horizontal relative pose of each sub-camera.

[0015] Sub-step 1.4: For each column of cameras, calibrate the adjacent sub-cameras from top to bottom: In the common field of view of each group of adjacent sub-cameras, acquire multiple sets of chessboard images with different poses. The chessboard images must appear in both cameras at the same time to ensure that the pose of the calibration board covers the overlapping area of ​​the field of view of the adjacent cameras, and calculate the vertical relative pose of each sub-camera.

[0016] Sub-step 1.5: Select the central sub-camera as the global reference, whose extrinsic parameters are the identity matrix and the zero vector. Starting from the central sub-camera, calculate the global extrinsic parameters of other sub-cameras in the same row through chain transformation.

[0017] Sub-step 1.6: Starting from each sub-camera in the center row, calculate the global extrinsic parameters of other sub-cameras in the same column through chain transformation.

[0018] Furthermore, step 3 includes the following sub-steps:

[0019] Sub-step 3.1: Eliminate lens distortion in each image using calibration parameters;

[0020] Sub-step 3.2: Employ a multi-level progressive histogram matching algorithm to align the color distribution of adjacent images to calibrate the colors;

[0021] Sub-step 3.3: Use a deep learning super-resolution model to perform super-resolution reconstruction of the image to achieve sub-pixel level detail reconstruction.

[0022] Furthermore, step 4 includes the following sub-steps:

[0023] Sub-step 4.1: Define the images acquired by two adjacent sub-cameras as adjacent image pairs. A schematic diagram of the N x M column image array after processing in step 3 is attached. Figure 3 As shown, each box represents a preprocessed single image to be stitched. The image array is stitched row by row: first, the images are grouped in pairs and stitched using a weighted stitching algorithm based on elliptic coefficients. Then, the stitched results are grouped in pairs again and the stitching process is repeated until each row contains only one seamless wide-format image, ultimately forming an N-row, 1-column image array. The flowchart for sub-step 4.1 is attached. Figure 4 As shown in the attached diagram, the splicing process is illustrated. Figure 5 As shown;

[0024] Sub-step 4.2: Perform vertical stitching on the N-row wide-format image processed in sub-step 4.1: First, group adjacent rows of images pairwise and stitch them together using a weighted stitching algorithm based on elliptic coefficients; then, continue grouping the stitched results pairwise and repeating the stitching process to finally generate a complete image with both a large field of view and high resolution. A schematic diagram of the stitching process in sub-step 4.2 is attached. Figure 6 As shown.

[0025] Furthermore, in sub-step 4.1, for an N-row, M-column image array, each row of images is stitched together row by row. The stitching process for a single row of images includes the following sub-steps:

[0026] Sub-step 4.1.1: Assume each row of the image array has m images to be stitched (m≥2). In the first round of stitching, let m = M, where M is the initial number of images in each row. The image grouping rule depends on the current value of m. After each round of stitching, m needs to be updated. The update rule is explained in sub-step 4.1.5. When m is even, group the images in pairs, forming K = m / 2 pairs of adjacent images from left to right. When m is odd, take the first (m-1) images and group them in pairs, forming K = (m-1) / 2 pairs of adjacent images from left to right. The remaining image will directly participate in the next round of grouping.

[0027] Sub-step 4.1.2: Determine the stitching region for each of the K adjacent image pairs. Let the width of the image to be stitched be X pixels and the height be Y pixels. X and Y need to be updated after each stitching round. Establish a Cartesian coordinate system with the top-left pixel of each image as the origin, the horizontal direction from left to right as the positive x-axis, and the vertical direction from top to bottom as the positive y-axis. Based on the proportion of the overlapping field of view of adjacent cameras, the width of the overlapping field of view between the K adjacent image pairs can be determined as Δx pixels. Then, for the left image in a pair of adjacent images, the coordinate range of the stitching region can be represented as [X-Δx, X-1]; while for the right image, the coordinate range of the stitching region can be represented as [0, Δx-1]. Let the left edge x-coordinate of the stitching region of the left image be a1 = X-Δx and the right edge x-coordinate be b1 = X-1 in this stitching round, and the left edge x-coordinate of the stitching region of the right image be a2 = 0 and the right edge x-coordinate be b2 = Δx-1.

[0028] Sub-step 4.1.3: Based on the characteristics of elliptic curves, construct a weight function that varies with the spatial position of pixels, as shown in equations (1) and (2). Here, the weight coefficients p1(x,y) and p2(x,y) are the pixel weight values ​​of two adjacent images in the (x,y) coordinates, respectively. (See attached...) Figure 7 As shown, by shifting p2(x,y) to the right along the x-axis, the splicing regions of the two weight coefficient distribution curves completely overlap. It can be observed that the image pixel superposition weight values ​​exhibit a smooth elliptical curve transition characteristic along the splicing direction.

[0029]

[0030] Sub-step 4.1.4: Based on the weight values ​​of each pixel calculated in sub-step 4.1.3, K adjacent image pairs are stitched together according to equation (3). The stitching pattern is reflected in the distribution of weight coefficients: overlapping areas are fused according to the weight ratio, while non-overlapping areas directly retain the original image data. Where I fused_k (x,y) represents the pixel value at (x,y) of the image formed by stitching together the k-th pair of adjacent images. k(x,y) and I k '(x,y) represent the pixel values ​​of the two images in the k-th adjacent image pair at the (x,y) coordinates, respectively. k ′(x-a1,y) represents the second image I k A rightward translation of (a1,0) is applied. The purpose of this operation is to spatially align the overlapping regions of the two images, thereby ensuring that the weight functions p1(x,y) and p2(x-a1,y) can be applied correctly to the corresponding overlapping pixels of the two images.

[0031] I fused_k (x,y)=p1(x,y)·I k (x,y)+p2(x-a1,y)·I k ′(x-a1,y) (3)

[0032] Sub-step 4.1.5: If the current m is 2, then the horizontal stitching of the image in this row is completed; if m is not 2 and m is even in this round, then let m = K; if m is not 2 and m is odd in this round, then let m = K + 1, that is, add the last image that did not participate in the grouping in sub-step 4.1.1, and then return to sub-step 4.1.1 to continue stitching.

[0033] Furthermore, in sub-step 4.1, if all images have been stitched together row by row to form an image array of N rows and 1 column, then sub-step 4.2 will be executed next.

[0034] Further, in sub-step 4.2, the pixel values ​​of the overlapping areas of vertically adjacent image pairs are stitched together in a weighted superposition manner, including the following sub-steps:

[0035] Sub-step 4.2.1: Suppose there are n images to be stitched in the current column (n≥2). In the first round of stitching, let n=N, where N is the initial column number. The image grouping rule depends on the current value of n. After each round of stitching, n needs to be updated. The update rule is explained in sub-step 4.1.5. When n is even, the images are grouped in pairs, with each pair of adjacent images from top to bottom forming L=n / 2 pairs of adjacent images. When n is odd, the first (n-1) images are grouped in pairs, with each pair of adjacent images from top to bottom forming L=(n-1) / 2 pairs of adjacent images. The remaining image will directly participate in the next round of grouping.

[0036] Sub-step 4.2.2: Stitch together the L groups of adjacent image pairs formed by grouping. The width of the overlapping area between the L groups of adjacent image pairs can be determined as Δy pixels based on the percentage of overlap between the fields of view of adjacent cameras. For the upper image in a pair of adjacent images, the ordinate range of the area to be stitched is [Y-Δy, Y-1]; while for the lower image, the coordinate range of the area to be stitched is [0, Δy-1]. Let the ordinate of the upper edge of the stitching area of ​​the upper image be c1 = Y-Δy, and the ordinate of the lower edge be d1 = Y-1; and the ordinate of the upper edge of the stitching area of ​​the lower image be c2 = 0, and the ordinate of the lower edge be d2 = Δy-1.

[0037] Sub-step 4.2.3: Based on the characteristics of elliptic curves, construct a weight function that varies with the spatial position of pixels, as shown in equations (4) and (5). Here, the weight coefficients q1(x,y) and q2(x,y) are the pixel weight values ​​of the two images in the l-th adjacent image pair at (x,y) coordinates, respectively, and the weight values ​​exhibit a smooth elliptic curve transition characteristic along the splicing direction;

[0038]

[0039] Sub-step 4.2.4: Based on sub-step 4.2.3, calculate the weight value of each pixel, and perform stitching on L groups of adjacent image pairs according to equation (6). The stitching rule is reflected in the distribution of weight coefficients: overlapping areas are fused according to the weight ratio, while non-overlapping areas directly retain the original image data. Where I fused_l (x,y) represents the pixel value of the image formed by stitching together the l-th pair of adjacent images in the (x,y) coordinate system. l (x,y) and I l '(x,y) represent the pixel values ​​of the two images in the l-th adjacent image pair at the (x,y) coordinates; l ′(x,y-c1) represents the second image I l A downward translation of (c1,0) is applied. The purpose of this operation is to spatially align the overlapping regions of the two images, thereby ensuring that the weight functions q1(x,y) and q2(x,y-c1) can be applied correctly to the corresponding overlapping region pixels of the two images.

[0040] I fused_l (x,y)=q1(x,y)·I l (x,y)+q2(x,y-c1)·I l ′(x,y-c1) (6)

[0041] Sub-step 4.2.5: If the current n is not 2 and n is even, then let n = L; if the current n is not 2 and n is odd, then let n = L + 1, that is, add the last image that did not participate in the grouping in sub-step 4.2.1, and then return to sub-step 4.2.1 to continue the vertical image stitching. If the current n is 2, then the image array stitching has been completed, and the complete stitched image has been successfully obtained.

[0042] This invention proposes an image calibration and stitching method based on sub-pixel reconstruction and elliptic coefficient weighting, which overcomes the accuracy limitations of traditional image stitching techniques. The super-resolution reconstruction process improves image registration accuracy to the sub-pixel level, enhancing the ability of camera array imaging systems to reproduce details in large field-of-view scenes. Meanwhile, the weighting algorithm based on elliptic curve characteristics ensures a natural transition at the stitching boundary through its unique mathematical continuity, effectively eliminating common problems such as seams and brightness jumps in traditional methods. This method not only achieves a balance between a large field of view and high resolution but also provides a reliable solution for applications with stringent image quality requirements, such as industrial inspection. Attached Figure Description

[0043] Figure 1 The flowchart shows the proposed image calibration and stitching method based on sub-pixel reconstruction and elliptic coefficient weighting.

[0044] Figure 2 A flowchart for image preprocessing.

[0045] Figure 3 This is a schematic diagram of an N×M image array to be stitched together.

[0046] Figure 4 This is a flowchart for stitching together a row of images in an N×M image array.

[0047] Figure 5 This is a schematic diagram of stitching together each row of an N×M image array.

[0048] Figure 6 This is a schematic diagram of stitching N rows of images together to form a complete image.

[0049] Figure 7 This is a schematic diagram of the weight distribution of two images during the stitching process of horizontally adjacent image pairs.

[0050] Figure 8 This is a schematic diagram of a 5×5 camera array.

[0051] Figure 9 Images of a chessboard in different poses.

[0052] Figure 10 A 5×5 image array captured by a 5×5 camera array.

[0053] Figure 11 This is the result of stitching together a 5×5 camera array.

[0054] The figure labels in the above figures are:

[0055] 1. Original input image with geometric distortion; 2. Image after geometric distortion correction; 3. Image after color calibration; 4. Output image after super-resolution reconstruction; 5. Local image before super-resolution reconstruction; 6. Local image after super-resolution reconstruction; 7. Single image to be stitched after preprocessing; 8. Wide image output by single-line stitching; 9. Complete stitched image; 10. Left image in a pair of adjacent images; 11. Right image in a pair of adjacent images; 12. Stitching region of adjacent image pairs; 13. Curve of weight coefficient p1(x,y) about the x-axis; 14. Curve of weight coefficient p2(x,y) about the x-axis.

[0056] It should be understood that the accompanying drawings are only schematic and are not drawn to scale. Detailed Implementation

[0057] The following detailed description of a typical embodiment of the image calibration and stitching method based on sub-pixel reconstruction and elliptic coefficient weighting using the present invention further illustrates the invention. It is important to note that the following embodiments are for illustrative purposes only and should not be construed as limiting the scope of protection of the present invention. Non-essential improvements and adjustments made to the present invention by those skilled in the art based on its content are still within the scope of protection of the present invention.

[0058] This embodiment includes the following steps:

[0059] Step 1: Perform precise calibration on each sub-camera in the 5x5 camera array in this example using a calibration board. A schematic diagram of the camera array is attached. Figure 8 As shown in Table 1, the intrinsic and extrinsic parameter matrices of each sub-camera in the third row after calibration are shown in Table 1.

[0060] Table 1 Partial Sub-camera Intrinsic and Extrinsic Parameter Matrix

[0061]

[0062] Step 2: Use the calibrated camera array to photograph the target scene, acquiring array images from multiple perspectives. All sub-cameras use uniform shooting parameters, a resolution of 4000×3000 pixels, and RAW image format. The overlap of the fields of view between adjacent cameras is 50%, ultimately resulting in an array of 25 original images in 5 rows and 5 columns. See the attached example. Figure 10 ;

[0063] Step 3: Preprocess the acquired images based on the calibration parameters, including geometric distortion correction, color calibration, and super-resolution reconstruction. The flowchart for this step is attached. Figure 2 As shown;

[0064] Step 4: Using the proposed weighted stitching algorithm based on elliptic coefficients, seamless stitching of the image array is achieved, generating a complete image with both a large field of view and high resolution. The generated image is shown in the attached image. Figure 11 As shown;

[0065] In step 1, the specific steps for camera calibration include:

[0066] Sub-step 1.1: Each sub-camera separately acquires 20 chessboard images in different poses, ensuring that the calibration board covers all areas of the camera's field of view and that the calibration board is fully visible in each image. Some of the images acquired during this process are attached. Figure 9 As shown;

[0067] Sub-step 1.2: Using OpenCV, calculate the intrinsic parameter matrix of each sub-camera based on the images acquired in sub-step 1.1;

[0068] Sub-step 1.3: For each row of cameras, calibrate the two adjacent sub-cameras from left to right: In the common field of view of each group of adjacent sub-cameras, acquire 5 sets of chessboard images with different poses to ensure that the pose of the calibration board covers the overlapping area of ​​the field of view of the adjacent cameras, and calculate the horizontal relative extrinsic parameters of each sub-camera.

[0069] Sub-step 1.4: For each column of cameras, calibrate the two adjacent sub-cameras from top to bottom: In the common field of view of each group of adjacent cameras, acquire 5 sets of chessboard images with different poses to ensure that the pose of the calibration board covers the overlapping area of ​​the field of view of the adjacent cameras, and calculate the vertical relative extrinsic parameters of each sub-camera.

[0070] Sub-step 1.5: Select the central sub-camera as the global baseline, and set its extrinsic parameters as the identity matrix and zero vector. Starting from the central sub-camera, calculate the global extrinsic parameters of other sub-cameras in the same row through a chain transformation;

[0071] Sub-step 1.6: Starting from each sub-camera in the center row, calculate the global extrinsic parameters of other sub-cameras in the same column through chain transformation.

[0072] In step 3, the specific steps of image preprocessing include:

[0073] Sub-step 3.1: Use calibration parameters to eliminate lens distortion in each image, that is, input the original input image 1 with geometric distortion, and output the image 2 after geometric distortion correction;

[0074] Sub-step 3.2: Use a multi-level progressive histogram matching algorithm to calibrate the colors and output the color-calibrated image;

[0075] Sub-step 3.3: The Real-ESRGAN network is used to perform 4x super-resolution reconstruction on the corrected image, increasing the image resolution from 4000×3000 pixels to 16000×12000 pixels, achieving sub-pixel level detail reconstruction. By comparing the magnified local areas before and after reconstruction, i.e., comparing the corresponding local areas of the final output image and the color-calibrated image, it can be observed that the high-frequency information of the reconstructed image is clearer than that of the original image.

[0076] In step 4, the specific sub-steps of image stitching include:

[0077] Sub-step 4.1: For the 5x5 image array processed in step 3, perform the following stitching operation row by row: First, group the images into pairs and stitch them using a weighted stitching algorithm based on elliptic coefficients. Then, continue to group the stitching results into pairs and repeat the stitching process until each row has only one seamlessly stitched wide image, finally obtaining a 5x1 image array.

[0078] Sub-step 4.2: Perform vertical stitching on the 5-row wide-format image processed in sub-step 4.1: First, group adjacent rows of images into pairs and stitch them together using a weighted stitching algorithm based on elliptic coefficients; then, continue to group the stitching results into pairs and repeat the stitching process to finally generate a complete image with both a large field of view and high resolution.

[0079] Step 4.1 involves performing a horizontal stitching process row by row on the 5x5 image array processed in step 3. In the first round of stitching, the first four images of each row of five images are grouped into pairs, i.e., the first two images are grouped together, and the third and fourth images are grouped together, forming two pairs of adjacent images. The width of the overlapping area of ​​each pair is Δx = 8000 pixels. The stitching area of ​​the left image is [8000, 15999] pixels, and the right image is [0, 7999] pixels. The stitching is completed using the weighted stitching algorithm based on the elliptic coefficient shown in equations (1), (2), and (3), and the two generated images are retained. In the second round, the first two images are stitched together again. In the third round, the image generated in the second round is stitched together with the fifth original image in the first round to obtain a 5x1 wide image array, where the resolution of a single image is 48000×12000 pixels.

[0080] Step 4.2 involves a vertical stitching process for the 5x1 intermediate image array obtained after step 4.1. In the first round of stitching, the first four images of the first five images are grouped into pairs, i.e., images 1-2 form one group, and images 3-4 form another group, forming two pairs of adjacent images. The height of the overlapping area in each group is Δy = 6000 pixels. Within each group, the vertical coordinate range of the stitching area of ​​the upper image is [6000, 11999] pixels, and the vertical coordinate range of the stitching area of ​​the lower image is [0, 5999] pixels. The first round of stitching is completed using the weighted stitching algorithm based on elliptic coefficients shown in equations (4), (5), and (6). After the first round of stitching, two intermediate results are retained and used in the next round of stitching along with the unstitched fifth original image. In the second round of stitching, there are three remaining images, and the first two are stitched together. In the third round of stitching, an intermediate image generated in the second round is stitched together with the remaining fifth original image from the first round, ultimately outputting a complete image with a resolution of 48000×36000 pixels.

[0081] This invention proposes an image calibration and stitching method based on sub-pixel reconstruction and elliptic coefficient weighting, overcoming the accuracy limitations of traditional image stitching techniques. The super-resolution reconstruction process improves image registration accuracy to the sub-pixel level, enhancing the ability of camera array imaging systems to reproduce details in large field-of-view scenes. Meanwhile, the adaptive weighting algorithm based on elliptic curve characteristics ensures a natural transition at stitching boundaries through its unique mathematical continuity, effectively eliminating common problems such as seams and brightness jumps in traditional methods. This combination of techniques not only achieves a balance between a large field of view and high resolution but also provides a reliable solution for applications with stringent image quality requirements, such as industrial inspection.

Claims

1. An image calibration and stitching method based on sub-pixel reconstruction and elliptic coefficient weighting, characterized in that, A super-resolution reconstruction module is introduced before image array stitching to improve stitching accuracy through sub-pixel-level image registration, achieving better stitching accuracy than single-pixel stitching. Addressing the edge misalignment and abrupt transitions in multi-camera array system field-of-view stitching, this method proposes a weighted stitching algorithm based on elliptic coefficients. This algorithm constructs an elliptic gradient weight model to achieve smooth transitions in image boundary regions, improving the overall visual effect of panoramic images. The method includes the following steps: Step 1: Use a calibration board to accurately calibrate each sub-camera in the N rows and M columns camera array; Step 2: Use a camera array to capture images of the target scene from multiple perspectives with overlapping fields of view; Step 3: Preprocess the acquired images based on the calibration parameters, including geometric distortion correction, color calibration, and super-resolution reconstruction; Step 4: Seamlessly stitch the image array using the proposed stitching algorithm based on elliptic coefficient weighting.

2. The image calibration and stitching method based on sub-pixel reconstruction and elliptic coefficient weighting according to claim 1, characterized in that, Step 1 involves using a calibration board to precisely calibrate each sub-camera in the N rows and M columns camera array. This includes the following sub-steps: Sub-step 1.1: Each sub-camera separately acquires several chessboard images in different poses, ensuring that all areas of the camera's field of view are covered, including the center, edges, and tilt direction, and that the calibration board is fully visible in each image; Sub-step 1.2: Based on the images acquired in sub-step 1.1, calculate the intrinsic parameter matrix for each sub-camera; Sub-step 1.3: For each row of cameras, calibrate the adjacent sub-cameras from left to right: In the common field of view of each group of adjacent sub-cameras, acquire multiple sets of chessboard images with different poses. The chessboard images must appear in both cameras at the same time to ensure that the calibration board pose covers the overlapping area of ​​the field of view of the adjacent cameras, and calculate the horizontal relative pose of each sub-camera. Sub-step 1.4: For each column of cameras, calibrate the adjacent sub-cameras from top to bottom: In the common field of view of each group of adjacent sub-cameras, acquire multiple sets of chessboard images with different poses. The chessboard images must appear in both cameras at the same time to ensure that the pose of the calibration board covers the overlapping area of ​​the field of view of the adjacent cameras, and calculate the vertical relative pose of each sub-camera. Sub-step 1.5: Select the central sub-camera as the global reference, whose extrinsic parameters are the identity matrix and the zero vector. Starting from the central sub-camera, calculate the global extrinsic parameters of other sub-cameras in the same row through chain transformation. Sub-step 1.6: Starting from each sub-camera in the center row, calculate the global extrinsic parameters of other sub-cameras in the same column through chain transformation.

3. The image calibration and stitching method based on sub-pixel reconstruction and elliptic coefficient weighting according to claim 1, characterized in that, In step 3, the acquired images are preprocessed based on the calibration parameters, including geometric distortion correction, color calibration, and super-resolution reconstruction. Specifically, this includes the following sub-steps: Sub-step 3.1: Eliminate lens distortion in each image using calibration parameters; Sub-step 3.2: Employ a multi-level progressive histogram matching algorithm to align the color distribution of adjacent images to calibrate the colors; Sub-step 3.3: Use a deep learning super-resolution reconstruction model to perform super-resolution reconstruction on the image to achieve sub-pixel level detail reconstruction.

4. The image calibration and stitching method based on sub-pixel reconstruction and elliptic coefficient weighting according to claim 1, characterized in that, In step 4, the proposed elliptic coefficient-weighted stitching algorithm is used to seamlessly stitch the image array, which specifically includes the following sub-steps: Sub-step 4.1: Define the images acquired by two adjacent sub-cameras as adjacent image pairs. Perform a stitching operation row by row on the N-row M-column image array processed in step 3: First, group the images into pairs and stitch them using a weighted stitching algorithm based on elliptic coefficients. Then, continue to group the stitching results into pairs and repeat the stitching process until each row has only one seamlessly stitched wide-width image, ultimately forming an N-row 1-column image array. Sub-step 4.2: Perform vertical stitching on the N-row wide-format image processed in sub-step 4.1: First, group adjacent rows of images into pairs and stitch them together using a weighted stitching algorithm based on elliptic coefficients; then, continue to group the stitching results into pairs and repeat the stitching process to finally generate a complete image with both a large field of view and high resolution.

5. The image calibration and stitching method based on sub-pixel reconstruction and elliptic coefficient weighting according to claim 4, characterized in that, In sub-step 4.1, the N rows and M columns of the image array are stitched together row by row, which specifically includes the following sub-steps: Sub-step 4.1.1: Assume each row of the image array has m images to be stitched (m≥2). In the first round of stitching, let m = M, where M is the initial number of images to be stitched in each row. The image grouping rule depends on the current value of m. After each round of stitching, m needs to be updated. The update rule is explained in sub-step 4.1.

5. When m is even, the images are grouped in pairs, with each pair of adjacent images from left to right forming K = m / 2 pairs of adjacent images. When m is odd, the first (m-1) images are grouped in pairs, with each pair of adjacent images from left to right forming K = (m-1) / 2 pairs of adjacent images. The remaining image will directly participate in the next round of grouping. Sub-step 4.1.2: Determine the stitching area for each of the K pairs of adjacent images. Let the width of the image to be stitched be X pixels and the height be Y pixels. After each stitching round, X and Y need to be updated. With the pixel at the top left corner of each image as the origin, and the horizontal direction from left to right as the positive direction of the x-axis and the vertical direction from top to bottom as the positive direction of the y-axis, establish a Cartesian coordinate system. Based on the proportion of the overlapping field of view of adjacent cameras, the width of the overlapping field of view between the K pairs of adjacent images can be determined as Δx pixels. Then, for the left image in a pair of adjacent images, the coordinate range of the area to be stitched can be represented as [X-Δx, X-1]; while for the right image, the coordinate range of the area to be stitched can be represented as [0, Δx-1]. Let the left edge horizontal coordinate of the area to be stitched in the left image be a1 = X-Δx and the right edge horizontal coordinate be b1 = X-1, and the left edge horizontal coordinate of the area to be stitched in the right image be a2 = 0 and the right edge horizontal coordinate be b2 = Δx-1 in this stitching round. Sub-step 4.1.3: Based on the characteristics of elliptic curves, construct a weight function that varies with the spatial position of the pixel, as shown in equations (1) and (2). In this equation, the weight coefficients p1(x,y) and p2(x,y) are the pixel weight values ​​of the two images in the (x,y) coordinates of the adjacent image pair. Shift p2(x,y) to the right along the x-axis so that the splicing area of ​​the two weight coefficient distribution curves completely overlaps. It can be observed that the superimposed weight values ​​of the image pixels exhibit a smooth elliptic curve transition characteristic along the splicing direction. Sub-step 4.1.4: Based on the weight values ​​of each pixel calculated in sub-step 4.1.3, the K groups of adjacent image pairs are stitched together according to equation (3). The stitching pattern is reflected in the distribution of weight coefficients: overlapping areas are fused according to the weight ratio, while non-overlapping areas directly retain the original image data, where I fused_ k(x,y) represents the pixel value of the image formed by stitching together the kth pair of adjacent images in the (x,y) coordinates. k (x,y) and I k '(x,y) represent the pixel values ​​of the two images in the k-th adjacent image pair at the (x,y) coordinates, respectively. k ′(x-a1,y) represents the second image I k A rightward translation of (a1,0) is applied. The purpose of this operation is to spatially align the overlapping regions of the two images, thereby ensuring that the weight functions p1(x,y) and p2(x-a1,y) can be applied correctly to the corresponding overlapping pixels of the two images. I fused_k (x,y)=p1(x,y)·I k (x,y)+p2(x-a1,y)·I k ′(x-a1,y) (3) Sub-step 4.1.5: If the current m is 2, then the horizontal stitching of the image in this row is completed; if m is not 2 and m is even in this round, then let m = K; if m is not 2 and m is odd in this round, then let m = K + 1, that is, add the last image that did not participate in the grouping in sub-step 4.1.1, and then return to sub-step 4.1.1 to continue stitching; if all images have been stitched row by row, forming an N-row 1-column image array, then proceed to sub-step 4.

2.

6. The image calibration and stitching method based on sub-pixel reconstruction and elliptic coefficient weighting according to claim 4, characterized in that, In sub-step 4.2, the N rows and 1 column image array is stitched together, which specifically includes the following sub-steps: Sub-step 4.2.1: Suppose there are n images to be stitched in the current column (n≥2). In the first round of stitching, let n=N, where N is the initial column number. The image grouping rule depends on the current value of n. After each round of stitching, n needs to be updated. The update rule is explained in sub-step 4.1.

5. When n is even, the images are grouped in pairs, with each pair of adjacent images from top to bottom forming L=n / 2 pairs of adjacent images. When n is odd, the first (n-1) images are grouped in pairs, with each pair of adjacent images from top to bottom forming L=(n-1) / 2 pairs of adjacent images. The remaining image will directly participate in the next round of grouping. Sub-step 4.2.2: Stitch together the L groups of adjacent image pairs formed by the above grouping. The width of the overlapping area between the L groups of adjacent image pairs can be determined as Δy pixels based on the proportion of the overlapping field of view of adjacent cameras. For the upper image in a group of adjacent image pairs, the vertical coordinate range of the area to be stitched is [Y-Δy, Y-1]; while for the lower image, the coordinate range of the area to be stitched is [0, Δy-1]. Let the vertical coordinate of the upper edge of the stitching area of ​​the upper image be c1 = Y-Δy and the vertical coordinate of the lower edge be c1 = Y-1 in this round of stitching, and the vertical coordinate of the upper edge of the stitching area of ​​the lower image be c2 = 0 and the vertical coordinate of the lower edge be d2 = Δy-1. Sub-step 4.2.3: Based on the characteristics of elliptic curves, construct a weight function that varies with the spatial position of the pixel, as shown in equations (4) and (5). The weight coefficients q1(x,y) and q2(x,y) are the pixel weight values ​​of the two images in the (x,y) coordinates of the l-th adjacent image pair, respectively. The weight values ​​exhibit a smooth elliptic curve transition characteristic along the splicing direction. Sub-step 4.2.4: Based on sub-step 4.2.3, calculate the weight value of each pixel, and perform stitching on L groups of adjacent image pairs according to equation (6). The stitching rule is reflected in the distribution of weight coefficients: overlapping areas are fused according to the weight ratio, while non-overlapping areas directly retain the original image data, where I fused_ l(x,y) represents the pixel value of the image formed by stitching together the l-th pair of adjacent images in the (x,y) coordinate system. l (x,y) and I l '(x,y) represent the pixel values ​​of the two images in the l-th adjacent image pair at the (x,y) coordinates; l ′(x,y-c1) represents the second image I l A downward translation of (c1,0) is applied. The purpose of this operation is to spatially align the overlapping regions of the two images, thereby ensuring that the weight functions q1(x,y) and q2(x,y-c1) can be applied correctly to the corresponding overlapping region pixels of the two images. I fused_l (x,y)=q1(x,x)·I l (x,y)+q2(x,y-c1)·I l ′(x,y-c1) (6) Sub-step 4.2.5: If the current n is not 2 and n is even, then let n = L; if the current n is not 2 and n is odd, then let n = L + 1 (that is, add the last image that did not participate in the grouping in sub-step 4.2.1), and then return to sub-step 4.2.1 to continue to perform vertical image stitching. If the current n is 2, then the image array stitching has been completed and the complete stitched image has been successfully obtained.

Images