A method and apparatus for image stitching of display panels based on multiple cameras

Abstract

This application discloses a method and apparatus for stitching display panel images based on multiple cameras, used to improve the smoothness and stitching effect of the stitched images. The method includes: acquiring several local images of a target panel captured by a multi-camera system, wherein one camera corresponds to one local image; performing geometric distortion correction on each of the local images; performing regional pixel alignment on the overlapping areas of two adjacent local images to determine a first seam region; aligning pixel grayscale information in the first seam region and performing image fusion to obtain the target image.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method and apparatus for stitching images on a display panel based on multiple cameras. Background Technology

[0002] With the development of display and imaging technologies, the demand for acquiring high-resolution, large-area images is increasing. In many applications, such as large display panel inspection or multi-view scene reconstruction, a single camera is insufficient to cover the entire target area; therefore, multi-camera systems are typically used to acquire local images of the target.

[0003] In existing technologies, local images captured by multiple cameras need to be stitched together to form a complete image. However, when dealing with overlapping areas of adjacent images, existing methods often suffer from poor grayscale and visual continuity at the seam. This poor continuity directly leads to unnatural appearances at the seams in the stitched image, thus affecting the overall stitching effect.

[0004] Therefore, existing technologies still suffer from poor continuity in the seam area when stitching multi-camera images, and it is necessary to take improvement measures to enhance the smoothness and stitching effect of the stitched images. Summary of the Invention

[0005] This application provides a method and apparatus for stitching display panel images based on multiple cameras, which can improve the smoothness and stitching effect of the stitched images.

[0006] The first aspect of this application provides a method for stitching display panel images based on multiple cameras, including:

[0007] Acquire several partial images of the target panel captured by a multi-camera system, wherein one camera corresponds to one partial image;

[0008] Geometric distortion correction is performed on each of the aforementioned local images;

[0009] The overlapping areas of two adjacent local images are aligned by pixels to determine the first seam area;

[0010] The pixel grayscale information of the first seam area is aligned and the image is fused to obtain the target image.

[0011] Optionally, the step of aligning pixel grayscale information and performing image fusion on the first seam area to obtain the target image includes:

[0012] Obtain the fusion coefficient corresponding to each pixel within the pre-stored seam area;

[0013] The first seam region is aligned with pixel grayscale information and image fusion is performed based on the fusion coefficient to obtain the target image.

[0014] Optionally, before obtaining the fusion coefficient corresponding to each pixel within the pre-stored seam area, the method further includes:

[0015] Acquire several calibrated local images that have been aligned with pixel grayscale information. These calibrated local images are obtained by the multi-camera system capturing and displaying grayscale images on a PG dot screen.

[0016] The sampling method is determined, including row sampling and column sampling;

[0017] The grayscale values ​​of several calibration local images are sampled according to the sampling method to obtain the grayscale distribution curve of each calibration local image;

[0018] Based on the grayscale distribution curve, determine the ideal grayscale curve for the second seam region of several calibrated local images;

[0019] The fusion coefficient for each pixel is calculated based on the ideal grayscale curve.

[0020] Optionally, determining the ideal grayscale curves for the second seam regions of the calibrated local images based on the grayscale distribution curves includes:

[0021] The least squares method is used to perform polynomial fitting on the gray-level distribution curves of adjacent images to determine the ideal gray-level curve of the second seam region.

[0022] Optionally, before sampling the grayscale values ​​of several calibration local images according to the sampling method to obtain the grayscale distribution curve of each calibration local image, the method further includes:

[0023] Calculate the average gray value of each pixel in the second seam region of several calibrated local images, and use the average gray value as the gray value of each pixel in the second seam region.

[0024] Optionally, the overlapping regions of two adjacent local images are aligned by pixel segmentation, including:

[0025] According to a preset partitioning rule, the overlapping regions of adjacent images in multiple local images are segmented.

[0026] Regional pixel alignment is performed using the pre-stored homography transformation matrix corresponding to each region.

[0027] Optionally, before acquiring the group of partial images of the target panel captured by the multi-camera system, the method further includes:

[0028] Obtain the average grayscale value of standard panel images at different grayscale levels captured by each camera of the multi-camera system at different exposure times;

[0029] Establish a polynomial relationship between the average gray value and the exposure time;

[0030] The compensation coefficients for each camera in the multi-camera system are calculated based on the polynomial relationship.

[0031] Optionally, performing geometric distortion correction on each of the several local images includes:

[0032] For each camera in the multi-camera system, acquire multiple pose images obtained through a circular dot matrix calibration map;

[0033] Identify the image coordinates and world coordinates of each marker point in the circular dot matrix calibration diagram;

[0034] Based on Zhang Zhengyou's calibration method, the camera intrinsic parameter matrix and distortion coefficients are calculated using the image coordinates and the world coordinates.

[0035] Distortion correction and pixel reprojection are performed on several local images based on the camera intrinsic parameter matrix and distortion coefficients.

[0036] A second aspect of this application provides a multi-camera-based display panel image stitching device, comprising:

[0037] The acquisition unit is used to acquire several partial images of the target panel captured by the multi-camera system, wherein one camera corresponds to one partial image;

[0038] A correction unit is used to perform geometric distortion correction on several of the local images respectively;

[0039] A pixel alignment unit is used to perform regional pixel alignment on the overlapping areas of two adjacent local images to determine the first seam area;

[0040] The grayscale alignment unit is used to align the pixel grayscale information of the first seam area and perform image fusion to obtain the target image.

[0041] Optionally, the grayscale alignment unit includes:

[0042] The first acquisition module is used to acquire the fusion coefficient corresponding to each pixel in the pre-stored seam area;

[0043] The alignment module is used to align the pixel grayscale information of the first seam area according to the fusion coefficient and perform image fusion to obtain the target image.

[0044] Optionally, the device further includes a computing unit, the computing unit comprising:

[0045] The second acquisition module is used to acquire several calibrated local images that have been aligned with pixel grayscale information. The several calibrated local images are obtained by the multi-camera system capturing and displaying grayscale images on a PG dot screen.

[0046] The first determining module is used to determine the sampling method, which includes row sampling and column sampling;

[0047] The sampling module is used to sample the gray values ​​of several calibration local images according to the sampling method to obtain the gray value distribution curve of each calibration local image;

[0048] The second determining module is used to determine the ideal grayscale curve of the second seam region of several of the calibrated local images based on the grayscale distribution curve.

[0049] The first calculation module is used to calculate the fusion coefficient corresponding to each pixel based on the ideal grayscale curve.

[0050] Optionally, the second determining module is specifically used for:

[0051] The least squares method is used to perform polynomial fitting on the gray-level distribution curves of adjacent images to determine the ideal gray-level curve of the second seam region.

[0052] Optionally, the computing unit further includes:

[0053] The second calculation module is used to calculate the average gray value of each pixel in the second seam region of several calibrated local images, and use the average gray value as the gray value of each pixel in the second seam region.

[0054] Optionally, the pixel alignment unit is specifically used for:

[0055] The overlapping regions of two adjacent local images are segmented according to a preset partitioning rule;

[0056] Regional pixel alignment is performed using the pre-stored homography transformation matrix corresponding to each region.

[0057] Optionally, the device further includes a compensation unit, the compensation unit being used for:

[0058] Obtain the average grayscale value of standard panel images at different grayscale levels captured by each camera of the multi-camera system at different exposure times;

[0059] Establish a polynomial relationship between the average gray value and the exposure time;

[0060] The compensation coefficients for each camera in the multi-camera system are calculated based on the polynomial relationship.

[0061] Optionally, the correction unit is specifically used for:

[0062] For each camera in the multi-camera system, acquire multiple pose images obtained through a circular dot matrix calibration map;

[0063] Identify the image coordinates and world coordinates of each marker point in the circular dot matrix calibration diagram;

[0064] Based on Zhang Zhengyou's calibration method, the camera intrinsic parameter matrix and distortion coefficients are calculated using the image coordinates and the world coordinates.

[0065] Distortion correction and pixel reprojection are performed on several local images based on the camera intrinsic parameter matrix and distortion coefficients.

[0066] A third aspect of this application provides an electronic device, comprising:

[0067] Processor, memory, input / output units, and bus;

[0068] The processor is connected to the memory, the input / output unit, and the bus;

[0069] The memory stores a program, and the processor calls the program to execute the method of the first aspect and any possible implementation of the first aspect.

[0070] The fourth aspect of this application provides a computer-readable storage medium storing a program that, when executed on a computer, causes the computer to perform the methods of the first aspect and any possible implementation thereof.

[0071] As can be seen from the above technical solutions, this application has the following advantages:

[0072] The method provided in this application first uses a calibrated multi-camera system to acquire several local images with spatial correspondence, then uses geometric distortion correction to eliminate image deformation caused by lens and angle, then performs sub-regional pixel alignment in the overlapping area of ​​two adjacent local images to accurately determine the first seam area, and finally achieves smooth stitching by aligning grayscale information and image fusion in the first seam area, thereby reducing the possibility of seam misalignment and brightness inconsistency in the target image during the stitching process, ensuring the overall continuity and high-quality display effect of the target image, and improving the smoothness and stitching effect of the target image. Attached Figure Description

[0073] Figure 1This is a flowchart illustrating an embodiment of the multi-camera-based display panel image stitching method in this application;

[0074] Figure 2 This is a flowchart illustrating an embodiment of aligning pixel grayscale information and performing image fusion in the first seam region in this application.

[0075] Figure 3 This is a flowchart illustrating an embodiment of calculating the fusion coefficient for each pixel in this application;

[0076] Figure 4 This is a flowchart illustrating one embodiment of pixel alignment in this application;

[0077] Figure 5 This is a schematic flowchart of one embodiment of calculating the compensation coefficient in this application;

[0078] Figure 6 This is a flowchart illustrating one embodiment of geometric distortion correction in this application;

[0079] Figure 7 This is a schematic diagram of the structure of one embodiment of the multi-camera-based display panel image stitching device in this application;

[0080] Figure 8 This is a schematic diagram of the structure of one embodiment of the electronic device in this application. Detailed Implementation

[0081] This application provides a method and apparatus for stitching display panel images based on multiple cameras, which improves the smoothness and stitching effect of the stitched images.

[0082] The method described in this application can be applied to servers, terminals, or other devices with logical processing capabilities; therefore, this application does not limit its application. For ease of description, the following description uses a server as the executing entity.

[0083] The embodiments of this application will now be described with reference to the accompanying drawings.

[0084] Please see Figure 1 , Figure 1 An embodiment of the multi-camera-based display panel image stitching method provided in this application includes:

[0085] 101. Acquire several partial images of the target panel captured by the multi-camera system, wherein one camera corresponds to one partial image;

[0086] The terminal first uses a multi-camera system to simultaneously capture images of the target panel. It's important to note that the new multi-camera system can be calibrated or adjusted beforehand to ensure that the optical center position, focal length, and angular parameters of different cameras remain stable during imaging, thus avoiding significant spatial deviations between images. The terminal stores the local images captured by each camera as independent data units and combines all local images into several local images, ensuring that each local image corresponds one-to-one with a fixed area of ​​the target panel, providing complete basic data for subsequent image processing steps.

[0087] 102. Perform geometric distortion correction on several local images respectively;

[0088] After acquiring several local images, the terminal performs geometric distortion correction. Due to the influence of camera lens and shooting angle, local images may have distortion problems such as barrel distortion or pincushion distortion. If not processed, edge misalignment will occur during image stitching. The terminal uses the intrinsic and extrinsic parameters obtained during camera calibration to correct each local image, restoring the distorted areas in the image to a geometric shape close to the true proportions. The local images processed in this way maintain consistency in geometric structure, laying an accurate foundation for subsequent image alignment and stitching.

[0089] 103. Perform pixel alignment on the overlapping areas of two adjacent local images to determine the first seam area;

[0090] After geometric distortion correction is completed for the local images, the terminal further analyzes the overlapping areas of two adjacent local images. To ensure the precision of the stitching, the terminal divides the overlapping area into several sub-regions, and then compares and adjusts the positions of pixels one by one within each sub-region, so that the two adjacent local images maintain a continuous pixel matching relationship at the boundary. In this way, the terminal can accurately determine the first seam area between the two images and ensure that there is no spatial misalignment in this seam area.

[0091] 104. Align the pixel grayscale information of the first seam area and perform image fusion to obtain the target image.

[0092] After the seam area is determined, the terminal performs pixel grayscale information alignment on the first seam area. Even though the cameras use the same specifications and the lighting conditions are basically consistent, slight differences in brightness or grayscale may still occur during imaging. To eliminate these differences, the terminal adjusts the pixel grayscale values ​​within the seam area to ensure consistent grayscale distribution in adjacent areas. After grayscale alignment, the terminal performs image fusion processing on the seam area, gradually stitching all local images into a complete target image, and smoothing the transition boundaries during the fusion process to avoid obvious stitching marks.

[0093] In this embodiment, the terminal first uses a multi-camera system to acquire several local images with spatial correspondence, then uses geometric distortion correction to eliminate image deformation caused by lens and angle, then performs sub-regional pixel alignment in the overlapping area of ​​two adjacent local images to accurately determine the first seam area, and finally achieves smooth stitching by aligning grayscale information and image fusion in the first seam area, thereby reducing the possibility of seam misalignment and brightness inconsistency in the target image during the stitching process, and ensuring the overall continuity and high-quality display effect of the target image.

[0094] Please see Figure 2 In some embodiments of this application, step 104 in the above embodiments, which aligns pixel grayscale information and performs image fusion in the first seam region to obtain the target image, may include the following steps:

[0095] 201. Obtain the fusion coefficient corresponding to each pixel within the pre-stored seam area;

[0096] The terminal first reads a pre-stored table of fusion coefficients. These fusion coefficients are pre-defined proportional parameters for each pixel within the seam area, used to correct the grayscale value of that pixel during image fusion. Each pixel corresponds to a fusion coefficient, and the terminal uses these coefficients to provide a clear numerical basis for subsequent grayscale adjustments.

[0097] 202. Align the pixel grayscale information of the first seam region according to the fusion coefficient and perform image fusion to obtain the target image.

[0098] After obtaining the fusion coefficient, the terminal calculates the grayscale values ​​of the pixels within the first seam area. If the measured grayscale value of a pixel is E and its corresponding fusion coefficient is C, the terminal adjusts the final grayscale value of that pixel to C*E. In this way, slight grayscale differences that originally existed within the seam area can be corrected, ensuring that adjacent local images maintain continuity and consistency in grayscale. After completing the grayscale adjustment, the terminal performs smooth fusion on the seam area, seamlessly stitching the local images into a complete target image.

[0099] In this embodiment, the terminal corrects the pixel grayscale values ​​one by one by introducing a fusion coefficient in the seam area, and completes image fusion on this basis, making the grayscale transition between local images more natural, thereby reducing the possibility of brightness abrupt changes or abrupt boundaries in the seam area, and improving the smoothness and display consistency of the target image.

[0100] Please see Figure 3In some embodiments of this application, before obtaining the fusion coefficient corresponding to each pixel in the pre-stored seam area in step 201 of the above embodiments, the multi-camera-based display panel image stitching method may further include the following steps:

[0101] 301. Acquire several calibrated local images that have been aligned with pixel grayscale information. These calibrated local images are obtained by capturing images of the PG dot screen of the grayscale image using a multi-camera system.

[0102] The terminal first uses a multi-camera system to capture standard grayscale images of the PG dot screen, obtaining several calibration local images containing multiple local regions. Since the PG dot screen can stably output grayscale images of different levels, the terminal can ensure that the calibration local images have a standardized benchmark for grayscale comparison and adjustment during acquisition. Then, the terminal performs preliminary pixel grayscale information alignment on these local images, ensuring that the overall brightness distribution between the images is within a unified reference frame.

[0103] 302. Determine the sampling method, which includes row sampling and column sampling;

[0104] After acquiring several calibrated local images, the terminal selects a sampling method based on the image stitching requirements. The sampling method can be either row sampling or column sampling. Row sampling is suitable for extracting grayscale information along the row direction, while column sampling is suitable for extracting grayscale information along the column direction. By determining the sampling method, the terminal provides a clear path for subsequent grayscale data acquisition. The sampling method is determined by the multi-camera system. When the multi-camera system uses horizontally positioned cameras and the captured images need to be stitched horizontally, then row sampling should be selected; when the multi-camera system uses vertically positioned cameras and the captured images need to be stitched vertically, then column sampling should be selected.

[0105] 303. Based on the sampling method, perform grayscale value sampling on several calibration local images to obtain the grayscale distribution curve of each calibration local image;

[0106] The terminal extracts pixel grayscale values ​​from several calibrated local images according to the selected sampling method and a preset sampling interval (the sampling interval can be set according to the situation; to improve accuracy, the sampling interval can be set to 0, that is, each pixel is sampled). The sampling results are then processed to generate a corresponding grayscale distribution curve. In this way, each calibrated local image can intuitively reflect the grayscale change with position through the shape of the curve, and the terminal can analyze the differences in grayscale representation between different local images based on this.

[0107] 304. Determine the ideal grayscale curves for the second seam regions of several calibrated local images based on the grayscale distribution curves;

[0108] After obtaining the grayscale distribution curve, the terminal compares and analyzes the grayscale performance of different calibrated local images in the second seam region. Combining this with actual sampling data, the terminal determines the ideal grayscale curve for this second seam region, ensuring it represents the standard grayscale transition effect that should be presented under conditions of no difference. This ideal grayscale curve serves as a benchmark, providing a target reference for calculating the fusion coefficient.

[0109] Specifically, the terminal uses the least squares method to iteratively obtain polynomial coefficients. These coefficients are then used as constant coefficients for the ideal grayscale curve of the second seam region, and polynomial fitting is performed on the grayscale distribution curves of adjacent images. During execution, the terminal first initializes the polynomial coefficients, then calculates the error between the fitted curve and the actual sampled grayscale values ​​to guide iterative adjustments. Various methods can be used to measure the fitting error, such as calculating the sum of squared residuals, calculating the sum of absolute residuals, minimizing the maximum residual, or using weighted residual minimization. The terminal iteratively adjusts the polynomial coefficients according to the selected error metric, gradually approximating the ideal grayscale transition law of the seam region. The polynomial coefficients obtained after iteration, i.e., the constant coefficients in the fitted curve, constitute the ideal grayscale curve, providing a standardized grayscale reference for subsequent calculation of the fusion coefficients for each pixel.

[0110] 305. Calculate the fusion coefficient corresponding to each pixel in the second seam area based on the ideal grayscale curve.

[0111] After obtaining the ideal grayscale curve, the terminal compares the actual grayscale value of each pixel within the second seam area with the ideal grayscale value, and then calculates the fusion coefficient for each pixel. If the actual grayscale value of a pixel is E, and its expected value on the ideal grayscale curve is R, then the fusion coefficient C can be set to R / E. In this way, the terminal generates a fusion coefficient for each pixel in the second seam area, and subsequently uses this coefficient to adjust the pixel grayscale value to achieve alignment and fusion.

[0112] In this embodiment, the terminal first acquires several calibrated local images using the grayscale image displayed on the PG dot screen, then generates a grayscale distribution curve through sampling, determines the ideal grayscale curve from the curve, and finally calculates the fusion coefficient corresponding to the pixel. This ensures that the fusion coefficient comes from a standardized grayscale reference, making the grayscale correction in the seam area accurate and consistent, thereby reducing the possibility of grayscale drift during the fusion process and improving the stability and reliability of the final target image.

[0113] Furthermore, before sampling grayscale values ​​from several calibrated local images according to the sampling method to obtain the grayscale distribution curve, the terminal processes the pixels in the second seam region. Specifically, the terminal first identifies each pixel within the seam region. For pixels in non-seam regions, their original grayscale values ​​are directly retained without adjustment. For pixels in the seam region, the terminal averages the original grayscale values ​​of the corresponding positions in the two overlapping images and uses this average as the grayscale value of the pixel. In this way, the terminal can effectively smooth the grayscale transition in the seam region while maintaining the original grayscale information in the non-seam regions, providing stable and continuous basic data for the subsequent generation of the grayscale distribution curve and the ideal grayscale curve.

[0114] Please see Figure 4 In some embodiments of this application, step 103 in the above embodiments, which performs region-by-region pixel alignment on the overlapping regions of two adjacent local images, may include the following steps:

[0115] 401. Perform region segmentation on the overlapping areas of two adjacent local images according to preset partitioning rules;

[0116] The terminal first divides the overlapping areas of adjacent local images into multiple sub-regions according to pre-defined partitioning rules. These partitioning rules can be based on pixel location, geometric grids, or other features to ensure relatively uniform image content within each sub-region and facilitate pixel alignment. By dividing the overlapping area into several controllable sub-regions, the terminal can employ precise alignment strategies for different regions during subsequent processing, improving the overall accuracy and stability of pixel alignment.

[0117] 402. Perform sub-region pixel alignment using the pre-stored homography transformation matrix corresponding to each region.

[0118] After region segmentation, the terminal acquires or calculates the corresponding homography transformation matrix for each sub-region. This matrix describes the geometric mapping relationship between adjacent images. The terminal uses the matrix to transform the pixel coordinates within the sub-region, achieving precise pixel alignment. By performing the transformation individually on each partition, the terminal can effectively correct local geometric differences between images, enabling high-precision alignment of adjacent local images in overlapping areas. This provides an accurate foundation for subsequent grayscale alignment of seam areas and image fusion.

[0119] In this embodiment, the terminal can effectively eliminate local geometric deviations between adjacent images by partitioning the overlapping area and using regional homography transformation for pixel alignment, thereby making the image content in the seam area more accurately aligned and improving the splicing accuracy and visual continuity of the final target image.

[0120] Please see Figure 5In some embodiments of this application, before acquiring several partial images of the target panel captured by the multi-camera system in step 101 of the above embodiments, the multi-camera-based display panel image stitching method may further include the following steps:

[0121] 501. Obtain the average grayscale value of standard panel images at different grayscale levels captured by each camera of the multi-camera system at different exposure times;

[0122] Before image stitching, the terminal first acquires the average grayscale values ​​of standard panel images taken by each camera in the multi-camera system at different exposure times. Specifically, the terminal sets up a fixed multi-camera imaging system, recording parameters such as lens, gain, exposure, and aperture of each camera to ensure stable and consistent imaging conditions. Then, a standard light source is selected, and a standard panel with uniform brightness at all four corners and no parallax is set as the imaging reference. The standard panel is illuminated by the light source at the same angle and brightness as in the subsequent detection stage, placing it at the center of the field of view of each camera. Based on this, the terminal displays multiple grayscale images (e.g., 25, 60, 128, and 225 grayscale levels) on the screen and sets different exposure time intervals in the camera software for image acquisition. The terminal calculates the average grayscale value of the standard panel area captured by each camera, thereby obtaining grayscale response data for each camera under different exposure time conditions.

[0123] 502. Establish a polynomial relationship between average grayscale value and exposure time;

[0124] After acquiring the average grayscale values ​​of each camera at different exposure times, the terminal fits the correspondence between grayscale values ​​and exposure times. The terminal uses the least squares method to perform polynomial fitting on the sample data between grayscale values ​​and exposure times, determining the order of the fit and the coefficients of each term, thereby establishing a polynomial relationship between the average grayscale value and exposure time. This polynomial relationship reflects the brightness response characteristics of different cameras under different exposure conditions, and can mathematically characterize the camera's exposure response curve, providing a basis for subsequent brightness compensation calculations.

[0125] 503. Calculate the compensation coefficients of each camera in a multi-camera system based on polynomial relationships.

[0126] After establishing a polynomial relationship between average grayscale value and exposure time, the terminal uses the exposure parameters of a reference camera as a standard and substitutes the average grayscale values ​​obtained by other cameras under the same standard panel conditions into the polynomial relationship to calculate the corresponding exposure values. The terminal calculates the ratio between this exposure value and the reference exposure value, and uses this ratio as the compensation coefficient for that camera. Subsequently, during the formal image acquisition stage, the terminal multiplies the pixel values ​​of the images output by each camera by the corresponding compensation coefficient, enabling the multi-camera system to achieve brightness consistency at the image input level, providing a balanced grayscale basis for the subsequent stitching process.

[0127] In this embodiment, the terminal first obtains the average grayscale value of each camera under different exposure times, then establishes a polynomial relationship between the average grayscale value and the exposure time through fitting, and finally calculates the compensation coefficient of each camera based on this relationship, so that the multi-camera system achieves brightness balance in the output stage. When the brightness consistency of the input image is guaranteed, the grayscale difference between adjacent images at the stitching boundary is effectively reduced, and the overlapping areas of the images can be naturally connected, thereby improving the continuity of the stitching area and ensuring that the overall image after stitching is smooth without discontinuities and has a consistent visual effect.

[0128] Please see Figure 6 In some embodiments of this application, step 104 in the above embodiments performs geometric distortion correction on several local images respectively, and may include the following steps:

[0129] 601. For each camera in a multi-camera system, acquire multiple pose images obtained through a circular dot matrix calibration map;

[0130] For each camera in the multi-camera system, the terminal acquires multiple pose images of a circular dot matrix calibration map for calibration. Specifically, the terminal uses a circular dot matrix calibration map as the calibration map. The row and column spacing between adjacent circular markers in the dot matrix can be, for example, 400 image pixels or other suitable pixel spacing, and the physical coordinates of each marker are known. The terminal places the calibration map in the calibration space, arranging it at different angles and poses, so that the camera captures a group of calibration images with different poses from multiple perspectives, thereby providing rich calibration samples for subsequent calculation of camera parameters.

[0131] 602. Identify the image coordinates and world coordinates of each marked point in the circular dot matrix calibration diagram;

[0132] The terminal performs marker point recognition on each acquired calibration image, using image processing algorithms to detect the centroid position of each marker point in the circular dot matrix, thereby obtaining the pixel coordinates of each marker point in the image. Simultaneously, based on the known geometry of the calibration image, the physical coordinates of these marker points in the world coordinate system are determined. In this way, the terminal obtains a one-to-one correspondence between image coordinates and world coordinates for each pose image, providing data input for subsequent parameter calculation.

[0133] 603. Based on Zhang Zhengyou's calibration method, calculate the camera intrinsic parameter matrix and distortion coefficients using image coordinates and world coordinates;

[0134] After acquiring the coordinate correspondence of each pose image, the terminal uses the Zhang Zhengyou calibration method to calculate the intrinsic parameter matrix and distortion coefficients of each camera. Specifically, the terminal establishes a mathematical model using multiple sets of image coordinates and world coordinates, and solves for intrinsic parameters such as the camera's focal length and principal point coordinates by minimizing projection errors, thereby obtaining the intrinsic parameter matrix; simultaneously, it solves for the radial and tangential distortion coefficients in the lens distortion model. The intrinsic parameter matrix characterizes the geometric properties of camera imaging, while the distortion coefficients reflect the degree of optical distortion of the lens; both are used together to describe the camera's imaging model.

[0135] 604. Perform distortion correction and pixel reprojection on several local images based on the camera intrinsic parameter matrix and distortion coefficients.

[0136] After completing camera calibration, the terminal performs distortion correction and pixel reprojection on several local images captured by the multi-camera system based on the calculated camera intrinsic parameter matrix and distortion coefficients. First, the terminal corrects radial and tangential distortions in the local images, eliminating image edge stretching or bending caused by lens structure. Then, to further improve pixel correspondence accuracy during subsequent stitching, the terminal adjusts the image angle and scale to ensure the panel boundary remains parallel to the camera's field of view edge, and achieves overall scale consistency through affine transformation. Finally, the terminal performs pixel reprojection according to the corrected geometric model and uses bilinear interpolation to generate corrected local images, thereby obtaining an image set with accurate geometry and minimal distortion, providing precise input for stitching.

[0137] In this embodiment, the terminal acquires multi-pose circular dot matrix calibration maps, identifies marker points, calculates camera parameters using the Zhang Zhengyou calibration method, and performs distortion correction and reprojection using the obtained intrinsic parameter matrix and distortion coefficients, thereby significantly improving the geometric accuracy and consistency of local images. Once image distortion is effectively eliminated, images captured by different cameras are more spatially and scale-wise matched, enabling accurate alignment at the pixel level in subsequent stitching processes. This ensures structural continuity and imaging consistency in the stitched area, ultimately improving the overall stitching effect of the target image.

[0138] Please see Figure 7 , Figure 7 An embodiment of the multi-camera-based display panel image stitching assembly provided in this application includes:

[0139] The acquisition unit 701 is used to acquire several partial images of the target panel captured by the multi-camera system, wherein one camera corresponds to one partial image;

[0140] The correction unit 702 is used to perform geometric distortion correction on several local images respectively;

[0141] Pixel alignment unit 703 is used to perform regional pixel alignment on the overlapping areas of two adjacent local images to determine the first seam area;

[0142] The grayscale alignment unit 704 is used to align the pixel grayscale information of the first seam area and perform image fusion to obtain the target image.

[0143] In this embodiment, the acquisition unit 701 first uses a calibrated multi-camera system to acquire a group of local images with spatial correspondence. The correction unit 702 then uses geometric distortion correction to eliminate image deformation caused by the lens and angle. Next, the pixel alignment unit 703 performs regional pixel alignment in the overlapping area of ​​adjacent images to accurately determine the seam area. Finally, the grayscale alignment unit 704 achieves smooth stitching in the seam area through grayscale information alignment and image fusion, thereby reducing the possibility of seam misalignment and brightness inconsistency in the target image during the stitching process, and ensuring the overall continuity and high-quality display effect of the target image.

[0144] Optionally, the grayscale alignment unit 704 includes:

[0145] The first acquisition module is used to acquire the fusion coefficient corresponding to each pixel in the pre-stored seam area;

[0146] The alignment module is used to align the pixel grayscale information of the first seam region according to the fusion coefficient and perform image fusion to obtain the target image.

[0147] Optionally, the device further includes a computing unit, which includes:

[0148] The second acquisition module is used to acquire several calibrated local images that have been aligned with pixel grayscale information. These several calibrated local images are obtained by the multi-camera system capturing images of the PG dot screen that displays grayscale images.

[0149] The first determining module is used to determine the sampling method, which includes row sampling and column sampling.

[0150] The sampling module is used to sample the gray values ​​of several calibration local images according to the sampling method, so as to obtain the gray distribution curve of each calibration local image;

[0151] The second determining module is used to determine the ideal grayscale curve of the second seam region of several calibrated local images based on the grayscale distribution curve.

[0152] The first calculation module is used to calculate the fusion coefficient corresponding to each pixel based on the ideal grayscale curve.

[0153] Optionally, the second determining module is specifically used for:

[0154] The least squares method is used to perform polynomial fitting on the gray-level distribution curves of adjacent images to determine the ideal gray-level curve of the second seam region.

[0155] Optionally, the computing unit may also include:

[0156] The second calculation module is used to calculate the average gray value of each pixel in the second seam region of several calibrated local images, and use the average gray value as the gray value of each pixel in the second seam region.

[0157] Optionally, the pixel alignment unit 703 is specifically used for:

[0158] The overlapping regions of two adjacent local images are segmented according to the preset partitioning rules;

[0159] Regional pixel alignment is performed using the pre-stored homography transformation matrix corresponding to each region.

[0160] Optionally, the device further includes a compensation unit, which is used for:

[0161] Obtain the average grayscale value of standard panel images at different grayscale levels captured by each camera of the multi-camera system at different exposure times;

[0162] Establish a polynomial relationship between average grayscale value and exposure time;

[0163] The compensation coefficients for each camera in a multi-camera system are calculated based on polynomial relationships.

[0164] Optionally, the correction unit 702 is specifically used for:

[0165] For each camera in a multi-camera system, acquire multiple pose images obtained through a circular dot matrix calibration map;

[0166] Identify the image coordinates and world coordinates of each marked point in the circular dot matrix calibration diagram;

[0167] Based on Zhang Zhengyou's calibration method, the camera intrinsic parameter matrix and distortion coefficients are calculated using image coordinates and world coordinates;

[0168] Distortion correction and pixel reprojection are performed on several local images based on the camera intrinsic parameter matrix and distortion coefficients.

[0169] In this implementation, the functions of each unit and module are as described above. Figures 1 to 6 The steps in the illustrated embodiments are the same and will not be repeated here.

[0170] Please see Figure 8 , Figure 8 One embodiment of the electronic device provided in this application includes:

[0171] Processor 801, memory 802, input / output unit 803, and bus 804;

[0172] The processor 801 is connected to the memory 802, the input / output unit 803, and the bus 804;

[0173] The memory 802 stores a program, which the processor 801 calls to execute. Figures 1 to 6 The steps in the illustrated embodiment.

[0174] In this embodiment, the function of processor 801 is the same as described above. Figures 1 to 6 The steps in the illustrated embodiments are the same and will not be repeated here.

[0175] This application also provides a computer-readable storage medium on which a program is stored. When the program is executed on a computer, it causes the computer to perform the aforementioned actions. Figures 1 to 6 The method in any possible implementation.

[0176] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0177] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between apparatuses or units through some interfaces, and may be electrical, mechanical, or other forms.

[0178] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0179] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0180] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A multi-camera based display panel image stitching method, characterized in that, The method comprises the following steps: acquiring a plurality of local images of a target panel captured by a multi-camera system, wherein one camera corresponds to one local image; performing geometric distortion correction on each of the plurality of local images; performing sub-regional pixel alignment on the overlapping regions of two adjacent local images to determine a first joint region; performing pixel grayscale information alignment and image fusion on the first joint region to obtain a target image; the pixel grayscale information alignment and image fusion on the first joint region to obtain a target image, comprising: acquiring a pre-stored fusion coefficient corresponding to each pixel in the joint region; performing pixel grayscale information alignment and image fusion on the first joint region according to the fusion coefficient to obtain a target image; before the step of acquiring a pre-stored fusion coefficient corresponding to each pixel in the joint region, the method further comprises: acquiring a plurality of calibration local images subjected to pixel grayscale information alignment, wherein the plurality of calibration local images are obtained by a PG point screen displaying a gray scale image; determining a sampling mode, wherein the sampling mode comprises row sampling and column sampling; sampling the gray scale values of the plurality of calibration local images according to the sampling mode to obtain a gray scale distribution curve of each calibration local image; determining an ideal gray scale curve of a second joint region of the plurality of calibration local images according to the gray scale distribution curve; calculating the fusion coefficient corresponding to each pixel in the second joint region according to the ideal gray scale curve.

2. The method of claim 1, wherein, the step of determining an ideal gray scale curve of a second joint region of the plurality of calibration local images according to the gray scale distribution curve, comprising: performing polynomial fitting on the gray scale distribution curves of adjacent images by using the least square method to determine the ideal gray scale curve of the second joint region.

3. The method of claim 1, wherein, before the step of sampling the gray scale values of the plurality of calibration local images according to the sampling mode to obtain a gray scale distribution curve of each calibration local image, the method further comprises: calculating the average gray scale value corresponding to each pixel point in the second joint region of the plurality of calibration local images, and taking the average gray scale value as the gray scale value of each pixel point in the second joint region.

4. The method of claim 1, wherein, the step of performing sub-regional pixel alignment on the overlapping regions of two adjacent local images, comprising: dividing the overlapping regions of two adjacent local images into regions according to a preset division rule; performing sub-regional pixel alignment by using a pre-stored homographic transformation matrix corresponding to each region.

5. The method of claim 1, wherein, before the step of acquiring a plurality of local images of a target panel captured by a multi-camera system, the method further comprises: acquiring the average gray scale values of different gray scale images of a standard panel captured by each camera of the multi-camera system under different exposure times; establishing a polynomial relationship between the average gray scale values and the exposure times; calculating the compensation coefficients of each camera of the multi-camera system according to the polynomial relationship.

6. The method according to any one of claims 1 to 5, characterized in that, the step of performing geometric distortion correction on each of the plurality of local images, comprising: for each camera in the multi-camera system, acquiring a plurality of pose images collected by a circular dot array calibration chart; identifying the image coordinates and world coordinates of each marker point in the circular dot array calibration chart; Based on Zhang Zhengyou calibration method, the camera intrinsic matrix and distortion coefficients are calculated by using the image coordinates and the world coordinates; According to the camera intrinsic matrix and distortion coefficients, distortion correction and pixel re-projection are performed on the local images. 7.A multi-camera based display panel image stitching apparatus, characterized in that, The display panel image stitching device for performing the method in any one of claims 1 to 6 comprises: An acquisition unit configured to acquire a plurality of local images of a target panel captured by a multi-camera system, wherein one camera corresponds to one local image; A correction unit configured to perform geometric distortion correction on the local images respectively; A pixel alignment unit configured to perform regional pixel alignment on the overlapping regions of two adjacent local images to determine a first seam region; A grayscale alignment unit configured to perform pixel grayscale information alignment on the first seam region and perform image fusion to obtain a target image.

8. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a program, and the program causes the computer to execute the method in any one of claims 1 to 6 when executed on the computer.

Images