A high-precision image stitching method and device and a storage medium

Abstract

The application discloses a high-precision image splicing method and device and a storage medium, and is used for improving the coordinate precision of panel repair. The method comprises the following steps: based on a left coordinate set and a right coordinate set, internal parameter matrices and external parameter matrices of a left-view camera and a right-view camera are respectively generated; any one of the left-view camera and the right-view camera is selected as a main camera, and the other one is a slave camera; a transformation matrix from the slave camera to the main camera is obtained in combination with the external parameter matrices of the two cameras; a pixel correspondence relationship between the slave camera and the main camera is calculated based on the transformation matrix, and a pixel mapping table is generated; a left-view to-be-spliced image and a right-view to-be-spliced image of the same to-be-detected panel are obtained by the main camera and the slave camera; the pixel mapping table is called to perform pixel remapping on the right-view to-be-spliced image, the right-view to-be-spliced image after remapping is fused with the left-view to-be-spliced image, and a complete image is output.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of image stitching, and in particular to a high-precision image stitching method, device and storage medium. BACKGROUND

[0002] Under the trend of continuous advancement of industrial production automation and intelligence, image stitching technology, as a key link in the field of computer vision, plays a crucial role in many business scenarios. For example, in the panel detection industry, as the size of the panel continues to increase and the detection accuracy requirement becomes increasingly higher, the image captured by a single camera often cannot cover the entire panel area, making it difficult to meet the demand for comprehensive and accurate detection. At this time, using multiple cameras to cooperatively capture and obtaining a complete panel image through image stitching technology has become a core means to ensure detection quality and efficiency. In this process, the image data captured by the left-view camera and the right-view camera, as well as the parameter information of the cameras, are crucial for achieving high-precision image stitching.

[0003] Currently, the common image stitching technology scheme is mainly based on the feature point matching method. The specific operation process is as follows: first, pre-process the images captured by the left and right cameras, and extract feature points such as corner points and edge points in the images; then describe the feature points by using a feature descriptor, and find corresponding feature point pairs in the two images by using a matching algorithm; then calculate the transformation relationship between the cameras according to these feature point pairs, and generate a pixel mapping table; finally, perform pixel remapping on one of the images by using the pixel mapping table, and fuse it with the other image to obtain the complete stitched image.

[0004] However, the existing image stitching technology scheme based on feature point matching is susceptible to image noise, illumination changes and occlusions, etc. in the process of feature point extraction and matching, resulting in inaccurate feature point matching, and further causing errors in the calculated camera transformation relationship, which ultimately affects the quality of the stitched image, resulting in misalignment, ghosting and other phenomena. In addition, for the circular feature region on the calibration board, the existing scheme only relies on simple area and circularity thresholds when screening the circular feature region, lacks comprehensive consideration of the integrity and consistency of the circular region, and is prone to cause the quality of the screened feature region to be uneven, affecting the accuracy of coordinate matching, thereby reducing the accuracy of the camera parameter matrix calculation, and further affecting the effect of image stitching. SUMMARY

[0005] The present application discloses a high-precision image stitching method, device and storage medium for improving the coordinate accuracy of panel repair.

[0006] The first aspect of the present application discloses a high-precision image stitching method, comprising:

[0007] The calibration board containing a plurality of circular regions is photographed by the left-view camera and the right-view camera respectively to obtain left-view circular region images and right-view circular region images;

[0008] Left-side circular feature regions are screened based on the left-view circular region images, and right-side circular feature regions are screened based on the right-view circular region images;

[0009] The center coordinates of the left-side circular feature regions are determined as left-side pixel coordinates, and the center coordinates of the right-side circular feature regions are determined as right-side pixel coordinates;

[0010] The left-side pixel coordinates are matched with the coordinates of the corresponding circles on the calibration board to obtain a left-side coordinate set, and the right-side pixel coordinates are matched with the coordinates of the corresponding circles on the calibration board to obtain a right-side coordinate set;

[0011] Based on the left-side coordinate set and the right-side coordinate set, internal parameter matrices and external parameter matrices of the left-view camera and the right-view camera are respectively generated;

[0012] Any one of the left-view camera and the right-view camera is selected as a master camera, and the other is selected as a slave camera, a transformation matrix from the slave camera to the master camera is obtained in combination with the external parameter matrices of the two cameras, and a coordinate system of the master camera is a stitching reference coordinate system;

[0013] The pixel correspondence relationship between the slave camera and the master camera is calculated based on the transformation matrix, and a pixel mapping table is generated;

[0014] A left-view to-be-stitched image and a right-view to-be-stitched image of a same to-be-detected panel photographed by the master camera and the slave camera are obtained;

[0015] The pixel mapping table is called to perform pixel remapping on the right-view to-be-stitched image, the right-view to-be-stitched image after remapping is fused with the left-view to-be-stitched image, and a complete image is output.

[0016] Optionally, the center coordinates of the left-side circular feature regions are determined as left-side pixel coordinates, and the center coordinates of the right-side circular feature regions are determined as right-side pixel coordinates; the left-side pixel coordinates are matched with the coordinates of the corresponding circles on the calibration board to obtain a left-side coordinate set, and the right-side pixel coordinates are matched with the coordinates of the corresponding circles on the calibration board to obtain a right-side coordinate set, including:

[0017] The left-side circular feature regions and the right-side circular feature regions are respectively sorted according to a rule of from top to bottom and from left to right, and the center coordinates of the regions after sorting are extracted as the left-side pixel coordinates and the right-side pixel coordinates;

[0018] Based on the pre-stored calibration board specification parameters, the left pixel coordinates and the right pixel coordinates are matched with the coordinates of the corresponding circles on the calibration board respectively to form the left coordinate set and the right coordinate set.

[0019] Optionally, after the internal parameter matrix and the external parameter matrix of the left-view camera and the right-view camera are generated based on the left coordinate set and the right coordinate set, any one of the left-view camera and the right-view camera is selected as a master camera and the other as a slave camera, and a transformation matrix of the slave camera to the master camera is obtained in combination with the external parameter matrices of the two cameras, before the coordinate system of the master camera is selected as a stitching reference coordinate system, the method further comprises:

[0020] The poses of the left circular region image and the right circular region image are adjusted to be consistent according to the internal parameter matrix and the external parameter matrix.

[0021] Optionally, the adjusting of the poses of the left circular region image and the right circular region image to be consistent according to the internal parameter matrix and the external parameter matrix comprises:

[0022] Based on the external parameter matrices of the left-view camera and the right-view camera, a rotation matrix and a translation vector of the left-view camera coordinate system and the right-view camera coordinate system relative to a calibration board world coordinate system are extracted respectively to determine an initial pose difference of the left circular region image and the right circular region image in a three-dimensional space.

[0023] Taking the pose of the left circular region image as a reference, a pose transformation parameter of the right circular region image to the left circular region image is calculated through the external parameter matrix, and the transformation parameter comprises a rotation compensation amount and a translation compensation amount.

[0024] Based on the pose transformation parameter, a coordinate conversion is performed on the right circular region image to convert the coordinates of all pixel points in the right circular region image from the right-view camera coordinate system to the left-view camera coordinate system, and the pose alignment is completed.

[0025] Optionally, the photographing of the calibration board containing a plurality of circular regions by the left-view camera and the right-view camera to obtain the left circular region image and the right circular region image comprises:

[0026] The calibration board containing a plurality of circular regions is photographed by the left-view camera and the right-view camera to obtain a left calibration board image and a right calibration board image.

[0027] The left calibration board image and the right calibration board image are subjected to binarization processing and connected domain segmentation respectively.

[0028] The binarization processing includes:

[0029] The left and right calibration plate images are respectively subjected to grayscale processing to obtain grayscale images.

[0030] The grayscale images are divided into non-overlapping sub-regions by using a local adaptive threshold algorithm, and a binarization threshold of the sub-regions is determined.

[0031] The grayscale images are converted into binary images based on the binarization threshold.

[0032] The connected domain segmentation includes:

[0033] Pixels of the binary images are traversed to mark all independent connected regions, and left and right circular region images are obtained by screening from the connected regions.

[0034] Optionally, any one of the left and right cameras is selected as a master camera, and the other is a slave camera, a transformation matrix from the slave camera to the master camera is obtained by combining the external parameter matrices of the two cameras, and the transformation matrix includes:

[0035] A formula for establishing the transformation relationship between the slave camera and the master camera is as follows:

[0036] The master camera is associated with the world coordinate system through a transformation relationship T_main: P_world = T_main * P_main.

[0037] The slave camera is associated with the world coordinate system through a transformation relationship T_slave: P_world = T_slave * P_slave.

[0038] By combining the two formulas, a transformation relationship between the slave camera coordinate system and the master camera coordinate system is obtained :

[0039] wherein, is a required homography matrix, also called a projection transformation matrix, is a coordinate point of a corresponding calibration plate on the master camera, is a coordinate point of a corresponding calibration plate on the slave camera.

[0040] Optionally, after the right-view to-be-stitched image is subjected to pixel remapping by using the pixel mapping table, the right-view to-be-stitched image after remapping and the left-view to-be-stitched image are fused to output a complete image, and the method further includes:

[0041] A multi-dimensional quality evaluation index system is constructed, which includes structural similarity, peak signal-to-noise ratio and average gray difference of pixels at a splicing seam.

[0042] The structural similarity value and the peak signal-to-noise ratio value of the complete image relative to the left side view image to be spliced and the right side view image to be spliced are calculated respectively, and the pixels at the splicing seam of the complete image are extracted to calculate the average gray difference of the pixels at the splicing seam.

[0043] According to the preset weights of each index, the structural similarity value and the peak signal-to-noise ratio value and the average gray difference are weighted and summed to obtain a splicing quality score.

[0044] If the splicing quality score is lower than a preset threshold, the steps after obtaining the left side view image to be spliced and the right side view image to be spliced are re-executed until the splicing quality score reaches the preset threshold.

[0045] The second aspect of the application provides a high-precision image splicing device, comprising:

[0046] A shooting unit is configured to shoot a calibration board comprising a plurality of circular regions through a left side view camera and a right side view camera respectively to obtain a left side circular region image and a right side circular region image.

[0047] A screening unit is configured to screen a left side circular feature region based on the left side circular region image and screen a right side circular feature region based on the right side circular region image.

[0048] A determination unit is configured to determine the center coordinates of the left side circular feature region as left side pixel coordinates and determine the center coordinates of the right side circular feature region as right side pixel coordinates.

[0049] A matching unit is configured to match the left side pixel coordinates with the coordinates of corresponding circles on the calibration board to obtain a left side coordinate set and match the right side pixel coordinates with the coordinates of corresponding circles on the calibration board to obtain a right side coordinate set.

[0050] A first generation unit is configured to generate internal parameter matrices and external parameter matrices of the left side view camera and the right side view camera respectively based on the left side coordinate set and the right side coordinate set.

[0051] A selection unit is configured to select any one of the left side view camera and the right side view camera as a master camera and the other as a slave camera, obtain a transformation matrix of the slave camera to the master camera in combination with the external parameter matrices of the two side cameras, and set the coordinate system of the master camera as a splicing reference coordinate system.

[0052] The second generation unit is configured to generate a pixel mapping table based on the pixel correspondence relationship between the slave camera and the master camera.

[0053] The acquisition unit is configured to acquire a left-view image and a right-view image of a same to-be-detected panel captured by the master camera and the slave camera.

[0054] The calling unit is configured to call the pixel mapping table to perform pixel remapping on the right-view image, and fuse the remapped right-view image with the left-view image to output a complete image.

[0055] The third aspect of the present application provides a high-precision image stitching device, comprising:

[0056] The processor, the memory, the input / output unit and the bus are connected.

[0057] The processor is connected with the memory, the input / output unit and the bus.

[0058] The memory stores a program, and the processor calls the program to execute the method of the first aspect and any optional method of the first aspect.

[0059] The fourth aspect of the present application provides a computer readable storage medium, and the computer readable storage medium stores a program, and the program is executed on a computer to execute the method of the first aspect and any optional method of the first aspect.

[0060] From the above technical solutions, the embodiments of the present application have the following advantages:

[0061] The present scheme first clearly selects the left and right circular feature regions based on the geometric features of the two-side circular region set. Compared with the single threshold judgment of the existing scheme, this comprehensive consideration of geometric features can more accurately eliminate incomplete or irregular circular regions caused by noise and occlusion, ensure that the selected circular feature regions have higher integrity and consistency, reduce coordinate matching errors from the source, and further improve the accuracy of camera parameter matrix calculation.

[0062] Then, by matching the gravity coordinates of the selected circular feature regions with the physical coordinates of the corresponding circular regions of the calibration board in the world coordinate system, left and right coordinate sets are formed, and the internal and external parameter matrices of the two cameras are calculated based on the coordinate sets, avoiding the problem of inaccurate feature point matching caused by image noise and light changes in the existing feature point matching scheme, making the calculation of the internal and external parameter matrices of the cameras more consistent with the actual shooting scene, improving the parameter precision, and fundamentally reducing the problems of misalignment and ghosting caused by transformation matrix errors.

[0063] Finally, when splicing the to-be-detected panel image, a high-precision pixel mapping table is called to perform pixel remapping on the right-view to-be-spliced image, and the right-view to-be-spliced image is fused with the left-view to-be-spliced image. The complete image after fusion effectively avoids splicing defects caused by inaccurate pixel correspondence. Meanwhile, the left-view camera is used as the main camera, and the coordinate system thereof is used as the splicing reference coordinate system, so that the stability and consistency of the splicing process are ensured. The complete panel image finally output not only has a coverage range meeting the detection requirement, but also has higher splicing precision and clarity, and can better adapt to the demand of the panel detection industry for comprehensive and accurate detection of large-size panels, and guarantee the accuracy and efficiency of subsequent detection tasks. BRIEF DESCRIPTION OF DRAWINGS

[0064] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0065] Figure 1 An embodiment flowchart of a high-precision image splicing method provided by the present application is shown in the figure.

[0066] Figure 2 An embodiment flowchart of obtaining a coordinate set in the high-precision image splicing method provided by the present application is shown in the figure.

[0067] Figure 3 An embodiment flowchart of adjusting the pose in the high-precision image splicing method provided by the present application is shown in the figure.

[0068] Figure 4 Another embodiment flowchart of adjusting the pose in the high-precision image splicing method provided by the present application is shown in the figure.

[0069] Figure 5 An embodiment flowchart of obtaining a two-side circular region set in the high-precision image splicing method provided by the present application is shown in the figure.

[0070] Figure 6 An embodiment flowchart of performing splicing quality evaluation on the complete image output in the high-precision image splicing method provided by the present application is shown in the figure.

[0071] Figure 7 An embodiment structure diagram of a high-precision image splicing device provided by the present application is shown in the figure.

[0072] Figure 8 Another embodiment structure diagram of a high-precision image splicing device provided by the present application is shown in the figure.

[0073] Figure 9 Fig. 16 is a schematic diagram of the left and right calibration points extracted in the present application;

[0074] Figure 10 Fig. 17 is a schematic diagram of the left and right calibration points after adjustment of posture extracted in the present application;

[0075] Figure 11 Fig. 18 is a schematic diagram of the stitching result of the calibration points in the present application;

[0076] Figure 12 Fig. 19 is a schematic diagram of the left and right camera taking screens of the present application;

[0077] Figure 13 Fig. 20 is a schematic diagram of the stitching result of the screens in the present application. DETAILED DESCRIPTION

[0078] In the following description, for purposes of explanation and not limitation, specific details are set forth, such as particular sequences of steps, techniques, etc., in order to provide a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application can be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known methods, devices, circuits, and

[0079] It will be understood that the term "includes," "including," "has," "having," "comprises," "comprising" or "contains," "containing" when used in the specification and the appended claims, means the presence of the stated features, integers, steps, operations, elements, or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, or groups thereof.

[0080] It will also be understood that the term "and / or," when used in the specification and the appended claims, means any one and / or all possible combinations of one or more of the associated listed items.

[0081] As used in the description of the application and the appended claims, the term "if" can be interpreted to mean "when" or "upon" or "in response to determining" or "in response to detecting" depending on the context. Similarly, the phrase "if it is determined" or "if [a described condition or event] is detected" can be interpreted to mean "upon determining" or "in response to determining" or "upon [the described condition or event] being detected" or "in response to [the described condition or event] being detected," depending on the context.

[0082] In addition, the terms "first," "second," "third," etc. are used herein only to describe different instances of elements, and are not intended to imply or suggest relative importance of the elements.

[0083] Reference throughout this application to "one embodiment" or "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the application. The appearances of the phrase "in one embodiment" or "in some embodiments" in various places in the specification are not necessarily all referring to the same embodiment, however, but can refer to one or more but not all embodiments. The terms "including," "comprising," "featuring," and variations thereof are meant to encompass the items listed thereafter and equivalents thereof as well as additional items. Although the application has been described with reference to specific embodiments, it will be apparent to those skilled in the art that various modifications and changes can be made thereto without departing from the scope of the application as set forth in the claims below. Accordingly, the aim of the application is to cover all such modifications and changes as fall within the scope of the application, including full equivalents and equivalents of the operational concepts underlying the application.

[0084] Based on this, the application discloses a high-precision image splicing method and device and a storage medium, which are used for improving the coordinate precision of panel repair.

[0085] The method of the application can be applied to a server, a device, a terminal or other devices with logical processing capability, and the application is not limited in this regard. For the convenience of description, the following will be described by taking the execution subject as a system as an example.

[0086] Please refer to Figure 1 The application provides one embodiment of a high-precision image splicing method, which comprises the following steps.

[0087] 101. A calibration board containing a plurality of circular regions is photographed by a left-view camera and a right-view camera respectively to obtain left and right circular region images;

[0088] 102. Left circular feature regions are screened out based on the left circular region image, and right circular feature regions are screened out based on the right circular region image;

[0089] 103. The center coordinates of the left circular feature regions are determined as left pixel coordinates, and the center coordinates of the right circular feature regions are determined as right pixel coordinates;

[0090] 104. The left pixel coordinates are matched with the coordinates of the corresponding circles on the calibration board to obtain a left coordinate set, and the right pixel coordinates are matched with the coordinates of the corresponding circles on the calibration board to obtain a right coordinate set;

[0091] 105. The internal parameter matrix and the external parameter matrix of the left-view camera and the right-view camera are respectively generated based on the left coordinate set and the right coordinate set;

[0092] Please continue to refer to Figure 9, step 101 first uses the left side view camera and the right side view camera to take a picture of the same calibration board, obtaining a left side circular region image and a right side circular region image. Among them, the calibration board is a pre-designed reference board, and the calibration board contains a plurality of regularly arranged circular regions (such as equidistantly arranged black circles, and the background is white), the number, size and arrangement of the circular regions are known parameters.

[0093] Step 102 filters based on the geometric characteristics of each region in the two-side circular region set, which includes at least area control parameters and circularity characteristics. The area control parameter is obtained by taking the corresponding circular area threshold value as the average value of the corresponding total region area before 10% multiplied by 0.8.

[0094] The circularity characteristic is an index for measuring the proximity of the region to an ideal circle, and the calculation formula is:

[0095] The circularity = 4π×region area / (region perimeter)², and the circularity of an ideal circle is 1. Set the circularity threshold value, and keep the regions whose circularity meets the threshold requirement.

[0096] Through the above screening, the left side circular feature region is obtained from the left side circular region set, and the right side circular feature region is obtained from the right side circular region set, so as to ensure that the feature points in the subsequent processing are real circles on the calibration board.

[0097] Steps 103 and 104 extract the center coordinates of the left side circular feature region and the right side circular feature region: by calculating the geometric center of each circular feature region, it is determined as the left side pixel coordinate and the right side pixel coordinate.

[0098] Based on the preset parameters of the calibration board (such as the arrangement number of the circular regions, the center distance between adjacent circular regions, the position of the world coordinate system origin, etc.), the physical coordinates of each circular region in the world coordinate system are obtained. According to the arrangement order of the circular regions on the calibration board, the left side pixel coordinates are matched with the world coordinate system physical coordinates of the corresponding circular regions one by one to form a left side coordinate set; similarly, the right side pixel coordinates are matched with the world coordinate system physical coordinates of the corresponding circular regions to form a right side coordinate set. Each coordinate set contains a plurality of "pixel coordinate-physical coordinate" mapping pairs.

[0099] Step 105 calculates the internal parameter matrix and the external parameter matrix of the left-view camera and the right-view camera respectively by Zhang Zhengyou calibration algorithm based on the left coordinate set and the right coordinate set. The mapping relationship between the left pixel coordinates and the corresponding physical coordinates in the world coordinate system is substituted into the camera perspective projection model to solve the internal parameters (including focal length, principal point coordinates, distortion coefficients) and the external parameters (rotation matrix, translation vector) of the left-view camera. Similarly, the same calculation is performed on the right coordinate set to obtain the internal parameter matrix and the external parameter matrix of the right-view camera, so as to quantify the imaging characteristics of the two cameras and the attitude relationship with the world coordinate system.

[0100] 106、Select any one of the left-view camera and the right-view camera as the master camera, and the other as the slave camera, obtain the transformation matrix from the slave camera to the master camera based on the external parameter matrices of the two cameras, and the coordinate system of the master camera is the stitching reference coordinate system;

[0101] 107、Calculate the pixel correspondence relationship between the slave camera and the master camera based on the transformation matrix, and generate a pixel mapping table;

[0102] 108、Obtain the left-view image to be stitched and the right-view image to be stitched of the same panel to be detected photographed by the master camera and the slave camera;

[0103] 109、Call the pixel mapping table to perform pixel remapping on the right-view image to be stitched, fuse the remapped right-view image to be stitched with the left-view image to be stitched, and output a complete image.

[0104] Please continue to refer to Figure 11 , in step 106, any one of the left-view camera and the right-view camera is selected as the master camera, and the other is selected as the slave camera, and the coordinate system of the master camera is set as the stitching reference. The transformation matrix from the slave camera to the master camera is derived based on the external parameter matrices of the two cameras. Specifically, the attitude relationship of the master camera with the world coordinate system and the attitude relationship of the slave camera with the world coordinate system are used to jointly calculate the transformation matrix that can convert the coordinates in the slave camera coordinate system to the master camera coordinate system, thereby providing a basis for subsequent pixel correspondence relationship calculation.

[0105] The formula for deriving the transformation matrix from the slave camera to the master camera is as follows:

[0106] The master camera is associated with the world coordinate system through the transformation relationship T_main: P_world = T_main * P_main;

[0107] The slave camera is associated with the world coordinate system through the transformation relationship T_slave: P_world = T_slave * P_slave;

[0108] By combining the two formulas, the transformation relationship from the slave camera coordinate system to the master camera coordinate system can be obtained :

[0109] ;

[0110] wherein, is the desired homography matrix, also called the projection transformation matrix, is the coordinate point extracted on the main camera corresponding to the calibration board, is the coordinate point extracted on the slave camera corresponding to the calibration board.

[0111] Step 107 calculates the pixel correspondence relationship from the camera to the main camera based on the transformation matrix obtained in step 106 and generates a pixel mapping table. By traversing each pixel coordinate of the camera image, the corresponding coordinate of the pixel in the main camera image is obtained through coordinate conversion by the transformation matrix. For the non-integer coordinates after conversion, an interpolation algorithm is used to determine the pixel value, and finally the corresponding relationship of all pixels is stored as a pixel mapping table, which directly records the mapping rule from the camera pixel to the main camera pixel.

[0112] In step 108, using the main camera and the slave camera that have completed parameter calibration, the same to-be-detected panel is synchronously photographed to obtain a left-view to-be-stitched image and a right-view to-be-stitched image. When photographing, it is necessary to ensure that the to-be-detected panel is completely within the field of view of the two cameras, and there is a certain overlapping area between the two images, so as to ensure the continuity of the edge information during stitching. The shooting environment needs to be consistent with the calibration stage to reduce the influence of external factors on the stitching accuracy.

[0113] Please continue to refer to Figure 12 and Figure 13 , step 109 calls the pixel mapping table generated in step 107 to perform pixel remapping on the right-view to-be-stitched image, and converts all pixels of the right-view to-be-stitched image to the main camera coordinate system according to the mapping relationship, so as to realize the spatial alignment with the left-view to-be-stitched image. Then, the remapped right image and the left image are fused, the pixel values in the overlapping area are adjusted by using the weighted average method, the pixel weight becomes smaller as the distance to the stitching edge becomes closer, and the edge is smoothed to eliminate the stitching traces, and finally a high-quality stitched image containing complete information of the to-be-detected panel is output.

[0114] In this embodiment, first, the left and right circular feature regions are screened based on the geometric features of the two sets of circular regions. Compared with the single threshold judgment in the existing scheme, this comprehensive consideration of geometric features can more accurately eliminate incomplete or irregular circular regions caused by noise and occlusion, ensure that the screened circular feature regions have higher integrity and consistency, reduce coordinate matching errors from the source, and thus improve the accuracy of the camera parameter matrix calculation.

[0115] Then, by matching the center coordinates of the screened circular feature areas with the physical coordinates of the corresponding circles on the calibration plate in the world coordinate system, left and right coordinate sets are formed, and the internal and external parameter matrices of the two cameras are calculated based on the coordinate sets, avoiding the problem of inaccurate feature point matching caused by image noise and changes in illumination in the existing feature point matching scheme, making the calculation of the internal and external parameter matrices of the cameras more in line with the actual shooting scene, improving the parameter accuracy, and fundamentally reducing the problems of misalignment and ghosting caused by transformation matrix errors.

[0116] Finally, when stitching the images of the panel to be detected, a high-precision pixel mapping table is called to perform pixel remapping on the right-view image to be stitched, and the remapped image is fused with the left-view image to be stitched. The complete image after fusion effectively avoids stitching defects caused by inaccurate pixel correspondence. At the same time, the left-view camera is used as the main camera, and its coordinate system is used as the stitching reference coordinate system, ensuring the stability and consistency of the stitching process. The complete panel image output finally not only covers the detection requirements, but also has higher stitching accuracy and clarity, better adapting to the demand of the panel detection industry for comprehensive and accurate detection of large-size panels, and ensuring the accuracy and efficiency of subsequent detection tasks.

[0117] Please refer to Figure 2 An embodiment of a method for obtaining a coordinate set is provided, comprising:

[0118] 201. Sort the left circular feature areas and the right circular feature areas according to the rule of "from top to bottom, from left to right", and extract the center coordinates of each area after sorting as the left pixel coordinates and the right pixel coordinates;

[0119] 202. Match the left pixel coordinates and the right pixel coordinates with the coordinates of the corresponding circles on the calibration plate based on the pre-stored calibration plate specification parameters, to form left and right coordinate sets.

[0120] Step 201 is used to determine the left pixel coordinates and the right pixel coordinates, providing regular feature point data for subsequent coordinate matching. Specifically, first, sorting operations are performed on the left circular feature areas and the right circular feature areas respectively, with the sorting rule set as "from top to bottom, from left to right". First, sort the center coordinates of each circular feature area according to the y-axis value (vertical position) from small to large, achieving the order of "from top to bottom". Then, for areas with the same y-axis value, sort the center coordinates according to the x-axis value (horizontal position) from small to large, completing the order of "from left to right". After sorting, extract the center coordinates of each area as the left pixel coordinates of the left circular area image and the right pixel coordinates of the right circular area image, ensuring that the left and right pixel coordinates are consistent with the actual arrangement order of the circles on the calibration plate.

[0121] Step 202 establishes the corresponding relationship between the pixel coordinates and the physical coordinates based on the normalized pixel coordinates and the known calibration board information. First, the pre-stored calibration board specification parameters are called, which include the number of rows and columns of the circular arrangement on the calibration board, the center distance between adjacent circles, the position of the origin of the world coordinate system, etc. According to this, the physical coordinates of each circle in the world coordinate system can be calculated. Then, the left coordinate set and the right coordinate set are formed. The specific formula is as follows:

[0122] According to the pre-stored calibration board specification (such as grid size, number of rows and columns, and origin position), the system automatically assigns a corresponding physical coordinate in the world coordinate system to each feature point identified in the image, denoted as 、 、……、 This process does not require manual intervention and manual input.

[0123] The center coordinates on the calibration board are relative to the actual physical size coordinates corresponding to the calibration:

[0124] ;

[0125] is the vertical serial number of the corresponding circle center arranged from top to bottom and left to right, is the row serial number of the corresponding circle center arranged from top to bottom and left to right, is the actual physical distance between the circle centers.

[0126] In this embodiment, the left and right circular feature regions are sorted according to the pre-set rules, so that the center coordinates extracted from both sides strictly correspond to the actual arrangement order of the circles on the calibration board, avoiding misalignment of the subsequent coordinate matching due to the disorder of the feature points, laying a foundation for high-precision coordinate association; based on the pre-stored calibration board specification parameters, the pixel coordinates are automatically matched with the physical coordinates of the calibration board, without the need for manual input or labeling of feature point coordinates, eliminating human operation errors, improving matching consistency, reducing dependence on the technical level of the operator, and simplifying the calibration process.

[0127] Please refer to Figure 3 , an embodiment of the method for adjusting the pose is provided, comprising:

[0128] 301. Adjusting the poses of the left circular region image and the right circular region image to be consistent according to the internal parameter matrix and the external parameter matrix.

[0129] Please continue to refer to Figure 10 , Figure 10To extract the adjusted calibration points from both cameras, the rotation and translation information of both cameras relative to the world coordinate system is extracted using the intrinsic and extrinsic parameter matrices of the left and right view cameras, thus determining the image pose differences. Using the pose of one image as a reference, the other image is rotated and translated through coordinate transformation to align the spatial positions of corresponding circular features in both images, ensuring pose consistency. The formula for adjusting the pose is as follows:

[0130] The pose of the left and right circular regions of the image is adjusted by calculating the corresponding homography matrix H.

[0131] ;

[0132] in, Reverse the target perspective to the source perspective. This is the inverse matrix of the camera's intrinsic parameters.

[0133] In this embodiment, the poses of the images on both sides are made consistent by adjustment, which solves the problem of image tilting and misalignment caused by differences in camera installation angle and shooting position, and avoids the impact of pose deviation on the accuracy of subsequent coordinate matching and camera parameter calculation.

[0134] Please see Figure 4 , Figure 4 An embodiment of a posture adjustment method refined in the third embodiment is provided, including:

[0135] 401. Based on the external parameter matrices of the left and right view cameras, extract the rotation matrix and translation vector of the left and right view camera coordinate systems relative to the calibration board world coordinate system, respectively, and determine the initial pose difference between the left and right circular region images in three-dimensional space.

[0136] 402. Based on the pose of the left circular region image, calculate the pose transformation parameters from the right circular region image to the left circular region image through the external parameter matrix. The transformation parameters include rotation compensation and translation compensation.

[0137] 403. Based on the attitude transformation parameters, perform coordinate transformation on the right circular region image, transforming the coordinates of all pixels in the right circular region image from the right view camera coordinate system to the left view camera coordinate system to complete the attitude alignment.

[0138] Step 401 is used to determine the initial pose difference of the two images in the three-dimensional space, which provides data basis for subsequent pose adjustment. Specifically, from the external parameter matrix of the left and right cameras, the corresponding rotation matrix and translation vector are extracted, respectively. The rotation matrix reflects the rotation direction of the camera coordinate system relative to the world coordinate system of the calibration board, and the translation vector represents the position offset of the camera coordinate system relative to the world coordinate system. By comparing the rotation matrix and translation vector of the two sides, the initial pose difference of the left and right circular region images in the three-dimensional space can be quantitatively determined.

[0139] Step 402 takes the left circular region image as the reference to calculate the pose transformation parameters required for the right circular region image to align. Based on the external parameters of the two cameras extracted in step 401, the pose transformation relationship from the right circular region image to the left circular region image is derived through matrix operation: where the rotation compensation is calculated by the inverse operation and multiplication of the rotation matrix of the two sides, which is used to offset the rotation deviation of the right image relative to the left image; the translation compensation is solved by combining the difference of the translation vectors of the two sides with the inverse operation of the rotation matrix, which is used to correct the position offset of the right image relative to the left image, and finally the complete pose transformation parameters including rotation compensation and translation compensation are obtained.

[0140] Step 403 realizes the pose alignment of the two images, providing a unified spatial reference for subsequent processing. According to the pose transformation parameters obtained in step 402, the coordinates of all pixel points in the right circular region image are converted, and the coordinates of each pixel point in the right camera coordinate system are mapped to the left camera coordinate system through the operation of rotation compensation and translation compensation. After the coordinate conversion is completed, the pose of the right circular region image is consistent with that of the left circular region image, and the spatial positions of the corresponding circular features in the two images are accurately aligned, eliminating the image misalignment problem caused by the pose difference of the cameras, and completing the pose alignment.

[0141] In this embodiment, the initial pose difference is quantified by extracting the rotation matrix and translation vector, avoiding the subjectivity and inaccuracy of judging the image pose deviation based on visual judgment alone, providing accurate and quantifiable basis for subsequent pose adjustment, reducing the subsequent adjustment deviation caused by misjudgment of the pose difference from the source, and ensuring the basic accuracy of the pose alignment. The specific rotation and translation compensation is calculated based on the left image, rather than using a general adjustment scheme, ensuring that the compensation parameters accurately match the actual pose difference of the two images, avoiding over-adjustment or insufficient adjustment, and improving the efficiency and accuracy of aligning the right image pose to the reference. The pose alignment is achieved through full-pixel coordinate conversion, ensuring that each pixel of the right image is accurately mapped to the left camera coordinate system, completely eliminating the local displacement and deformation of the image caused by the pose misalignment, providing a unified and accurate spatial reference for subsequent camera parameter optimization and image stitching, and indirectly ensuring the seamlessness and high precision of the final stitched image.

[0142] Referring to Figure 5 The application provides an embodiment of a method for obtaining a two-side circular region set, comprising:

[0143] 501, capturing a calibration board containing a plurality of circular regions by a left-view camera and a right-view camera to obtain a left calibration board image and a right calibration board image;

[0144] 502, performing binaryzation processing and connected domain segmentation on the left calibration board image and the right calibration board image respectively;

[0145] 503, performing grayscale processing on the left calibration board image and the right calibration board image respectively to obtain a grayscale image;

[0146] 504, dividing the grayscale image into a plurality of non-overlapping sub-regions by using a local adaptive threshold algorithm, and determining a binaryzation threshold value of each sub-region;

[0147] 505, converting the grayscale image into a binary image based on the binaryzation threshold value;

[0148] 506, traversing the pixels of the binary image, marking all independent connected regions, and screening a two-side circular region set from the connected regions.

[0149] Steps 501 and 502 preliminarily extract potential circular regions from the original images by performing binaryzation processing and connected domain segmentation on the obtained left calibration board image and right calibration board image. The binaryzation processing is used to distinguish the target regions (circular regions on the calibration board) from the background in the image, and the connected domain segmentation is used to mark the target regions after binaryzation as independent units.

[0150] Step 503 is a preliminary preparation for binaryzation processing, and performs grayscale processing on the left calibration board image and the right calibration board image respectively. By converting the color image (if it is a color image) into a grayscale image, the luminance information of the pixels is retained and the color interference is removed, so that the image is represented only by the grayscale values of 0~255, the complexity of the subsequent threshold calculation is simplified, and the stability of the binaryzation processing is ensured.

[0151] Step 504 uses a local adaptive threshold algorithm to improve the pertinence of the threshold value. The grayscale image is divided into a plurality of non-overlapping sub-regions, the mean value and the standard deviation of the pixel grayscale of each sub-region are calculated, and the binaryzation threshold value of the sub-region is determined according to the formula “threshold value=mean value-k×standard deviation” (wherein k is a preset coefficient, and the value range is 1.0~2.0).

[0152] Step 505 converts the gray-scale image into a binary image based on the sub-region threshold value determined in step 504. For each sub-region, pixels with a gray-scale value higher than the threshold value are set as background, and pixels with a gray-scale value lower than the threshold value are set as target, so that the circular regions on the calibration board are presented in the form of continuous black blocks, which are clearly distinguished from the background.

[0153] Step 506 extracts potential circular regions through connected component segmentation. All pixels of the binary image are traversed, and the neighborhood labeling method is used to label mutually connected black pixels (target pixels) as the same connected region, and each labeled region is regarded as an independent potential target. Subsequently, small noise regions are filtered out according to the basic characteristics (such as the area range) of the regions, and finally the left circular region set and the right circular region set are obtained.

[0154] The embodiment forms a set of coherent image preprocessing procedures. Firstly, the left and right view images are subjected to gray-scale processing to remove color interference and simplify subsequent operations. Secondly, a local adaptive threshold algorithm is used to dynamically determine the sub-region threshold value, which effectively adapts to local light differences and avoids the problem of target loss or excessive noise caused by global threshold value. Thirdly, a binary image with clear distinction between target and background is obtained based on threshold conversion. Finally, the left and right circular region sets are extracted through connected component labeling and filtering. The whole procedure can not only efficiently eliminate external interference and accurately separate the circular features of the calibration board, but also provide high-quality candidate regions for subsequent circular feature screening, coordinate matching and camera parameter calculation, thereby ensuring the accuracy and stability of subsequent image stitching from the source, and improving the automation and reliability of the preprocessing process without complex manual intervention.

[0155] Referring to Figure 6 The application provides an embodiment of a method for evaluating the splicing quality of an output complete image, which comprises the following steps:

[0156] 601. A multi-dimensional quality evaluation index system is constructed, which comprises structural similarity, peak signal-to-noise ratio and average gray-scale difference of pixels at the splicing seam;

[0157] 602. The structural similarity value and the peak signal-to-noise ratio value of the complete image relative to the left-view image to be spliced and the right-view image to be spliced are calculated respectively, and the pixels at the splicing seam of the complete image are extracted to calculate the average gray-scale difference of the pixels at the splicing seam;

[0158] 603. The structural similarity value, the peak signal-to-noise ratio value and the average gray-scale difference are weighted and summed according to the preset weights of the respective indicators to obtain a splicing quality score;

[0159] 604. If the splicing quality score is lower than a preset threshold value, the steps after the left-view image to be spliced and the right-view image to be spliced are re-executed until the splicing quality score reaches the preset threshold value.

[0160] The core goal of step 601 is to build a comprehensive and scientific stitching quality evaluation index system, breaking the limitations of single index evaluation and achieving multi-dimensional coverage of the stitching effect. Specifically, the system designs indicators around the three core dimensions of "structural consistency", "fidelity" and "edge smoothness":

[0161] Structural similarity (SSIM): Focus on the complete image after stitching and the left and right view images to be stitched. By comparing the brightness, contrast and structural information of the images, it quantitatively judges whether the stitching process has destroyed the texture, contour and other structural features of the left and right view images to be stitched, avoiding deformation of the target area due to stitching;

[0162] Peak signal-to-noise ratio (PSNR): Taking image noise level as the core of evaluation, it calculates the mean square error between the complete image and the original image to be stitched, reflecting whether additional noise is introduced during the stitching process or whether the image clarity is reduced due to coordinate conversion and fusion operations, ensuring the fidelity of the stitched image;

[0163] Average pixel gray difference at the stitching seam: Specifically designed for the visual effect of the stitching edge, it extracts the gray values of the adjacent pixels on both sides of the stitching seam and calculates the average difference to directly measure the degree of visibility of the stitching seam - the smaller the difference, the more natural the transition of the stitching edge, avoiding visible "line-shaped" stitching marks.

[0164] The three are complementary to each other, covering both the overall quality of the image and the local stitching details.

[0165] Step 602 is the data collection stage of quality evaluation, which obtains the actual values of the indicators set in step 601 through specific calculations, providing objective data support for subsequent scoring. The specific operation is divided into two parts: the first part, calculate the structural similarity value and the peak signal-to-noise ratio value. Take the left and right view images to be stitched as reference images and the complete image after stitching as the image to be evaluated, and calculate:

[0166] When calculating the structural similarity value, traverse the image through a sliding window, compare the brightness average, contrast standard deviation and structural correlation of the reference image and the image to be evaluated block by block, and finally take the average value of the whole image as the final structural similarity value (value range 0-1, the closer to 1, the better the structural consistency);

[0167] When calculating the peak signal-to-noise ratio, first determine the maximum pixel value of the image, then calculate the mean square error of the reference image and the image to be evaluated, and through the formula: Get the peak signal-to-noise ratio value.

[0168] Where n is the number of pixels, unit dB, the higher the value, the better the fidelity and the less the noise.

[0169] Second part, calculate the average of the pixel gray difference at the splicing seam. First, determine the specific position of the splicing seam in the complete image through the previous splicing logic, then select a certain width of the area along the splicing seam, extract the gray values of all adjacent pixel pairs in the area, calculate the absolute value of the gray difference of each pair of pixels, and finally take the average of all the difference values to get the average of the gray difference (the smaller the value, the less obvious the splicing seam).

[0170] Step 603 is to convert the multi-dimensional index value obtained in step 602 into a single and intuitive splicing quality score, realizing the quantitative judgment of the splicing effect. Specifically, first, standardize the index. Due to the differences in the value range and evaluation direction of different indexes, the index values are first converted to a unified evaluation dimension.

[0171] Second, set the index weight. According to the needs of the actual application scene, different weights are assigned to the three indexes. For example, if the focus is on "structural consistency" and "fidelity", the structural similarity value weight is set to 0.4, the peak signal-to-noise ratio value weight is set to 0.4, and the gray difference average weight is set to 0.2. If the visual effect of the splicing seam is required to be extremely high, the gray difference average weight can be appropriately increased to ensure that the total weight is 1.

[0172] Finally, calculate the weighted score. Multiply the standardized index values by the corresponding weights, and then add the products to get the final splicing quality score.

[0173] Step 604 is the closed-loop optimization link of splicing quality, which determines the quality threshold. According to the accuracy standard of the actual application, the minimum threshold of the splicing quality score is preset, and then the score comparison and judgment are performed.

[0174] If the score is greater than or equal to the threshold, it means that the splicing effect meets the standard (such as no deformation of the structure, no obvious splicing seam, and low noise), and the complete image is directly output, and the process is ended.

[0175] If the score is less than the threshold, the reason for not meeting the standard needs to be analyzed (such as low score due to high gray difference average, which may be due to improper fusion algorithm parameters; low peak signal-to-noise ratio value, which may be due to large interpolation error during remapping), and then the re-execution mechanism is triggered, that is, all steps after "obtaining the left and right images to be spliced" are re-executed, including: calling the pixel mapping table for pixel remapping (rechecking whether the mapping table has errors), adjusting the image fusion algorithm parameters (such as increasing the width of the transition area for weighted fusion), recalculating the complete image after fusion, and then executing the evaluation process of steps 602-603 again.

[0176] In this embodiment, by constructing a multi-dimensional evaluation system, quantifying the indicators, weighting the scores and iterating the optimization, a complete splicing quality guarantee closed loop is formed, and the comprehensive benefits are remarkable. The multi-dimensional index system covers structural consistency, fidelity and splicing seam smoothness, avoids missing defects of single index, replaces subjective evaluation, and ensures objective and unified judgment; quantitative calculation provides accurate data support for scoring, takes into account the matching degree of the original images on both sides, and accurately locates the splicing problems; standardized weighted scoring converts complex indicators into intuitive scores, adapts to different scene weight requirements, and simplifies the evaluation process; the iterative optimization mechanism triggers re-execution through threshold judgment, forces correction of substandard splicing, and ensures that the final output image meets the requirements in structure, fidelity and visual fluency, improving the reliability and practicality of the overall splicing process.

[0177] Please refer to Figure 7 An embodiment of a high-precision image splicing device is provided, comprising:

[0178] The shooting unit 701 is configured to shoot a calibration board containing a plurality of circular regions through the left-view camera and the right-view camera respectively to obtain left and right circular region images.

[0179] Optionally, the shooting unit 701 is further configured to:

[0180] The shooting unit 701 is configured to shoot the calibration board containing a plurality of circular regions through the left-view camera and the right-view camera to obtain left and right calibration board images.

[0181] The left and right calibration board images are respectively subjected to binaryzation processing and connected domain segmentation.

[0182] The binaryzation processing comprises:

[0183] The left and right calibration board images are respectively subjected to grayscale processing to obtain grayscale images.

[0184] The grayscale images are divided into non-overlapping sub-regions by using a local adaptive threshold algorithm, and the binaryzation threshold of the sub-regions is determined.

[0185] The grayscale images are converted into binary images based on the binaryzation threshold.

[0186] The connected domain segmentation comprises:

[0187] All independent connected regions are marked by traversing the pixels of the binary image, and the left and right circular region images are obtained by screening the connected regions.

[0188] The screening unit 702 is configured to screen left and right circular feature regions based on the left and right circular region images respectively.

[0189] The determination unit 703 is configured to determine the center coordinates of the left circular feature region as the left pixel coordinates and the center coordinates of the right circular feature region as the right pixel coordinates.

[0190] The matching unit 704 is configured to match the left pixel coordinates with the coordinates of the corresponding circles on the calibration board to obtain the left coordinate set, and match the right pixel coordinates with the coordinates of the corresponding circles on the calibration board to obtain the right coordinate set.

[0191] Optionally, the determination unit 703 and the matching unit 704 are further configured to:

[0192] sort the left circular feature region and the right circular feature region according to the rule of from top to bottom and from left to right respectively, and extract the center coordinates of each region after the sorting as the left pixel coordinates and the right pixel coordinates;

[0193] match the left pixel coordinates and the right pixel coordinates with the coordinates of the corresponding circles on the calibration board based on the pre-stored calibration board specification parameters, to form the left coordinate set and the right coordinate set.

[0194] The first generation unit 705 is configured to generate the internal parameter matrix and the external parameter matrix of the left-view camera and the right-view camera respectively based on the left coordinate set and the right coordinate set.

[0195] Optionally, the method further includes an adjustment unit 706 configured to:

[0196] adjust the poses of the left circular region image and the right circular region image to be consistent according to the internal parameter matrix and the external parameter matrix.

[0197] Optionally, the adjustment unit 706 is further configured to:

[0198] extract the rotation matrix and the translation vector of the coordinate system of the left-view camera and the right-view camera relative to the world coordinate system of the calibration board based on the external parameter matrix of the left-view camera and the right-view camera, to determine the initial pose difference of the two images in the three-dimensional space;

[0199] take the pose of the left circular region image as a reference, calculate the pose transformation parameters of the right circular region image to the left circular region image through the external parameter matrix, and the transformation parameters include the rotation compensation and the translation compensation;

[0200] perform coordinate conversion on the right circular region image based on the pose transformation parameters, to convert the coordinates of all pixel points in the right circular region image from the coordinate system of the right-view camera to the coordinate system of the left-view camera, and complete the pose alignment.

[0201] The selecting unit 707 is configured to select any one of the left-view camera and the right-view camera as a master camera and the other as a slave camera, obtain a transformation matrix from the slave camera to the master camera in combination with external parameter matrices of the two cameras, and set a coordinate system of the master camera as a stitching reference coordinate system;

[0202] Optionally, the selecting unit 707 is further configured to:

[0203] A formula for establishing the transformation relationship between the slave camera and the master camera is as follows:

[0204] The master camera is associated with the world coordinate system through the transformation relationship T_main: P_world = T_main * P_main.

[0205] The slave camera is associated with the world coordinate system through the transformation relationship T_slave: P_world = T_slave * P_slave.

[0206] By combining the two formulas, a transformation relationship from the slave camera coordinate system to the master camera coordinate system can be obtained as follows:

[0207] wherein, is a required homography matrix, also referred to as a projection transformation matrix, is a coordinate point extracted on the corresponding calibration board of the master camera, is a coordinate point extracted on the corresponding calibration board of the slave camera.

[0208] The second generating unit 708 is configured to calculate a pixel correspondence relationship between the slave camera and the master camera based on the transformation matrix, and generate a pixel mapping table.

[0209] The acquiring unit 709 is configured to acquire a left-view to-be-stitched image and a right-view to-be-stitched image of a same to-be-detected panel captured by the master camera and the slave camera.

[0210] The calling unit 710 is configured to call the pixel mapping table to perform pixel remapping on the right-view to-be-stitched image, fuse the remapped right-view to-be-stitched image and the left-view to-be-stitched image, and output a complete image.

[0211] Optionally, the method further includes a constructing unit 711 configured to:

[0212] construct a multi-dimensional quality evaluation index system including structural similarity, peak signal-to-noise ratio, and average pixel gray difference at a stitching seam.

[0213] Optionally, the method further includes a calculating unit 712 configured to:

[0214] ​The structural similarity value and the peak signal-to-noise ratio value of the complete image relative to the left-view image to be spliced and the right-view image to be spliced are calculated respectively, and the pixel at the splicing joint of the complete image is extracted, and the average gray difference of the pixel at the splicing joint is calculated.

[0215] Optionally, the sum unit 713 is further included, and the sum unit 713 is configured to:

[0216] According to the preset index weight, the structural similarity value, the peak signal-to-noise ratio value and the average gray difference are weighted and summed to obtain the splicing quality score.

[0217] Optionally, the loop unit 714 is further included, and the loop unit 714 is configured to:

[0218] If the splicing quality score is lower than the preset threshold, the steps after the left-view image to be spliced and the right-view image to be spliced are re-executed until the splicing quality score reaches the preset threshold.

[0219] The specific implementation process refers to the embodiments, which will not be repeated here. Figures 1 to 6

[0220] Please refer to Figure 8 The application provides a high-precision image splicing device, which comprises:

[0221] The processor 801, the memory 802, the input and output unit 804 and the bus 803.

[0222] The processor 801 is connected with the memory 802, the input and output unit 804 and the bus 803.

[0223] The memory 802 stores a program, and the processor 801 calls the program to execute the method in the foregoing Figure 1 、 Figure 2 、 Figure 3 , Figure 4 、 Figure 5 and Figure 6 .

[0224] The application provides a computer readable storage medium, and the computer readable storage medium stores a program, and the program is executed on a computer to execute the method in the foregoing Figure 1 、 Figure 2 、 Figure 3 , Figure 4 、 Figure 5 and Figure 6 .

[0225] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system, the device and the unit described above can refer to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0226] ​In several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented in other manners. For example, the described device embodiments are merely schematic. The division of the units is merely a logical function division. There can be another division manner for the actual implementation, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections can be indirect couplings or communication connections through some interfaces, devices or units, and can be in electrical, mechanical or other forms.

[0227] The units described as separated components can or can not be physically separated, and the components displayed as units can or can not be physical units, i.e., can be located in one place, or can be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purposes of the embodiments.

[0228] In addition, each functional unit in the embodiments of the present application can be integrated in a processing unit, or each unit can exist physically as a separate unit, or two or more units can be integrated in one unit. The integrated unit can be implemented in the form of hardware, or in the form of a software functional unit.

[0229] When the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on such an understanding, the technical solutions of the present application essentially, or the part that contributes to the prior art, or all or a part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the embodiments of the present application. The foregoing storage medium includes: U disk, mobile hard disk, read-only memory (ROM, read-only memory), random access memory (RAM, random access memory), magnetic disk or optical disk, and various other media that can store program codes.

Claims

1. A high-precision image stitching method, characterized in that, The method comprises the following steps: Respectively through the left side view camera and the right side view camera, a calibration board containing several circular regions is photographed to obtain a left side circular region image and a right side circular region image; Based on the left side circular region image, a left side circular feature region is screened out, and based on the right side circular region image, a right side circular feature region is screened out; The center coordinates of the left side circular feature region are determined as left side pixel coordinates, and the center coordinates of the right side circular feature region are determined as right side pixel coordinates; The left side pixel coordinates are matched with the coordinates of the corresponding circular region on the calibration board to obtain a left side coordinate set, and the right side pixel coordinates are matched with the coordinates of the corresponding circular region on the calibration board to obtain a right side coordinate set; Based on the left side coordinate set and the right side coordinate set, internal parameter matrices and external parameter matrices of the left side view camera and the right side view camera are respectively generated; Any one of the left side view camera and the right side view camera is selected as a master camera, and the other one is selected as a slave camera, and a transformation matrix of the slave camera to the master camera is obtained in combination with the external parameter matrices of the two side cameras, and a coordinate system of the master camera is a stitching reference coordinate system; Based on the transformation matrix, a pixel correspondence relationship between the slave camera and the master camera is calculated to generate a pixel mapping table; A left side view to-be-stitched image and a right side view to-be-stitched image of the same to-be-detected panel photographed by the master camera and the slave camera are obtained; The pixel mapping table is called to perform pixel remapping on the right side view to-be-stitched image, the right side view to-be-stitched image after remapping is fused with the left side view to-be-stitched image, and a complete image is output; After the internal parameter matrices and the external parameter matrices of the left side view camera and the right side view camera are respectively generated based on the left side coordinate set and the right side coordinate set, before any one of the left side view camera and the right side view camera is selected as the master camera and the other one is selected as the slave camera, and the transformation matrix of the slave camera to the master camera is obtained in combination with the external parameter matrices of the two side cameras, and the coordinate system of the master camera is the stitching reference coordinate system, the method further comprises the following steps: The poses of the left side circular region image and the right side circular region image are adjusted to be consistent according to the internal parameter matrices and the external parameter matrices; The step of adjusting the poses of the left side circular region image and the right side circular region image to be consistent according to the internal parameter matrices and the external parameter matrices comprises the following steps: Based on the external parameter matrices of the left side view camera and the right side view camera, a rotation matrix and a translation vector of the left side view camera coordinate system and the right side view camera coordinate system relative to a calibration board world coordinate system are respectively extracted to determine an initial pose difference of the left side circular region image and the right side circular region image in a three-dimensional space; Taking the pose of the left side circular region image as a reference, a pose transformation parameter of the right side circular region image to the left side circular region image is calculated through the external parameter matrices, and the transformation parameter comprises a rotation compensation amount and a translation compensation amount. Coordinate conversion is performed on the right-side circular region image based on the pose transformation parameter, so that coordinates of all pixel points in the right-side circular region image are converted from the right-side camera coordinate system to the left-side camera coordinate system, and pose alignment is completed.

2. The high-precision image stitching method of claim 1, wherein, The center coordinates of the left-side circular feature region are determined as left-side pixel coordinates, and the center coordinates of the right-side circular feature region are determined as right-side pixel coordinates; the left-side pixel coordinates are matched with the coordinates of the corresponding circles on the calibration board to obtain a left-side coordinate set, and the right-side pixel coordinates are matched with the coordinates of the corresponding circles on the calibration board to obtain a right-side coordinate set, including: The left-side circular feature region and the right-side circular feature region are respectively sorted according to the rule of from top to bottom and from left to right, and the center coordinates of the sorted regions are extracted as the left-side pixel coordinates and the right-side pixel coordinates; Based on the pre-stored calibration board specification parameters, the left-side pixel coordinates and the right-side pixel coordinates are respectively matched with the coordinates of the corresponding circles on the calibration board to form the left-side coordinate set and the right-side coordinate set.

3. The high-precision image stitching method of claim 1, wherein, The calibration board containing a plurality of circular regions is respectively photographed by the left-side camera and the right-side camera to obtain a left-side circular region image and a right-side circular region image, including: The calibration board containing a plurality of circular regions is photographed by the left-side camera and the right-side camera to obtain a left-side calibration board image and a right-side calibration board image; The left-side calibration board image and the right-side calibration board image are respectively subjected to binaryzation processing and connected domain segmentation; The binaryzation processing includes: The left-side calibration board image and the right-side calibration board image are respectively subjected to grayscale processing to obtain grayscale images; A local adaptive threshold algorithm is used to divide the grayscale images into non-overlapping sub-regions, and binaryzation threshold values of the sub-regions are determined; The grayscale images are converted into binary images based on the binaryzation threshold values; The connected domain segmentation includes: Pixels of the binary images are traversed, all independent connected regions are marked out, and left-side circular region images and right-side circular region images are selected from the connected regions.

4. The high-precision image stitching method of claim 1, wherein, Any one of the left-side camera and the right-side camera is selected as a master camera, and the other is selected as a slave camera, a transformation matrix of the slave camera to the master camera is obtained in combination with the external parameter matrix of the two cameras, including: A formula for establishing a transformation relationship between the slave camera and the master camera is as follows: The master camera is associated with the world coordinate system through a transformation relationship T_main: P_world = T_main * P_main; The slave camera is associated with the world coordinate system through a transformation relationship T_slave: P_world = T_slave * P_slave; Solving the two equations simultaneously, the transformation from the camera coordinate system to the principal camera coordinate system is obtained : wherein, H is the desired homography matrix, also called the projection transformation matrix, P is the coordinate point extracted on the main camera for the corresponding calibration board, P' is the coordinate point extracted on the secondary camera for the corresponding calibration board.

5. The high-precision image stitching method according to any one of claims 1 to 4, characterized in that, After the pixel mapping table is called to perform pixel remapping on the right-side view to-be-stitched image, the remapped right-side view to-be-stitched image and the left-side view to-be-stitched image are fused, and a complete image is output, further including: A multi-dimensional quality evaluation index system including structural similarity, peak signal-to-noise ratio, and average pixel gray difference at a stitching seam is constructed; respectively, and extracting pixels at the stitching seam of the complete image to calculate the average gray difference of the pixels at the stitching seam; performing weighted summation on the structural similarity value, the peak signal-to-noise ratio value and the average gray difference according to preset weights of the indicators to obtain a stitching quality score; if the stitching quality score is lower than a preset threshold, re-executing the steps after the left-view image to be stitched and the right-view image to be stitched are obtained until the stitching quality score reaches the preset threshold.

6. A high-precision image stitching device, characterized by, The device for performing the method according to any one of claims 1 to 5 comprises: a shooting unit configured to shoot a calibration board containing a plurality of circular regions through a left-view camera and a right-view camera respectively to obtain a left-view circular region image and a right-view circular region image; a screening unit configured to screen a left-view circular feature region based on the left-view circular region image and a right-view circular feature region based on the right-view circular region image; a determination unit configured to determine a center coordinate of the left-view circular feature region as a left-view pixel coordinate and a center coordinate of the right-view circular feature region as a right-view pixel coordinate; a matching unit configured to match the left-view pixel coordinate with a coordinate of a corresponding circle on the calibration board to obtain a left-view coordinate set and match the right-view pixel coordinate with the coordinate of the corresponding circle on the calibration board to obtain a right-view coordinate set; a first generation unit configured to generate an internal parameter matrix and an external parameter matrix of the left-view camera and the right-view camera respectively based on the left-view coordinate set and the right-view coordinate set; a selection unit configured to select any one of the left-view camera and the right-view camera as a master camera and the other as a slave camera, obtain a transformation matrix of the slave camera to the master camera in combination with the external parameter matrices of the two cameras, and set a coordinate system of the master camera as a stitching reference coordinate system; a second generation unit configured to calculate a pixel correspondence relationship between the slave camera and the master camera based on the transformation matrix and generate a pixel mapping table; an acquisition unit configured to acquire a left-view image to be stitched and a right-view image to be stitched of a same detection panel shot by the master camera and the slave camera; a calling unit configured to call the pixel mapping table to perform pixel remapping on the right-view image to be stitched, fuse the remapped right-view image to be stitched with the left-view image to be stitched, and output a complete image.

7. A high-precision image stitching device, characterized by, The device comprises: a processor, a memory, an input / output unit and a bus; the processor is connected with the memory, the input / output unit and the bus; the memory stores a program, and the processor invokes the program to perform the high-precision image stitching method according to any one of claims 1 to 5.

8. A computer readable storage medium, which stores a program, and the program performs the high-precision image stitching method according to any one of claims 1 to 5 when executed on a computer.

Images