Image splicing method based on grating scale encoder positioning and weight fusion

Abstract

The invention discloses an image splicing method based on grating scale encoder positioning and weight fusion, and belongs to the field of image processing. The method comprises a system calibration stage and a real-time splicing stage, wherein the system calibration stage comprises calibration data acquisition, camera pose calculation and mapping relation establishment; the real-time splicing stage comprises workpiece image acquisition, real-time coordinate mapping, image relative displacement calculation, weight accumulation to a global buffer area, judgment of processing states of all sub-images and normalized fusion output. The splicing process is completely independent of matching of image features, that is, the image content is not required, the splicing speed is high, and the real-time performance is high. A direct mapping relation between encoder readings and camera world poses is established through global motion calibration, and each sub-image is positioned based on absolute coordinates. No accumulated error exists, and the precision of the global image can be ensured. As long as the mechanical structure is kept stable, one-time calibration can be achieved, and then the calibration data can be used for multiple times of jigsaw puzzle in the follow-up process.

Description

Technical Field

[0001] This invention relates to image processing, and more particularly to an image stitching method based on grating ruler encoder positioning and weight fusion. Background Technology

[0002] In the field of industrial measurement, which involves the digital acquisition and processing of large-size, high-precision images, the physical resolution and field of view of a single camera sensor are limited. For large-size workpiece measurement scenarios, it is usually necessary to take multiple local images and then use image stitching technology to synthesize these images into a complete global image, which is then used for subsequent image detection and measurement.

[0003] Existing image stitching technologies face the following drawbacks and limitations for high-resolution, high-precision industrial applications:

[0004] (1) Limitations of feature matching algorithms that depend on image content: Currently, the mainstream image stitching algorithms based on the spatial domain (such as feature matching methods based on SIFT feature detector, FAST corner detector, SURF feature detector, etc.) will decrease in reliability, accuracy and speed when processing complex images or images without any features. This is manifested in the following ways: images with complex textures or images with similarity are prone to low feature discrimination between sub-images, which can easily generate a large number of incorrect matching point pairs, resulting in incorrect stitching positions; while images with simple backgrounds (such as solid color backgrounds) lack significant feature points, which can also lead to matching failures and incorrect stitching positions; in addition, due to the computational complexity of feature point extraction kernel matching algorithms, it is difficult to meet the efficiency requirements of industrial inspection for real-time stitching of a large number of sub-images.

[0005] (2) Insufficient adaptability to complex motion models in the physical world: Many simple stitching methods assume that there is only translational motion between images. However, in actual motion, due to factors such as the flatness error of the mechanical guide rail and the installation error of the camera, there will be not only translation but also rotation between sub-images. In addition, if the image stitching method is based only on adjacent sequences, the error will accumulate as the number of images increases, which will lead to serious positional deviations at the beginning and end of the global image, and the global measurement accuracy cannot be guaranteed. Summary of the Invention

[0006] Purpose of the invention: The purpose of this invention is to provide an image stitching method based on grating ruler encoder positioning and weight fusion, which establishes a global world coordinate system, maps each sub-image to world coordinates through mapping relationships, and then realizes the stitching of multiple images through an adaptive weight fusion method.

[0007] Technical solution: An image stitching method based on grating ruler encoder positioning and weight fusion, characterized by including a system calibration stage and a real-time stitching stage;

[0008] The system calibration phase includes the following steps:

[0009] S1. Calibration data acquisition: The control motion platform drives the camera to scan the checkerboard calibration board and acquire images of the calibration board at multiple different positions; at each acquisition position i, the system synchronously records the calibration board image Ii (i=1,2,...,N) and the corresponding grating ruler encoder reading Gi=(Gxi,Gyi,Gzi).

[0010] S2. Calculate the camera pose Li based on the nearest corner point for each sampling position i;

[0011] S3. Establishing the mapping relationship: Associate the encoder reading Gi at each position with the calculated camera pose Li to generate a calibration mapping table;

[0012] The real-time stitching stage includes the following steps:

[0013] P1. Workpiece Image Acquisition: Acquire sub-image I of the workpiece to be measured, and synchronously read the current reading of the grating ruler encoder: M=(Mx,My,Mz);

[0014] P2, Real-time coordinate mapping: Input the current encoder reading G into the mapping function Fmap, and calculate the current camera pose in the world coordinate system by looking up the calibration mapping table and performing linear interpolation: (Wxcur,Wycur,θcur)=Fmap(Mx,My,Mz);

[0015] P3. Image relative displacement calculation: For each pixel (x,y) in the atomic image, calculate its new coordinates (x′,y′) in the global coordinate system through rotation and translation transformations, where the rotation angle is θcur and the translation amount is (Wxcur,Wycur).

[0016] P4. Weight Accumulation to Global Buffer: The transformed sub-image I′ is weighted and accumulated with its corresponding weighted image W′: Global Image Buffer: Local weight buffer: Dweight = Dweight + W′; where the weight map W′ is generated based on the adaptive weight map generation of geometric distance. For any pixel point p(i,j) in the image, its weight value w(i,j) is determined by the minimum distance from the point to the boundary of the effective region: w(i,j) = min{d1,d2,…,dN}, where dk represents the distance to the k-th boundary, and N is 4 or 5 depending on the field of view mode;

[0017] P5. Determine the processing status of all sub-images: Check if all sub-images have been processed. If there are still unprocessed sub-images, return to step P1 to continue processing the next sub-image. If all sub-images have been processed, proceed to step P6.

[0018] P6. Normalized Fusion Output: After accumulating all sub-images, perform normalized fusion calculation, Ifinal=Dimage / Dweight, to generate the final seamless stitched panoramic image Ifinal.

[0019] Furthermore, in step S2, for each sampling position i, the following processing is performed:

[0020] S21. Extract the pixel coordinates of all corner points in the calibration board image;

[0021] S22. Select the corner point closest to the center of the image as the reference point;

[0022] S23. Based on the pixel coordinates of the nearest corner point and the known world coordinates, combined with the camera intrinsic parameters, calculate the camera pose Li=(Wxi,Wyi,θi) in the world coordinate system, where Wxi and Wyi represent the coordinates of the camera center in the world coordinate system, and θi represents the rotation angle of the camera coordinate system relative to the world coordinate system.

[0023] Furthermore, in step S3, the format of each mapping data is: [Raster Ruler X] [Raster Ruler Y] [Raster Ruler Z] [Camera World Coordinates X] [Camera World Coordinates Y] [Camera Rotation Angle].

[0024] Furthermore, in step P3, during the relative displacement calculation of the images, the relative displacement of each image relative to the first image is calculated based on the first image, and then the images are stitched together.

[0025] Furthermore, in step P4:

[0026] When the system is in a rectangular field of view, the 4-distance calculation method is used: d1=jE, d2=iE, d3=H-1-jE, d4=W-1-iE; where E: edge buffer width, W, H: image width and height, i, j: pixel coordinates;

[0027] When the system is in a circular field of view, the 5-distance calculation method is used, which means that the circular boundary distance is added to the first 4 rectangular boundaries: Where (xc, yc) are the coordinates of the center of the circular field of view, and R is the radius of the circular field of view. For the center point (xc, yc), we directly take: d5 = R - Ed5 = RE.

[0028] Finally, the final weight value is determined by iterating through and comparing the results: .

[0029] Beneficial effects: The stitching process is completely independent of image feature matching, meaning it has no requirements on image content, and the stitching speed is fast and highly real-time. A direct mapping relationship between encoder readings and camera world pose is established through global motion calibration, with each sub-image positioned based on absolute coordinates. There is no cumulative error, ensuring the accuracy of the global image. High repeatability is achieved; as long as the mechanical structure remains stable, calibration can be performed once, and the calibration data can be used for multiple stitching operations. Attached Figure Description

[0030] Figure 1 This is a flowchart illustrating the method of the present invention;

[0031] Figure 2 This is a schematic diagram of the hardware configuration of an embodiment;

[0032] Figure 3 It is a graph of motion calibration data;

[0033] Figure 4 It involves acquiring two images to be stitched together;

[0034] Figure 5 It is an intermediate image stitched together after weighted fusion processing;

[0035] Figure 6 It is the intermediate image with accumulated weights;

[0036] Figure 7 This is the final image after it has been stitched together. Detailed Implementation

[0037] To make the technical solution of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0038] Example:

[0039] The system implementing the method consists of the following hardware components, such as Figure 2 As shown:

[0040] High-resolution area array camera: used to acquire sub-images of the calibration plate and workpiece.

[0041] High-precision optical encoder: mounted on the X, Y, and Z motion axes, providing real-time position feedback.

[0042] Motion control platform: enables the platform to achieve precise three-dimensional motion.

[0043] Chessboard calibration board: used for system calibration.

[0044] Industrial computers: responsible for image processing, data storage, and algorithm execution.

[0045] Light source system: Provides stable lighting conditions to ensure image quality.

[0046] like Figure 1 As shown, the specific implementation method is as follows:

[0047] During the system calibration phase:

[0048] Step 1: Calibration Data Acquisition

[0049] (1) Fix the chessboard squares on the motion platform.

[0050] (2) Control the motion platform to move along the xy axis respectively, and sample multiple positions evenly within the measurement range (e.g., sample at intervals of 4mm).

[0051] (3) At each sampling position, trigger the camera to acquire a checkerboard image and record the grating ruler readings on the x and y axes.

[0052] Step 2: Camera pose calculation

[0053] (1) The first checkerboard corner point whose position is closest to the center of the image is taken as the origin of the world coordinate system.

[0054] (2) For pose calculation at each sampling position, firstly, the row and column positions in the calibration grid are calculated based on the interpolation between the current grating ruler reading and the first calibration point. Then, the coordinates of the checkerboard corner points are accurately located in the image, and finally, the image pixel coordinates are converted into world coordinate system coordinates through geometric transformation.

[0055] (3) For each calibration location, calculate and record the following data:

[0056] [Raster Ruler X] [Raster Ruler Y] [Raster Ruler Z] [Camera World Coordinates X] [Camera World Coordinates Y] [Camera Rotation Angle]

[0057] The final calibration data obtained are as follows Figure 3 .

[0058] During the real-time stitching stage:

[0059] Step 1: Control the motion platform to move to the designated position, acquire the corresponding image, and read the values ​​of the grating ruler, such as... Figure 4 As shown

[0060] Specific operation: Control the motion platform to move to the sampling position according to the preset path. After moving into position, read the real-time position data of the three-axis optical encoder through hardware synchronization signals. Establish an "image-position" correspondence to ensure that each sub-image is accurately bound to the corresponding encoder reading.

[0061] Step 2: For each sub-map, find the four nearest calibration points in the calibration table, perform bilinear interpolation, and obtain its world coordinates.

[0062] Specific steps: Use the current encoder reading M corresponding to the sub-map as the query key and search in a pre-generated calibration mapping table. Locate the four calibration points closest to the current reading and use bilinear interpolation to calculate the current position of the camera center in the world coordinate system. The bilinear interpolation calculation method is as follows:

[0063]

[0064]

[0065] .

[0066] Step 3: Calculation of relative displacement of the image

[0067] Specific steps: Using the first image as a reference, calculate its pose (fx, fy, fa) in the world coordinate system according to the steps above. Similarly, for the current image, calculate its pose (tx, ty, ta). Calculate the displacement of the current image relative to the reference image:

[0068]

[0069]

[0070] Convert physical displacement to pixel displacement:

[0071]

[0072]

[0073] Where perpixel is the pixel equivalent.

[0074] Step 4: Accumulate weights into the global buffer

[0075] Specific operation: After multiplying sub-image I and weight image W by pixels, the data of each sub-image can be copied and superimposed onto the global image buffer based on dx and dy. Similarly, based on dx and dy, the weight map is copied and superimposed onto the global weight buffer: Dweight = Dweight + W′.

[0076] The intermediate image (global image buffer) stitched together after weighted fusion processing is as follows: Figure 5 As shown. The intermediate image (global weight buffer) with weight accumulation is as follows. Figure 6 As shown.

[0077] Step 5: Determine the processing status of all subgraphs

[0078] Specific steps: For each subgraph, repeat steps 1 to 4 until all subgraphs have been processed.

[0079] Step 6: Normalize the final accumulated image to obtain the final stitched image, such as... Figure 7 As shown

[0080] Specific operation: For the global image buffer Dimage and the global weight buffer Dweight, perform pixel-by-pixel normalization calculation: Ifinal = Dimage / Dweight, to obtain the final stitched image.

[0081] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of this patent should be determined by the appended claims.

Claims

1. An image stitching method based on grating ruler encoder positioning and weight fusion, characterized in that, This includes the system calibration phase and the real-time stitching phase; The system calibration phase includes the following steps: S1. Calibration data acquisition: The control motion platform drives the camera to scan the checkerboard calibration board and acquire images of the calibration board at multiple different positions; at each acquisition position i, the system synchronously records the calibration board image Ii (i=1,2,...,N) and the corresponding grating ruler encoder reading Gi=(Gxi,Gyi,Gzi). S2. Calculate the camera pose Li based on the nearest corner point for each sampling position i; S3. Establishing the mapping relationship: Associate the encoder reading Gi at each position with the calculated camera pose Li to generate a calibration mapping table; The real-time stitching stage includes the following steps: P1. Workpiece Image Acquisition: Acquire sub-image I of the workpiece to be measured, and synchronously read the current reading of the grating ruler encoder: M=(Mx,My,Mz); P2, Real-time coordinate mapping: Input the current encoder reading G into the mapping function Fmap, and calculate the current camera pose in the world coordinate system by looking up the calibration mapping table and performing linear interpolation: (Wxcur,Wycur,θcur)=Fmap(Mx,My,Mz); P3. Image relative displacement calculation: For each pixel (x,y) in the atomic image, calculate its new coordinates (x′,y′) in the global coordinate system through rotation and translation transformations, where the rotation angle is θcur and the translation amount is (Wxcur,Wycur). P4. Weight Accumulation to Global Buffer: The transformed sub-image I′ is weighted and accumulated with its corresponding weighted image W′: Global Image Buffer: Local weight buffer: Dweight = Dweight + W′; where the weight map W′ is generated based on the adaptive weight map generation of geometric distance. For any pixel point p(i,j) in the image, its weight value w(i,j) is determined by the minimum distance from the point to the boundary of the effective region: w(i,j) = min{d1,d2,…,dN}, where dk represents the distance to the k-th boundary, and N is 4 or 5 depending on the field of view mode; P5. Determine the processing status of all sub-images: Check if all sub-images have been processed. If there are still unprocessed sub-images, return to step P1 to continue processing the next sub-image. If all sub-images have been processed, proceed to step P6. P6. Normalized Fusion Output: After accumulating all sub-images, perform normalized fusion calculation, Ifinal=Dimage / Dweight, to generate the final seamless stitched panoramic image Ifinal.

2. The image stitching method based on grating ruler encoder positioning and weight fusion according to claim 1, characterized in that, In step S2, for each sampling position i, the following processing is performed: S21. Extract the pixel coordinates of all corner points in the calibration board image; S22. Select the corner point closest to the center of the image as the reference point; S23. Based on the pixel coordinates of the nearest corner point and the known world coordinates, combined with the camera intrinsic parameters, calculate the camera pose Li=(Wxi,Wyi,θi) in the world coordinate system, where Wxi and Wyi represent the coordinates of the camera center in the world coordinate system, and θi represents the rotation angle of the camera coordinate system relative to the world coordinate system.

3. The image stitching method based on grating ruler encoder positioning and weight fusion according to claim 1, characterized in that, In step S3, the format of each mapping data is: [Raster Ruler X] [Raster Ruler Y] [Raster Ruler Z] [Camera World Coordinates X] [Camera World Coordinates Y] [Camera Rotation Angle].

4. The image stitching method based on grating ruler encoder positioning and weight fusion according to claim 1, characterized in that, In step P3, during the relative displacement calculation of the images, the relative displacement of each image relative to the first image is calculated based on the first image, and then the images are stitched together.

5. The image stitching method based on grating ruler encoder positioning and weight fusion according to claim 1, characterized in that, In step P4: When the system is in a rectangular field of view, the 4-distance calculation method is used: d1=jE, d2=iE, d3=H-1-jE, d4=W-1-iE; where E: edge buffer width, W, H: image width and height, i, j: pixel coordinates; When the system is in a circular field of view, the 5-distance calculation method is used, which means that the circular boundary distance is added to the first 4 rectangular boundaries: Where (xc, yc) are the coordinates of the center of the circular field of view, and R is the radius of the circular field of view. For the center point (xc, yc), we directly take: d5 = R - Ed5 = RE. Finally, the final weight value is determined by iterating through and comparing the results: .

Images