Calibration module and calibration method for calibration of linear scanning camera array combined imaging system

Abstract

The invention belongs to the field of image processing, and discloses a calibration module for calibrating a line scanning camera array combined imaging system, which comprises a calibration plate, the calibration plate is provided with a stripe grid area which comprises a plurality of stripe grids which are arranged at equal intervals from top to bottom and is used for calibrating distortion parameters of the line scanning camera; and the chessboard grid area is formed by arranging a plurality of grids with the same specification according to a grid array form, and is used for the line scanning camera to calibrate the inclination angle in the moving direction of the calibration board. The calibration module is used for carrying out distortion calibration and inclination angle calibration on the line scanning cameras through the stripe grid area and the chessboard grid area, so that images obtained by the line scanning cameras can be integrated into a complete image without distortion after distortion correction and inclination angle correction. Meanwhile, the invention further discloses a calibration method based on the calibration module.

Description

Technical Field

[0001] This invention belongs to the field of image processing, and specifically relates to a calibration module and calibration method for calibrating a line scan camera array combined imaging system. Background Technology

[0002] A line scan camera is an imaging device widely used in machine vision systems, such as imaging and inspection systems for continuous or long-sized targets like printed materials, sheets, and films. Its working principle is similar to that of a common printer. It acquires images at high speed, moving from one line to the next, and then stitches the images together to form a single picture after all images have been scanned. A line scan imaging system mainly consists of a line array sensor, a motor, a driver, a motion control card or PLC, and mechanical units. These components work together to ensure that the acquired images are synchronized with the conveyor belt.

[0003] However, due to the resolution requirements of the inspection targets, a single line scan camera usually cannot cover the entire inspection area of ​​the product. Therefore, multiple line scan cameras are typically required to form an imaging array. Due to a series of interferences such as machining errors and assembly errors, the imaging resolution (unit: mm / pixel), spatial position, installation angle, installation height, and imaging distortion degree of each line scan camera are different. This poses significant challenges to the application of line scan camera arrays in precision optical measurement. These problems result in numerous difficulties for large-format line scan array imaging systems in precision optical measurement and inspection applications. Summary of the Invention

[0004] Based on this, the first objective of the present invention is to provide a calibration module for calibrating a combined imaging system of line scan camera arrays. This calibration module provides line scan cameras with stripe grid areas and checkerboard grid areas for self-distortion calibration and tilt angle calibration, so that the images obtained by each line scan camera can be integrated into a complete and distortion-free image after distortion correction and tilt angle correction.

[0005] In addition, the present invention also discloses a calibration method based on the calibration module.

[0006] To achieve the above-mentioned objectives, the present invention adopts the following technical solution:

[0007] A calibration module for calibrating a line scan camera array combined imaging system includes a calibration board;

[0008] The calibration plate is equipped with:

[0009] Striped grid area: includes multiple striped grids arranged at equal intervals from top to bottom, used for calibrating distortion parameters of the line scan camera;

[0010] The checkerboard grid area consists of multiple uniformly sized grids arranged in a grid array, used to calibrate the tilt angle of the line scan camera in the direction of movement of the calibration plate.

[0011] In the calibration module for calibrating a combined imaging system of a line scan camera array described above, the calibration board is further provided with color bands and / or dot grid areas for calibrating the relative positions of the images of each line scan camera on the horizontal plane.

[0012] The angle between the extension direction of the ribbon and the forward direction of the calibration plate is an acute angle.

[0013] The dot matrix grid area is formed by arranging multiple spots of the same size in a grid array.

[0014] In the calibration module described above for calibrating a line scan camera array combined imaging system, the color band extends from the edge of the calibration plate to a position near the stripe grid area; the checkerboard grid area and the dot grid area are located on both sides of the color band.

[0015] In the calibration module described above for calibrating a combined imaging system of a line scan camera array, the checkerboard grid area, the dot grid area, and the color band are each in pairs and are located on both sides of the striped grid area.

[0016] In the calibration module for calibrating a line scan camera array combined imaging system, the calibration board is further provided with a staggered grid area, which is composed of multiple grids arranged at equal intervals from top to bottom; the multiple grids are periodically staggered.

[0017] In the calibration module described above for calibrating a combined imaging system of a line scan camera array, the calibration board is further provided with a starting color band and / or an auxiliary grid area;

[0018] The starting color band extends from top to bottom;

[0019] The auxiliary grid area is formed by arranging multiple auxiliary grids sequentially from top to bottom; the density of the auxiliary grids in the auxiliary grid area is greater than the density of the striped grids in the striped grid area;

[0020] The starting color band and / or auxiliary grid area are set at the edge of the calibration plate.

[0021] In the calibration module described above for calibrating a combined imaging system of a line scan camera array, there are at least 3 striped grids and at least 2 rows of grids within the field of view of the line scan camera array. Preferably, there are at least 6 striped grids and at least 5 rows of grids within the field of view of the line scan camera array.

[0022] In the calibration module described above for calibrating a combined imaging system of a line scan camera array, the stripe grid area is also used to: adjust the height of the line scan camera and the offset angle perpendicular to the moving direction of the calibration plate, so that the resolution of the line scan camera and the moving accuracy of the calibration plate are consistent.

[0023] Meanwhile, the present invention also discloses a calibration method for a combined imaging system of a line scan camera array, the method involving a line scan camera array, a calibration module as described in any of the above claims, and a moving device; the line scan camera array is composed of multiple line scan cameras;

[0024] The method includes the following steps:

[0025] Step 1: Place the calibration plate on the moving device and below the line scan camera array, so that the length direction of the stripe grid is consistent with the movement of the calibration plate;

[0026] Step 2: Adjust the height of the line scan camera and the offset angle perpendicular to the direction of movement of the calibration plate to make the resolution of the line scan camera consistent with the movement accuracy of the calibration plate.

[0027] Step 3: Obtain an image using a line scan camera, and obtain the image distortion parameters by the center positions of different stripe grids in the image; determine the tilt angle of the line scan camera by calculating the offset angle of the grid in the checkerboard area of ​​the image relative to the grid in the checkerboard area on the calibration board.

[0028] Step 4: Correct the images obtained by each line scan camera according to the distortion parameters and offset angle of each line scan camera;

[0029] Step 5: Adjust and stitch together the images obtained by each line scan camera according to their relative positions on the horizontal plane to obtain the image of the calibration plate;

[0030] If the image obtained in step 5 is confirmed to be an accurate and distortion-free imaging result, then the relative position, tilt angle, distortion parameters, and resolution of each line scan camera on the horizontal plane will be used as parameters for subsequent imaging.

[0031] In the above method, the distortion parameter is determined as follows:

[0032] Step 310: Obtain the center positions of multiple striped grid cells in the image, and obtain the actual center position coordinate sequence of multiple striped grid cells E'={ y'1, y'2, y'3, …, y'} n};

[0033] Step 311: Select the m central stripes from a plurality of striped grids, and calculate the array of adjacent point spacing D = {d'1, d'2, d'3, …, d'}. m-1}, thus obtaining the mean d'0 of array D;

[0034] Step 312: Calculate the theoretically distortion-free center position coordinate sequence E={y1, y2, y3, …, y} of multiple striped grids based on the mean d'0. n};

[0035] Step 313: Assign the actual center position coordinate sequence E' = {y'1, y'2, y'3, …, y'} n} and the distortion-free center position coordinate sequence E={ y1, y2, y3, …, y n A distortion model is constructed based on the theoretical undistorted coordinate y and distorted coordinate y', wherein the distortion model consists of n multivariate linear equations; the distortion parameter k can be obtained by solving the distortion model using the least squares method.

[0036] The distortion model is n quadratic polynomials or n cubic polynomials;

[0037] The quadratic polynomial is y' = a0y + a1y + a2y 2 ;

[0038] The cubic polynomial y' = a0 + a1y + a2y 2 +a3y 3 ;

[0039] a0, a1, a2, and a3 are the coefficients of the distortion model, and these coefficients are constants.

[0040] When the constructed distortion model is a quadratic polynomial, m≥3; when the distortion model is a cubic polynomial, m≥4.

[0041] In the above method, the method for determining the tilt angle is as follows:

[0042] Step 320: Obtain the vertex coordinates of the grid cells in the chessboard area of ​​the image;

[0043] Step 321: Fit the coordinates of the same side vertex of the same row of cells into a straight line in the direction perpendicular to the movement of the calibration plate, and calculate the slope of the straight line;

[0044] Step 322: Repeat step 321 to obtain the slope of the straight line formed by the cells in different rows, and calculate the average slope of the multiple straight lines;

[0045] Step 323: Calculate the tilt angle of the line scan camera based on the average slope;

[0046] The method for obtaining the relative positions of each line scan camera in the horizontal plane is as follows:

[0047] The image matching method is used to determine the relative horizontal position of two adjacent line scan cameras by using the color bands and / or dot grid areas in the images of two adjacent line scan cameras as the objects of image overlay.

[0048] Once the relative horizontal positions of any two line scan cameras are obtained, the relative coordinate values ​​of each line scan camera are obtained with the position of the first line scan camera as the zero point.

[0049] When stitching images together, the positions of each image are adjusted according to their relative coordinate values.

[0050] The beneficial effects of this invention are as follows:

[0051] The calibration plate of this invention provides a striped grid area and a checkerboard grid area for line scan cameras to perform self-distortion calibration and tilt angle calibration, so that the images obtained by each line scan camera can be integrated into a complete, distortion-free image after distortion correction and tilt angle correction.

[0052] In a preferred embodiment of the present invention, the calibration plate is further provided with color bands and dot matrix grid areas for different line scan cameras to calibrate their horizontal positions;

[0053] Meanwhile, the present invention also discloses a calibration method for a line scan camera array developed based on the above-mentioned innovative calibration board. This calibration method, combined with a calibration module, can obtain distortion-free joint imaging results of the line scan camera array. Attached Figure Description

[0054] Figure 1A This is one implementation of the calibration module;

[0055] Figure 1B This is one implementation of the calibration module;

[0056] Figure 1C This is one implementation of the calibration module;

[0057] Figure 2 This is a schematic diagram of the projection of a line scan camera array onto a horizontal plane.

[0058] Figure 3 This is a schematic diagram of the acquisition of a striped raster image;

[0059] Figure 4 A schematic diagram for determining the center position of the dark stripes in the horizontal direction of a striped grid;

[0060] Figure 5 This is a schematic diagram of the chessboard grid area image when the tilt angle φ is 0.

[0061] Figure 6 This is a schematic diagram showing the chessboard grid area image when the tilt angle φ is positive.

[0062] Figure 7 This is a schematic diagram showing the tilt angle φ of the chessboard grid area image when it is negative.

[0063] Figure 8 This is a schematic diagram of corner point detection for each cell in the chessboard grid area when the tilt angle φ is positive;

[0064] Figure 9 This is a schematic diagram of corner detection for each cell in the chessboard grid area when the tilt angle φ is negative;

[0065] Figure 10 This is a schematic diagram of the straight-line fitting of the corner points of each cell in the chessboard grid area when the tilt angle φ is positive;

[0066] Figure 11 This is a schematic diagram of the straight-line fitting of the corner points of each cell in the chessboard grid area when the tilt angle φ is negative;

[0067] Figure 12 This is a schematic diagram of obtaining a distortion-free image by taking pixels from a distorted image and filling them with pixels using a known tilt angle φ.

[0068] Figure 13 This is a schematic diagram illustrating the conversion principle of image distortion.

[0069] Figure 14 This is a schematic diagram illustrating the conversion principle of image distortion.

[0070] Figure 15A Images obtained from the camera on Line 1;

[0071] Figure 15B Images obtained from the camera on Line 2;

[0072] Figure 15C Images obtained from the camera on Line 3;

[0073] Figure 15D Images obtained from the camera on Line 4;

[0074] Figure 15E Images obtained from the camera on Line 5;

[0075] Figure 15F Images obtained from the camera on Line 6;

[0076] Figure 15G Images obtained from the camera on Line 7;

[0077] Figure 15H Images obtained from the camera on Line 8;

[0078] Figure 15I Images obtained from the scanning camera on Line 9;

[0079] Figure 15J Images obtained from the scanning camera on Line 10;

[0080] Figure 16A The horizontally aligned image obtained by image registration from the No. 1 line scan camera;

[0081] Figure 16B The horizontally aligned image obtained by image registration of the image acquired by line scan camera No. 2;

[0082] Figure 16C The horizontally aligned image obtained by image registration of the image acquired by the No. 3 line scan camera;

[0083] Figure 16D The horizontally aligned image obtained by image registration of the image acquired by the No. 4 line scan camera;

[0084] Figure 16E The horizontally aligned image obtained by image registration from the No. 5 line scan camera;

[0085] Figure 16F The horizontally aligned image obtained by image registration of the image acquired by the No. 6 line scan camera;

[0086] Figure 16G The horizontally aligned image obtained by image registration of the image acquired by the No. 7 line scan camera;

[0087] Figure 16H The horizontally aligned image obtained by image registration of the image acquired by the No. 8 line scan camera;

[0088] Figure 16I The horizontally aligned image obtained by image registration of the image acquired by the line scan camera No. 9;

[0089] Figure 16J The horizontally aligned image obtained by image registration of the image acquired by the No. 10 line scan camera;

[0090] Figure 17 The image captured by line scan camera i and line scan camera i-1 are vertically shifted, cropped, and stitched together to obtain a distortion-free combined image. Detailed Implementation

[0091] The technical solution of the present invention will be further illustrated below through specific embodiments. Those skilled in the art should understand that the embodiments described are merely illustrative of the present invention and should not be construed as limiting the invention in any way.

[0092] Example 1

[0093] refer to Figures 1A to 1C A calibration module includes a calibration board 1, wherein the calibration board 1 is provided with:

[0094] Striped grating area 2: includes multiple striped gratings arranged at equal intervals from top to bottom, used for calibrating distortion parameters of the line scan camera;

[0095] Chessboard grid area 3: It is formed by arranging multiple uniform grids in a grid array, and is used to calibrate the tilt angle of the line scan camera in the direction of movement of the calibration plate 1.

[0096] refer to Figure 1A , Figure 1A In the simplest implementation of the present invention, the distortion parameters of the line scan camera and the tilt angle in the moving direction of the calibration plate 1 can be calibrated through the striped grid area 2 and the checkerboard grid area 3.

[0097] In actual use, the calibration plate 1 is driven at a constant speed from left to right by the external moving device. In the following description, the direction of movement of the calibration plate 1 is defined as the x-direction, and the direction perpendicular to the movement of the calibration plate 1 is defined as the y-direction; both the x-direction and the y-direction are the directions of movement on the horizontal plane.

[0098] Multiple line scan cameras are mounted above the calibration plate 1. The arrangement of the line scan cameras can be found in the following reference. Figure 3 Multiple line scan cameras are used to ensure that the width of calibration board 1 (corresponding to the product under test) can be covered;

[0099] The method for determining the distortion parameters of the striped grid area 2 can be found in Section 1, Distortion Calibration, of Example 2.

[0100] The tilt angle of the online scanning camera in the x-direction of the chessboard grid area 3 can be referred to Section 2, Tilting Angle Calibration, of Example 2;

[0101] Preferably, refer to Figure 1B and Figure 1C The calibration plate 1 is also provided with a color band 4 and / or a dot matrix grid area 5 for calibrating the relative position of the images of each line scan camera on the horizontal plane; the angle between the extension direction of the color band 4 and the forward direction of the calibration plate 1 is an acute angle; the dot matrix grid area 5 is formed by multiple spots of the same size arranged in a grid array.

[0102] The function of color band 4 and dot grid area 5 is to determine the relative horizontal position of adjacent line scan cameras, and the two can operate independently to determine the relative horizontal position of the line scan cameras.

[0103] Since color band 4 is not completely coincident with the x-direction or the y-direction, but forms an acute angle with the x-direction, as long as there is an overlapping area in the images at different horizontal positions, the accuracy of image registration can be determined based on the degree of overlap of color band 4 and whether the two ends of the stripe grid are aligned.

[0104] The function of dot grid area 5 is similar. Since dot grid area 5 is composed of multiple spots, it is obvious that image registration has not been achieved as long as misalignment occurs.

[0105] Ideally, color band 4 and dot grid area 5 exist simultaneously. Color band 4 is used to determine the relative position of the two images, and dot grid area 5 can help determine whether the two images have been accurately registered based on whether the dots overlap.

[0106] Preferably, the color band 4 extends from the edge of the calibration plate 1 to a position close to the striped grid area 2; the checkerboard grid area 3 and the dot matrix grid area 5 are respectively located on both sides of the color band 4;

[0107] When the above-mentioned design of distributing the color strips on both sides of the color strip 4 is adopted, the pattern structure of the calibration plate 1 of the present invention can be more compact, and at the same time, the functional areas are more clearly divided, making it clearer to use.

[0108] In practical use, the checkerboard grid area 3 can be closer to the striped grid area 2, or the dot matrix grid area 5 can be closer to the striped grid area 2. This has no substantial impact on the function of the calibration plate 1 of the present invention.

[0109] Preferably, the chessboard grid area 3, the dot matrix grid area 5, and the color band 4 are each in pairs and symmetrically arranged on both sides of the striped grid area 2;

[0110] This symmetrical or near-symmetrical design allows for calibration of many subsequent operations. For example, the tilt angle can be calculated using two symmetrical checkerboard grid areas 3 to calibrate the data. Furthermore, during image registration, two images can be quickly registered using only two tilted color bands 4, without the need for additional patterns. Additionally, the presence of two dot grid areas 5 during image registration further confirms the accuracy and reliability of the registration.

[0111] Preferably, the calibration plate 1 is further provided with a staggered grid area 6, which is composed of multiple grids arranged at equal intervals from top to bottom; the multiple grids are arranged in a staggered manner periodically.

[0112] The calibration plate 1 is also provided with a starting color band 7 and / or an auxiliary grid area 8;

[0113] The starting color band 7 extends from top to bottom;

[0114] The auxiliary grid area 8 is formed by arranging multiple auxiliary grids sequentially from top to bottom; the density of the auxiliary grids in the auxiliary grid area 8 is greater than the density of the striped grids in the striped grid area 2;

[0115] The starting color band 7 and / or auxiliary grid area 8 are set at the edge of the calibration plate 1.

[0116] The function of the starting color band 7 is to indicate that the line scan camera calibration plate 1 has entered the field of view, and the line scan camera can work according to the preset program;

[0117] Auxiliary grid area 8 is an auxiliary tool for striped grid area 2. When calculating distortion parameters, if the calculation results are not reliable, distortion parameters can be calculated through auxiliary grid area 8. The calculation principle is the same. Since the grid density of auxiliary grid area 8 is greater, its calculation workload is greater, and of course, the calculation accuracy is also higher.

[0118] The text above describes that the chessboard grid area 3 and the dot grid area 5 are located on both sides of the color band 4, and the color band 4 is designed to be tilted. Therefore, the chessboard grid area 3 and the dot grid area 5 are generally chosen to be triangular. Of course, at least one of them can also be other shapes, such as trapezoids.

[0119] A color band 4 is provided on the outer side of the chessboard grid area 3; the color band 4 extends from one side of the calibration plate 1 to a position near the edge of the striped grid area 2;

[0120] The outer side of the chessboard grid area 3 is provided with a dot matrix grid area 5; the dot matrix grid area 5 is formed by multiple dots of the same size arranged in a grid array; the color band 4 separates the chessboard grid area 3 and the dot matrix grid area 5 on both sides; both the chessboard grid area 3 and the dot matrix grid area 5 are triangular.

[0121] Within the field of view of the online scanning camera array, there are at least 3 striped grids and at least 2 rows of grids. Preferably, within the field of view of the online scanning camera array, there are at least 6 striped grids and at least 5 rows of grids.

[0122] The dimensions of this calibration module are not strictly limited and should be designed according to the specifications of the materials to be tested. In actual design, the width of calibration plate 1 is generally not less than 50mm, preferably not less than 80mm; the length is not limited, such as being longer than 100mm, or longer than 200mm, or longer than 300mm.

[0123] The design logic for the spacing and grid size of adjacent stripes is as follows: the spacing of the stripes is determined according to the resolution requirements of the imaging system. The spacing of the stripes is generally set to meet the requirements of 5 to 10 times the resolution or higher; the grid size is designed to meet the requirements of 10 to 100 times the resolution or higher. For example, the spacing of the stripes can be selected from 10 to 1000 μm; each grid is a square, which is composed of multiple black and white alternating grids; the side length of each grid can be selected from 100 to 10000 μm.

[0124] The color band 4 is generally a black color band 4, and its tilt angle is 30~60°, preferably 45°; similarly, the diameter of the spot is 100~10000μm;

[0125] The density of the auxiliary grid in the auxiliary grid area 8 can be 1.5 to 3 times the density of the striped grid in the striped grid area 2; the density of the grid in the staggered grid area 6 is 0.3 to 0.9 times the density of the striped grid in the striped grid area 2, and the spacing between adjacent grids is 20 to 2000 μm.

[0126] The functions and roles of the above-mentioned intervals are described in more detail in Example 2.

[0127] Additionally, this embodiment also illustrates another implementation of the calibration board, see reference. Figure 1C , Figure 1C In the middle, the staggered grid area was moved to the central area.

[0128] Example 2

[0129] A calibration method for a combined imaging system using a line scan camera array, comprising a line scan camera array, a calibration module as described above, and a moving device; wherein the line scan camera array consists of multiple line scan cameras.

[0130] A schematic diagram of the projection of a line scan camera array onto the horizontal plane is shown. Figure 2 In actual production, it is not limited to such Figure 2 The layout shown is based on the principle that multiple line scan cameras can cover the entire image range.

[0131] Assuming the target's movement direction is the x-axis and the target's direction from the inside out is the y-axis, the sensor direction of the line scan camera is theoretically parallel to the y-axis. The system calibration method described in this embodiment is for 2D line scan cameras, therefore the z-axis (i.e., the direction perpendicular to the target surface) is not considered. Meanwhile, the resolution η0 of the line scan camera in the x-axis depends on the scanning accuracy requirements of the target movement, and is usually a fixed value, with the line scan acquisition synchronization signal provided by the displacement encoder. Theoretically, the resolution of the line scan camera in the y-axis must be the same as the resolution in the x-axis to ensure proportional reconstruction of the target in the scanned image.

[0132] The following explains the parameters that need to be calibrated for each line scan camera in a line scan camera array:

[0133] Table 1 shows the core calibration requirements of this invention. The relative position, tilt angle, distortion parameters, and resolution of each line scan camera on the horizontal plane need to be pre-calibrated before being applied to subsequent production.

[0134] Table 1. Parameters Required for Calibration of Linear Scan Camera Array

[0135] Camera number Location (mm) Inclination angle (°) Resolution (um / pixel) Distortion parameters Remark 1 <![CDATA[p1(x1, y1)]]> <![CDATA[φ1]]> <![CDATA[η1]]> <![CDATA[k(p1, i)| i∈{0,1,2,3}]]> Can be set as reference zero. 2 <![CDATA[p2(x2, y2)]]> <![CDATA[φ2]]> <![CDATA[η2]]> <![CDATA[k(p2, i)| i∈{0,1,2,3}]]> / 3 <![CDATA[p3(x3, y3)]]> <![CDATA[φ3]]> <![CDATA[η3]]> <![CDATA[k(p3, i)| i∈{0,1,2,3}]]> / 4 <![CDATA[p4(x4, y4)]]> <![CDATA[φ4]]> <![CDATA[η4]]> <![CDATA[k(p4, i)| i∈{0,1,2,3}]]> / 5 <![CDATA[p5(x5, y5)]]> <![CDATA[φ5]]> <![CDATA[η5]]> <![CDATA[k(p5, i)| i∈{0,1,2,3}]]> / 6 <![CDATA[p6(x6, y6)]]> <![CDATA[φ6]]> <![CDATA[η6]]> <![CDATA[k(p6, i)| i∈{0,1,2,3}]]> / 7 <![CDATA[p7(x7, y7)]]> <![CDATA[φ7]]> <![CDATA[η7]]> <![CDATA[k(p7, i)| i∈{0,1,2,3}]]> / 8 <![CDATA[p8(x8, y8)]]> <![CDATA[φ8]]> <![CDATA[η8]]> <![CDATA[k(p8, i)| i∈{0,1,2,3}]]> / 9 <![CDATA[p9(x9, y9)]]> <![CDATA[φ9]]> <![CDATA[η9]]> <![CDATA[k(p9, i)| i∈{0,1,2,3}]]> / 10 <![CDATA[p 10 (x 10 , y0)]]> <![CDATA[φ 10 ]]> <![CDATA[η 10 ]]> <![CDATA[k(p 10 , i)| i∈{0,1,2,3}]]> / refer to <![CDATA[p0(x0, y0)]]> <![CDATA[φ0]]> <![CDATA[η0]]> / Reference position selection: Linear scanning camera No. 1 tolerance / <![CDATA[[φ up , f down ]]> <![CDATA[[η up ,or down ]]> / Manually set value Calibration method Registration method Trigonometric method Proportion method <![CDATA[Polynomial calibration method: Y f = Y i + ΔY(Y i )]]> <![CDATA[ΔY(Y i () is a polynomial of degree 2-3.

[0136] The calibration method of the present invention includes the following steps:

[0137] Step 1: Based on the imaging accuracy requirements of the line scan camera (assuming it is η0, in μm), set the line scan camera's trigger pulse signal. That is, the line scan camera scans once and collects one line of data for every unit of accuracy the target moves.

[0138] Step 2: Ensure the calibration plate's length direction is strictly parallel to the target's movement direction. Adjust the calibration plate's position on the moving device (rotating detection wheel or horizontal displacement platform) so that the stripes on the calibration plate are parallel to the image's x-axis. The moving device can be set to move back and forth, and the line scan camera can continuously collect line data regardless of whether the signal is positive or negative. If the stripes on the image are parallel to the image's x-axis, the adjustment is considered to have met the target requirements.

[0139] Step 3: Adjust the height and focus of each line scan camera to ensure that the imaging field of view of each line scan camera is consistent, and that the pixel resolution and target movement accuracy in the camera scanning direction are consistent.

[0140] The specific adjustment method is as follows:

[0141] (1) Assuming the imaging accuracy requirement of the line scan camera is η0 and the number of pixels in the camera row is n, then n*η0 is the target field of view of each camera.

[0142] (2) Given that the width of each stripe in the horizontal stripe cluster on the calibration plate is d, when the camera contains n*η0 / d stripes, the pixel resolution and imaging accuracy of the line scan camera can be considered to be consistent.

[0143] (3) Place the stripe grid in the imaging field of the line scan camera (the moving device stops and the line scan camera works in the internally triggered contact acquisition mode), adjust the camera angle so that the spacing of the stripe image acquired by the camera is the smallest (at the same time, the number of stripes in the image is the largest, and the automatic detection and display function can be set on the software to quantify and display, and assist in debugging). At this time, it can be assumed that the sensor of the line scan camera and the stripe grid are perpendicular in the y direction.

[0144] (4) Based on step (3), adjust the camera height and focus so that the imaging field of view contains nη0 / d stripes. Whether the imaging field of view of the line scan camera is consistent can be judged by the number of stripe grids contained in the field of view of each camera. If the scanning direction of the camera contains the same number of stripe grids, the field of view of the line scan camera can be considered to be the same.

[0145] Step 4: Use equally spaced striped grids to calibrate the distortion of the line scan cameras and obtain the distortion parameters of each line scan camera.

[0146] Step 5: Use the checkerboard grid area on the calibration board image to calibrate the tilt angle of the line scan camera.

[0147] Steps 4 and 5 are not in any particular order;

[0148] After obtaining the distortion parameters and tilt angle of each line scan camera in steps 4 and 5, these two parameters are input into the image of the line scan camera, and a mature correction model in this field is used to perform image correction; the corrected image is then processed in step 6.

[0149] Step 6: Adjust and stitch together the images obtained by each line scan camera according to their relative positions on the horizontal plane to obtain the image of the calibration plate.

[0150] The following provides a detailed explanation of steps 4, 5, and 6:

[0151] 1. Distortion calibration

[0152] In line scan camera imaging, due to lens optical distortion, the distortion increases with distance from the image center pixel; this distortion can be expressed as a polynomial function, such as: y' = a0 + a1y + a2y 2 +a3y 3 y is the undistorted coordinate, and y' is the distorted coordinate. A second-order polynomial is usually sufficient to represent the distortion model, while a third-order polynomial can achieve a more accurate correction effect (in practical applications, the choice can be made based on the correction effect and computational complexity; a second-order polynomial is usually sufficient. When using a second-order polynomial as the distortion correction model, the minimum number of stripe grid point pairs is 3; when using a third-order polynomial as the distortion correction model, the minimum number of stripe grid point pairs is 4, otherwise the model will fail to solve).

[0153] The specific calibration process is as follows:

[0154] Step 41: Place the striated portion of the calibration plate with a known striated grid width within the imaging field of view of the line scan camera and keep it stationary. Then, use the line scan camera to continuously acquire line images to obtain the striated grid image, such as... Figure 3 As shown:

[0155] Step 42: Determine the center position of the dark stripes in the horizontal direction on the striped raster image, such as... Figure 4 As shown. The coordinate sequence of the dark fringe center positions in the horizontal direction (corresponding to the pixels of the camera's linear array sensor) is obtained: E'={ y'1,y'2, y'3, …, y' n Assume the horizontal width of the image is I. width Then take image I width / 2 is the center position.

[0156] Step 43: Take the coordinates of m (e.g., 8) points near the center, and calculate the distance array between adjacent points D={d'1, d'2, d'3, …, d' m-1};

[0157] Step 44: Calculate the mean d'0 of array D.

[0158] Step 45: From the point closest to the center (Image I) width Starting from position / 2, calculate the distortion-free theoretical positions of other points outwards with a step size (mean d'0), resulting in the distortion-free theoretical coordinate array E={ y1, y2, y3, …, y n};

[0159] Step 46: Create the distorted coordinate array E'={ y'1, y'2, y'3, …, y'} n} and the theoretical coordinate array { y1, y2, y3, …, y n}, forming pairwise corresponding coordinate pairs. A distortion model is constructed based on the theoretical undistorted coordinate y and distorted coordinate y', wherein the distortion model consists of n multivariate linear equations; the distortion parameter k is obtained by solving the distortion model using the least squares method.

[0160] The distortion model consists of n quadratic polynomials or n cubic polynomials; the quadratic polynomial is y' = a0y + a1y + a2y. 2 The cubic polynomial y' = a0 + a1y + a2y 2 +a3y 3 a0, a1, a2, and a3 are the coefficients of the distortion model, which are constants. The least squares method is used to solve the equation to obtain the distortion parameter k, which is either {a0, a1, a2, a3} or {a0, a1, a2}.

[0161] Using distortion parameters in conjunction with mature models for image correction is a standard technique in this field and will not be elaborated upon here.

[0162] 2. Tilt Angle Calibration

[0163] like Figure 5 , Figure 6 and Figure 7As shown, when the tilt angle φ of the line scan camera is not 0, if φ is positive (rotating counterclockwise relative to the vertical direction), the scanned chessboard grid image shows a parallelogram grid tilted to the right; if φ is negative (rotating clockwise relative to the vertical direction), the scanned chessboard grid image shows a parallelogram grid tilted to the left. Assuming the angle of the hypotenuse of the detected parallelogram in the image is α, then the mathematical relationship is tan |α| = sin |φ|. If tilting to the right is negative (rotating clockwise relative to the vertical direction) and tilting to the left is positive (rotating counterclockwise relative to the vertical direction), then the mathematical relationship can be described as tan α = -sin φ.

[0164] Therefore, the formula for calculating the tilt angle (or verticality) of a line scan camera is: φ = arcsin (-tanα).

[0165] The calibration calculation steps are as follows:

[0166] Step 51: On the acquired image, find all the corner points of the chessboard grid area and arrange the corner points in order of x and y coordinate values ​​(from smallest to largest).

[0167] like Figure 8 and Figure 9 As shown, Figure 8 This is a schematic diagram of corner point detection for each cell in the chessboard grid area when the tilt angle φ is positive; Figure 9 This is a schematic diagram of corner detection for each cell in the chessboard grid area when the tilt angle φ is negative;

[0168] Step 52: Fit a straight line in the y-direction to the corner points detected in each cell of the chessboard grid area;

[0169] refer to Figure 10 and Figure 11 , Figure 10 This is a schematic diagram of the straight-line fitting of the corner points of each cell in the chessboard grid area when the tilt angle φ is positive; Figure 11 This is a schematic diagram of the straight-line fitting of the corner points of each cell in the chessboard grid area when the tilt angle φ is negative;

[0170] Step 53: Calculate the slope of the family of lines in the y-direction k = tan (π / 2 – α) = - tanα, and calculate the average value of all fitted lines to obtain the average slope k*.

[0171] Step 54: Calculate the tilt angle of the line scan camera: φ = arcsin (k*).

[0172] Step 55: Scan the camera image online and correct the image with the center line as the origin in the y-direction.

[0173] The correction method is as follows:

[0174] Create a blank image with a width of (width - (height / 2) sin φ) in the x-direction and a height of (1 + cos φ) / 2 in the y-direction. Based on the coordinate mapping relationship, extract pixels from the distorted image and perform pixel filling (considering that digital images are discrete, bilinear interpolation can be used to extract pixel grayscale information during pixel filling) to obtain a distortion-free image, such as... Figure 12 As shown. At this point, the image captured by the line scan camera and the physical object of the calibration board are in the same proportion.

[0175] Combination Figure 13 and Figure 14 The conversion principle of image distortion is as follows:

[0176] Undistorted coordinates (x, y) => Distorted coordinates (x', y')

[0177] x' = x – x0 + (y-y0) tan φ + x0; => x' = x + (y-y0) tan φ ;

[0178] y' = (y-y0) / cos φ + y0; => y' = y0 + (y-y0) / cos φ ;

[0179] y0 is half the length of the line scan camera sensor. For example, for a 2K line scan camera, y0 = 1024, which is a value greater than 0.

[0180] Distorted coordinates (x', y') => Undistorted coordinates (x, y), which is the formula for correcting sharp distortion:

[0181] x = x' - (y'-y0) sin φ ;

[0182] y = y0 + (y'-y0) cos φ ;

[0183] Figure 13 This is a schematic diagram of image correction when the tilt angle φ is positive; Figure 14 This is a schematic diagram of image correction when the tilt angle φ is negative.

[0184] 3. Calibration of the relative positions of each line scan camera on the horizontal plane

[0185] The calibration method can be referenced as follows:

[0186] Step 61: Using the No. 1 line scan camera (the first camera to scan the target calibration plate in the target movement direction) as the reference image, take a complete image of the calibration plate, as follows: Figure 15AAs shown in the figure below; the image capture results of other line scan cameras are shown in the figure below. Since line scan camera 1 is the first to contact the calibration plate to be scanned, the other line scan cameras will have a horizontal displacement relative to line scan camera 1.

[0187] Figures 15B to 15J A schematic diagram of images acquired by scanning cameras on lines 2 through 10;

[0188] Step 62: Perform image registration on line scan camera i-1 using the calibration board image acquired by line scan camera i-1 to obtain the positional offset dx of line scan camera i-1 relative to line scan camera i-1. i and dy i .

[0189] The offset can be obtained by accumulating the coordinates of the camera scanned along line 1 to get the corresponding camera position information: p i (x) i , y i ).

[0190] By horizontally shifting the images captured by all cameras and the No. 1 line scan camera, a horizontally aligned image can be obtained, such as... Figures 16A to 16J As shown.

[0191] In step 62, image matching can use color bands and dot grid areas to overlay images. When the spots in the color bands and dot grid areas of two adjacent line scan cameras overlap, the two images are considered to have completed image registration, and dx is obtained. i and dy i ;

[0192] Step 63: Extract the vertical stripe grid portion from each image obtained in step 62, based on the physical size L of the stripe grid and the number of pixels n contained in the stripe grid. p Solve for the resolution of each camera: η = L / n p The unit is um / pixel.

[0193] In step 3 above, it has been confirmed that the pixel resolution and imaging accuracy of the line scan camera are consistent. However, the resolution determined in this step is not very accurate. After correction in step 63, the obtained resolution is a reliable resolution and will be used as the parameter for the final resolution.

[0194] Step 64: After vertically shifting, cropping, and stitching the images captured by line scan camera i and line scan camera i-1, a distortion-free joint imaging result can be obtained, such as... Figure 17 As shown.

[0195] The embodiments presented herein are merely selected implementations based on combinations of all possible embodiments. The appended claims should not be limited to the embodiments described herein. Some numerical ranges used in the claims include sub-ranges within them, and variations within these ranges should also be covered by the appended claims.

Claims

1. A calibration module for calibrating a line scan camera array combined imaging system, characterized in that, Including calibration plate; The calibration plate is equipped with: Striped grid area: includes multiple striped grids arranged at equal intervals from top to bottom, used for calibrating distortion parameters of the line scan camera; The checkerboard grid area consists of multiple uniformly sized grids arranged in a grid array, used to calibrate the tilt angle of the line scan camera in the direction of movement of the calibration plate.

2. The calibration module for calibrating a line scan camera array combined imaging system according to claim 1, characterized in that, The calibration plate is also provided with color bands and / or dot grid areas for calibrating the relative positions of the images of each line scan camera on the horizontal plane; The angle between the extension direction of the ribbon and the forward direction of the calibration plate is an acute angle. The dot matrix grid area is formed by arranging multiple spots of the same size in a grid array.

3. The calibration module for calibrating a line-scan camera array combined imaging system according to claim 2, characterized in that, The color band extends from the edge of the calibration plate to a position near the striped grid area; the checkerboard grid area and the dot matrix grid area are located on both sides of the color band.

4. The calibration module for calibrating a line scan camera array combined imaging system according to claim 2, characterized in that, The chessboard grid area, dot matrix grid area, and color band are each divided into two and symmetrically arranged on both sides of the striped grid area.

5. The calibration module for calibrating a line scan camera array combined imaging system according to claim 1, characterized in that, The calibration plate is also provided with a staggered grid area, which is composed of multiple grids arranged at equal intervals from top to bottom; the multiple grids are arranged in a staggered manner periodically.

6. The calibration module for calibrating a line-scan camera array combined imaging system according to claim 1, characterized in that, The calibration plate is also provided with a starting color band and / or an auxiliary grid area; The starting color band extends from top to bottom; The auxiliary grid area is formed by arranging multiple auxiliary grids sequentially from top to bottom; the density of the auxiliary grids in the auxiliary grid area is greater than the density of the striped grids in the striped grid area; The starting color band and / or auxiliary grid area are set at the edge of the calibration plate.

7. The calibration module according to any one of claims 1 to 6, characterized in that, Within the field of view of the online scanning camera array, there are at least 3 striped grids and at least 2 rows of grids. Preferably, within the field of view of the online scanning camera array, there are at least 6 striped grids and at least 5 rows of grids.

8. The calibration module according to claim 1, characterized in that, The striped grid area is also used to: adjust the height of the line scan camera and the offset angle perpendicular to the direction of movement of the calibration plate, so that the resolution of the line scan camera and the movement accuracy of the calibration plate are consistent.

9. A calibration method for a line scan camera array combined imaging system, characterized in that, The method relates to a line scan camera array, a calibration module as described in any one of claims 1 to 8, and a mobile device; the line scan camera array consists of a plurality of line scan cameras; The method includes the following steps: Step 1: Place the calibration plate on the moving device and below the line scan camera array, so that the length direction of the stripe grid is consistent with the movement of the calibration plate; Step 2: Adjust the height of the line scan camera and the offset angle perpendicular to the direction of movement of the calibration plate to make the resolution of the line scan camera consistent with the movement accuracy of the calibration plate. Step 3: Obtain an image using a line scan camera, and obtain the image distortion parameters by the center positions of different stripe grids in the image; determine the tilt angle of the line scan camera by calculating the offset angle of the grid in the checkerboard area of ​​the image relative to the grid in the checkerboard area on the calibration board. Step 4: Correct the images obtained by each line scan camera according to the distortion parameters and offset angle of each line scan camera; Step 5: Adjust and stitch together the images obtained by each line scan camera according to their relative positions on the horizontal plane to obtain the image of the calibration plate; If the image obtained in step 5 is confirmed to be an accurate and distortion-free imaging result, then the relative position, tilt angle, distortion parameters, and resolution of each line scan camera on the horizontal plane will be used as parameters for subsequent imaging.

10. The method according to claim 9, characterized in that, The method for determining the distortion parameters is as follows: Step 310: Obtain the center positions of multiple striped grid cells in the image, and obtain the actual center position coordinate sequence of multiple striped grid cells E'={ y'1, y'2, y'3, …, y'} n }; Step 311: Select the m central stripes from a plurality of striped grids, and calculate the array of adjacent point spacing D={d'1, d'2, d'3, …, d' m-1 }, thus obtaining the mean d'0 of array D; Step 312: Calculate the theoretically distortion-free center position coordinate sequence E={ y1,y2, y3, …, y} for multiple striped grids based on the mean d'0. n }; Step 313: Assign the actual center position coordinate sequence E' = {y'1, y'2, y'3, …, y'} n } and the distortion-free center position coordinate sequence E={ y1, y2, y3, …, y n A distortion model is constructed based on the theoretical undistorted coordinate y and distorted coordinate y', wherein the distortion model consists of n multivariate linear equations; the distortion parameter k can be obtained by solving the distortion model using the least squares method. The distortion model is n quadratic polynomials or n cubic polynomials; The quadratic polynomial is y' = a0y + a1y + a2y 2 ; The cubic polynomial y' = a0 + a1y + a2y 2 +a3y 3 ; a0, a1, a2, and a3 are the coefficients of the distortion model, and these coefficients are constants. When the constructed distortion model is a quadratic polynomial, m≥3; when the distortion model is a cubic polynomial, m≥4.

11. The method according to claim 9, characterized in that, The method for determining the tilt angle is as follows: Step 320: Obtain the vertex coordinates of the grid cells in the chessboard area of ​​the image; Step 321: Fit the coordinates of the same side vertex of the same row of cells into a straight line in the direction perpendicular to the movement of the calibration plate, and calculate the slope of the straight line; Step 322: Repeat step 321 to obtain the slope of the straight line formed by the cells in different rows, and calculate the average slope of the multiple straight lines; Step 323: Calculate the tilt angle of the line scan camera based on the average slope; The method for obtaining the relative positions of each line scan camera in the horizontal plane is as follows: The image matching method is used to determine the relative horizontal position of two adjacent line scan cameras by using the color bands and / or dot grid areas in the images of two adjacent line scan cameras as the objects of image overlay. Once the relative horizontal positions of any two line scan cameras are obtained, the relative coordinate values ​​of each line scan camera are obtained with the position of the first line scan camera as the zero point. When stitching images together, the positions of each image are adjusted according to their relative coordinate values.

Images