Array multi-camera image stitching calibration board design method and calibration method

Abstract

This invention discloses a design method and calibration method for an array multi-camera image stitching calibration board. The design method includes: determining the width and spacing width of the marker patterns based on camera resolution; determining the maximum height of the marker patterns based on the AOI field of view height; determining the number of marker patterns per group and their height difference; determining the number of pattern groups on the calibration board; and setting the calibration board based on the above parameters. The calibration method includes: acquiring images from adjacent cameras, extracting marker patterns and assigning them numbers and sequence types; using the pattern closest to the reference point in the first camera as a benchmark, searching for a matching pattern with the same number and sequence type in the second camera; and completing the stitching using the center coordinates of the two patterns as the stitching point. This invention uses bar pattern encoding with progressively decreasing heights to quickly complete stitching without complex feature matching, achieving pixel-level accuracy, strong versatility, and support for cascading expansion.

Description

Technical Field

[0001] This document relates to the field of array multi-camera image stitching technology, and in particular to a design method and calibration method for an array multi-camera image stitching calibration board. Background Technology

[0002] A camera array, composed of relatively economical area scan cameras arranged horizontally, is used for visual inspection of wide-format products such as sheets and strips. Since multiple independent area scan cameras use the same field of view W, there is overlap between the cameras. Before implementing the specific inspection process, the camera images need to be stitched together to eliminate the overlapping areas. Several requirements must be met to achieve this stitching: 1. Design a calibration board that can be tailored to the specific camera's field of view, resolution, and number of cameras. A calibration board designed to meet different requirements should enable a convenient and efficient calibration process. 2. Utilizing the characteristics of the patterns on the calibration board, number the patterns and determine their order type. Design an algorithm specifically for the numbering and order type of the patterns on the calibration board to implement the splicing calibration function. Current calibration board designs mostly employ regular patterns such as checkerboards, dot matrices, or uniform stripes. When used for multi-camera stitching, these calibration boards typically rely on image feature point matching or grayscale correlation algorithms to find corresponding points in adjacent camera images. This algorithm is highly complex and prone to matching failures when lighting changes, images are blurred, or overlapping areas lack texture. Furthermore, existing calibration boards lack a structured design that matches the camera array arrangement, failing to guarantee sufficient and uniquely identifiable calibration features in each camera's field of view. This makes it difficult to balance stitching accuracy and efficiency. Summary of the Invention This invention provides a design method and calibration method for an array multi-camera image stitching calibration board, aiming to solve the above-mentioned problems.

[0003] According to an embodiment of the present invention, a method for designing an array multi-camera image stitching calibration board is provided, comprising: The width of the marking pattern and the spacing between the marking patterns on the calibration board are determined based on the camera resolution. The height of the longest marker pattern in the calibration board is determined based on the vertical AOI field of view height of the camera. The number of marking patterns on the calibration plate is determined based on the width of the marking patterns, the spacing width of the marking patterns, and the field of view of a single camera; The height difference of a single set of marker patterns is determined based on the height dimension of the largest marker pattern. The number of a single group of marking patterns on the calibration board is determined based on the number of marking patterns, the width of the marking patterns, the spacing width of the marking patterns, the field of view width of a single camera, and the field of view overlap width. The calibration board is set based on the parameters obtained above.

[0004] According to an embodiment of the present invention, a method for image stitching and calibration of an array of multi-camera cameras is provided, comprising: Acquire calibration board images from two adjacent cameras, designate the camera to which one image belongs as the first camera, and the camera to which the other image belongs as the second camera; The marker patterns are extracted from the two images respectively, and the center pixel coordinates of each marker pattern are calculated. After sorting them from left to right according to the center coordinates, each marker pattern is assigned a number and sequence type based on its height information. Using the preset stitching reference point in the first camera image as a reference, select the marker pattern closest to the reference point as the reference marker pattern, and record its number and sequence type; In the second camera image, find a marker pattern with the same number and the same order type as the reference marker pattern as the matching marker pattern; The center pixel coordinates of the reference marker pattern and the matching marker pattern are used as the stitching points of the first camera and the second camera, respectively, to complete image stitching.

[0005] This invention utilizes a systematic design of parameters such as the width, spacing, and height variation patterns of the marking patterns on the calibration board. This allows the calibration board to flexibly adapt to array systems of different specifications based on camera field of view, resolution, and overlapping areas, resulting in strong versatility. During the stitching process, no complex feature matching algorithms are required; simply extracting the marking patterns and determining their numbering and sequence type quickly identifies the stitching points, making the process simple and efficient. Furthermore, the dual encoding matching mechanism based on numbering and sequence type ensures the accuracy and robustness of identifying identical marking patterns, making it less susceptible to interference from lighting and noise, and achieving pixel-level stitching accuracy. Simultaneously, multiple cameras can be calibrated at once and cascade expansion is supported, effectively saving on-site debugging time and providing a reliable data foundation for subsequent image detection in multi-camera array systems.

[0006] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and in order to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0007] To more clearly illustrate the technical solutions in one or more embodiments of this specification or in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0008] Figure 1 This is a flowchart illustrating the design method of an array multi-camera image stitching calibration board according to an embodiment of the present invention. Figure 2 This is a flowchart of the array multi-camera image stitching and calibration method according to an embodiment of the present invention; Figure 3 This is a schematic diagram of the camera field of view and overlap according to an embodiment of the present invention; Figure 4 This is a schematic diagram of the calibration plate pattern according to an embodiment of the present invention; Figure 5 This is a schematic diagram of the calibration plate strip pattern design according to an embodiment of the present invention; Figure 6 This is a schematic diagram of the second design of the calibration plate strip pattern according to an embodiment of the present invention; Figure 7 This is a schematic diagram of a single set of bar patterns in the calibration plate of an embodiment of the present invention; Figure 8 This is a schematic diagram of a camera sensor chip pixel according to an embodiment of the present invention; Figure 9 This is a schematic diagram of the camera AOI according to an embodiment of the present invention; Figure 10 The image effect after canny processing according to an embodiment of the present invention; Figure 11 This is a schematic diagram of the reference point position in an embodiment of the present invention; Figure 12 This is a schematic diagram illustrating the sequence type of an embodiment of the present invention; Figure 13 This is a schematic diagram of the two camera stitching points before stitching, according to an embodiment of the present invention. Figure 14 This is a schematic diagram of the stitching points of two cameras in an embodiment of the present invention. Figure 15 This is a schematic diagram of multiple cameras before stitching together, according to an embodiment of the present invention; Figure 16 This is a schematic diagram of multiple cameras stitched together according to an embodiment of the present invention. Detailed Implementation

[0009] To enable those skilled in the art to better understand the technical solutions in one or more embodiments of this specification, the technical solutions in one or more embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this specification, and not all of the embodiments. Based on one or more embodiments of this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of this document.

[0010] The pattern on the calibration plate is as follows Figure 4-6 As shown in the diagram, 4 is a schematic diagram of the overall layout of the calibration plate. The bar pattern on the calibration plate is designed as follows: Figure 5 or Figure 6 All of them can achieve splicing and calibration functions. This document mainly focuses on... Figure 5 This style is explained.

[0011] Method Example 1 According to embodiments of the present invention, a method for designing an array multi-camera image stitching calibration board is provided. Figure 1 This is a flowchart of the array multi-camera image stitching calibration board design method according to an embodiment of the present invention, as follows: Figure 1 As shown, the array multi-camera image stitching calibration board design method according to an embodiment of the present invention specifically includes: S11. Determine the width of the marking pattern and the spacing width of the marking pattern on the calibration board according to the camera resolution; Based on the requirements of the detection target and accuracy, the parameters of the detection system, such as the camera field of view, working distance, and overlap area width, were designed first. All cameras in the array multi-camera system use the same parameters. Therefore, the resolution R of each camera can be calculated.

[0012] The definition is as follows: R = W / L(1); Where R is the camera resolution, i.e., the actual size corresponding to a single pixel in the image; here, the camera resolution is taken to be the same in both the horizontal and vertical directions, in mm; W is the camera's horizontal field of view, i.e., the actual object size corresponding to the width of the camera image, in mm; for example... Figure 3 As shown. L is the size of the camera's image sensor, in mm. The formula for calculating L is as follows; L = r x × c(2); Where, r x c represents the number of pixels on the camera sensor in the horizontal direction; c represents the pixel size of the camera sensor, in mm. Figure 8 As shown.

[0013] To accurately extract the edges of the bar patterns in the image, and considering that a denser number of bars is more conducive to calculating the stitching position, it is stipulated that the minimum width of the bar pattern should be 10 pixels. Therefore, the width of the bar pattern... The calculation formula is as follows: (3); To ensure the uniformity and symmetry of the calibration board pattern, and to facilitate edge extraction and center localization in subsequent image processing, the width of the bar pattern... The width is the same as the bar pattern, that is: = (4); and like Figure 7 As shown.

[0014] S12. Determine the height of the longest marker pattern on the calibration board based on the vertical AOI field of view height of the camera. ; The largest bar pattern cannot exceed the vertical field of view height of the image, such as... Figure 9 As shown. Let the height of the camera's vertical field of view (AOI) be H, in mm. The calculation formula is as follows: (5); Where m is the gap amount to prevent the bar pattern from moving out of the vertical AOI height setting due to installation errors, and m is generally taken as 1 to 2 mm; H is the vertical AOI field of view height of the camera, which is obtained by the following formula: H = H' - 2 × h(6); H'= c(7); Where: H' is the maximum field of view of the camera in the vertical direction, in mm; h represents the number of pixels in the vertical direction of the camera's image sensor; h represents the height of the shield after the camera's AOI area is set, in mm.

[0015] S13. Determine the number of marking patterns on the calibration plate based on the width of the marking pattern and the spacing width of the marking patterns; The bar patterns on the calibration plate are divided into multiple groups. Each group has the same number, arrangement, and size of bar patterns, and groups share a single bar pattern. This bar pattern is the shortest bar pattern within its group, denoted as I. min (The same principle applies if the tallest bar pattern is used as the shared bar pattern between each group.) Figure 7As shown, the formula for calculating the number N of bar patterns within a single group is: N = Ceiling(W / [(x1+x2)×2])×2+1(8); Where Ceiling(x) represents rounding x up; This way, the sum of the widths of all bar patterns and the distances between them within a single group will be greater than the width of the camera's field of view W. Simultaneously, N must be an odd number to ensure that the bar pattern I with the maximum height within a single group... max The pattern consists of symmetrically distributed stripes radiating from a central point, with the height of the stripes gradually decreasing towards both sides. For example... Figure 7 As shown.

[0016] S14. Determine the height difference of a single set of marker patterns based on the height dimension of the largest marker pattern; A single group of bar patterns with the bar pattern at its maximum height. Centered on the center, the bar patterns on both sides are symmetrically distributed, with the height of the bar patterns decreasing on each side. The formula for calculating the height difference Δy is: (9); Specifically, the highest bar pattern The height is The height of the stripes on its adjacent sides is -Δy, the next level is -2×Δy, and so on, until the smallest bar pattern is reached. The height is determined by the arithmetic progression of the heights, which results in each bar having a unique and identifiable height value.

[0017] With the highest bar pattern The height of the bar pattern at the corresponding position can be obtained by subtracting Δy from the height of the bar pattern. For ease of processing, the calculated Δy in the above formula can be rounded to one decimal place.

[0018] Number the bars within a single group, starting with the highest-ranking bar. The calculation formula is as follows: =(N-1) / 2+1(10); For example, when N=5, =3, meaning the third bar pattern from left to right within a single group is the tallest pattern, numbered 3. The patterns on the left and right sides are numbered 2 and 1 respectively, and 4 and 5 respectively. Number 1 corresponds to the bar pattern with the lowest height. .

[0019] S15. Determine the number of a single group of marking patterns on the calibration board based on the number of marking patterns, the width of the marking patterns, the spacing width of the marking patterns, the field of view width of a single camera, and the field of view overlap width. Since the total width of a single set of bar patterns is greater than the camera's field of view, the number of bar pattern sets on the calibration plate is set to... Meanwhile, since the overlapping area between cameras is half the field of view of a single camera, therefore The calculation formula is: (11); in, The width of the field of view overlap. The total width of a single group of bar patterns. The portion of the total width of a single pattern group that exceeds the field of view of a single camera is represented by (W / 2) divided by the integer result of this excess portion. If the result is not an integer, round it down to the nearest integer.

[0020] The range of the number of cameras that can be calibrated at one time during splicing calibration should be: [2, Within this range, it can be guaranteed that each camera's field of view contains only the highest-value bar pattern during a single calibration. This condition is a prerequisite for the correct execution of subsequent splicing and calibration procedures.

[0021] S16. Set the calibration board based on the parameters obtained above.

[0022] Based on the marking pattern width, interval width, maximum marking pattern height, number of marking patterns within a single group, height difference, and number of consecutive groups obtained from the above steps, refer to... Figure 4 The calibration plate layout shown has multiple sets of bar patterns arranged continuously along the length of the calibration plate, with the sets separated by the smallest pattern. The calibration plate design drawings are completed, mechanical drawings that can be processed are generated, and the calibration plate is processed and manufactured.

[0023] This invention utilizes a systematic design of parameters such as the width, spacing, and height variation patterns of the marking patterns on the calibration board. This allows the calibration board to flexibly adapt to array systems of different specifications based on camera field of view, resolution, and overlapping areas, resulting in strong versatility. During the stitching process, no complex feature matching algorithms are required; simply extracting the marking patterns and determining their numbering and sequence type quickly identifies the stitching points, making the process simple and efficient. Furthermore, the dual encoding matching mechanism based on numbering and sequence type ensures the accuracy and robustness of identifying identical marking patterns, making it less susceptible to interference from lighting and noise, and achieving pixel-level stitching accuracy. Simultaneously, multiple cameras can be calibrated at once and cascade expansion is supported, effectively saving on-site debugging time and providing a reliable data foundation for subsequent image detection in multi-camera array systems.

[0024] Method Example 2 According to embodiments of the present invention, a method for calibrating multi-camera array image stitching is provided. Figure 2 This is a flowchart of the array multi-camera image stitching and calibration method according to an embodiment of the present invention, as follows: Figure 2 As shown, the array multi-camera image stitching calibration method according to an embodiment of the present invention specifically includes: S21. Acquire calibration board images from two adjacent cameras, designate the camera to which one image belongs as the first camera, and the camera to which the other image belongs as the second camera; During the stitching calibration, the cameras are stitched and calibrated in pairs according to the array arrangement order. That is, starting from one end of the array, the first and second cameras are stitched together as the first group, and after that, the second and third cameras are stitched together as the second group, and so on.

[0025] Cameras are stitched together in pairs for calibration. Since the overlapping area of ​​the cameras is half the field of view of a single camera, the center position of the stitched area between the two cameras in the camera image coordinate system is set as P', located at 3 / 4 of the field of view of the previous camera. P' is the reference position. P'=W c ×3 / 4(12) like Figure 11 As shown, P' is the position of the center point of the previous camera stitching area in the camera image coordinate system, and W... c The width of the field of view in the single-camera image coordinate system; all units are pixels. The reference point P' is located at 3 / 4 of the field of view of the first camera, that is, at the middle position of the overlapping area of ​​the two cameras, ensuring that a complete strip pattern can always be found near P' as a stitching reference.

[0026] S22. Extract the marker patterns from the two images respectively and calculate the center pixel coordinates of each marker pattern. After sorting them from left to right according to the center coordinates, assign a number and sequence type to each marker pattern based on the height information of each marker pattern. Edge extraction is performed on the image using the Canny algorithm, such as... Figure 10 The image effect after canny processing according to an embodiment of the present invention; Unqualified and interfering edges are filtered out. The filtering conditions are that the area of ​​the edge points and the edge pattern cannot intersect with the image edge. Specifically, if the number of pixels in a certain edge region is less than the preset area threshold, it is judged as noise interference and filtered out; if an edge pattern intersects with the image boundary, it is judged as an incomplete bar pattern and filtered out.

[0027] Calculate the center position coordinates P of all segmented edge patterns, in pixels.

[0028] S23. Using the preset stitching reference point in the first camera image as a reference, select the marker pattern closest to the reference point as the reference marker pattern, and record its number and sequence type; This indicates the number of the bar pattern segmented within the camera's view. Number The calculation method is to calculate the center point coordinates of all extracted bar patterns in the image. Sort the bars from smallest to largest, and assign them a sequence number k from left to right, {k∈Z+ | 1≤k≤NI}; Where NI represents the number of extracted bar patterns. Among these NI bar patterns, there must be a highest-ranking bar pattern, whose number is NI. It is known. Record the sequence number k of each bar pattern, let... The serial number is Calculate the sequence number of other bar patterns and... The absolute value of the difference Δa k The calculation formula is as follows: 1≤k≤NI and k≠ (13); Then, the number of each bar pattern is calculated using the following formula: ; S24. In the second camera image, find a marker pattern with the same number and the same order type as the reference marker pattern as a matching marker pattern; The method for determining the sequence type O is: first search for the bar pattern with the highest height. As can be seen from the previous calibration plate design process, the highest bar in the image is... And the number of cameras is in [2, There is one and only one camera within the range. Similarly, there is one and only one bar pattern with the smallest height. .lie in and The bar patterns can be ordered in two ways based on their height: ascending or descending. For example... Figure 12 This is a schematic diagram of the sequence type of an embodiment of the present invention, with the highest bar pattern defined. The center coordinates are Minimum bar pattern The center coordinates are The center coordinates of the other bar patterns are Therefore, there are 4 possible sequences for the bar pattern: a. The highest bar pattern, O=1; b. Minimum bar pattern, O=2; c. Ascending order, O=3; < < ; d. Descending order, O=4; < < ; S25. Using the center pixel coordinates of the reference mark pattern and the matching mark pattern as the stitching points of the first camera and the second camera respectively, image stitching is completed.

[0029] Specifically, this includes: finding the bar pattern closest to the reference point P'; and calculating all points P in the first camera. k Given the set D of distances to P', find the minimum value among them: set up (15); ; Find the bar pattern corresponding to min(D), denoted as [D]. Record its number as The sequence type is .

[0030] Following the above process, segment and extract the bar patterns from the second camera, then calculate their numbers and sequence types; among these bar patterns, select the one with the smallest center coordinates and the corresponding number. and sequence type The bar pattern is set to ; Calculate separately and The pixel coordinates corresponding to the center position within each camera. and ,in Round the calculated value down to the nearest integer. Round the calculated value up to the nearest integer.

[0031] (16); (17); and These are the stitching coordinates of each of the two adjacent cameras; once determined, the image stitching can be completed. For example... Figure 13 The image shown is the one before stitching. Figure 14 The image shows the result after stitching.

[0032] When the total number of cameras in the array exceeds the number of groups Ng that can be covered in a single calibration, the last camera that has been stitched is taken as the new starting camera. The next unstitched camera adjacent to this starting camera is selected, and steps S21 to S25 are repeated until all cameras have been stitched and calibrated. Figure 15 The image shown is a sequence of images independently acquired by each camera before stitching, such as... Figure 16 The image shown is the complete wide-format panoramic image formed after stitching.

[0033] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A design method for an array multi-camera image stitching calibration board, characterized in that, include: The width of the marking pattern and the spacing between the marking patterns on the calibration board are determined based on the camera resolution. The height of the longest marker pattern in the calibration board is determined based on the vertical AOI field of view height of the camera. The number of marking patterns on the calibration plate is determined based on the width of the marking patterns, the spacing width of the marking patterns, and the field of view of a single camera; The height difference of a single set of marker patterns is determined based on the height dimension of the largest marker pattern. The number of a single group of marking patterns on the calibration board is determined based on the number of marking patterns, the width of the marking patterns, the spacing width of the marking patterns, the field of view width of a single camera, and the field of view overlap width. The calibration board is set based on the parameters obtained above.

2. The method according to claim 1, characterized in that, The process of determining the width of the marking pattern and the spacing width of the marking pattern on the calibration board based on the camera resolution specifically includes: The camera's resolution is determined based on the camera's horizontal field of view and the size of the camera's image sensor. The width of the marking pattern and the spacing width of the marking pattern are set according to the resolution.

3. The method according to claim 1, characterized in that, The process of determining the height of the longest marker pattern on the calibration board based on the vertical AOI field of view height of the camera specifically includes: Obtain the vertical AOI field of view height of the camera; The height of the largest marker pattern is determined based on the AOI field of view height.

4. The method according to claim 1, characterized in that, The step of determining the number of marking patterns on the calibration plate based on the width of the marking pattern and the spacing width of the marking patterns specifically includes: With the constraint that the total width of a single group of marker patterns is greater than the field of view of a single camera, the number of marker patterns in a single group is determined based on the width of the marker patterns and the spacing width, and the number is an odd number.

5. The method according to claim 1, characterized in that, The step of determining the height difference of a single set of marker patterns based on the height dimension of the largest marker pattern specifically includes: The height difference between adjacent marker patterns is determined based on the height dimension of the largest marker pattern and the number of marker patterns within a single group.

6. The method according to claim 1, characterized in that, The determination of the number of individual groups of marking patterns on the calibration board based on the number of marking patterns, the width of the marking patterns, and the spacing width of the marking patterns specifically includes: The number of consecutively arranged marker pattern sets is determined based on the difference between the width of the overlapping area between adjacent cameras, the total width of a single set of marker patterns, and the field of view width of a single camera.

7. A calibration method based on the array multi-camera image stitching calibration board design method according to any one of claims 1 to 6, characterized in that, include: Acquire calibration board images from two adjacent cameras, designate the camera to which one image belongs as the first camera, and the camera to which the other image belongs as the second camera; The marker patterns are extracted from the two images respectively, and the center pixel coordinates of each marker pattern are calculated. After sorting them from left to right according to the center coordinates, each marker pattern is assigned a number and sequence type based on its height information. Using the preset stitching reference point in the first camera image as a reference, select the marker pattern closest to the reference point as the reference marker pattern, and record its number and sequence type; In the second camera image, find a marker pattern with the same number and the same order type as the reference marker pattern as the matching marker pattern; The center pixel coordinates of the reference marker pattern and the matching marker pattern are used as the stitching points of the first camera and the second camera, respectively, to complete image stitching.

8. The method according to claim 7, characterized in that, Assigning a number to each marked pattern specifically includes: The highest marker pattern in each image is determined. Using the number of the highest marker pattern as the baseline, the marker patterns on its left and right sides are numbered sequentially in descending order of their distance from the highest marker pattern.

9. The method according to claim 7, characterized in that, The order types include four types: highest mark pattern, lowest mark pattern, ascending order, and descending order.

10. The method according to claim 7, characterized in that, When the total number of cameras in the array exceeds the number of groups that can be covered by a single calibration, the last camera that has been stitched is taken as the new starting camera, and the next unstitched camera adjacent to the starting camera is selected to repeat the steps of the method described in claim 7 until all cameras are stitched.

Images