Camera array calibration device and method

Abstract

The invention discloses a camera array calibration device and method, and relates to the field of camera calibration, in the camera array calibration device, a sample pattern design module adopts a sampling Mohr method close to a Nyquist condition to design a pattern on a calibration target; when the position of the camera array is calibrated, the measurement calibration module firstly adjusts an internal rotation angle before position calibration, then performs position calibration of the camera array, and determines the in-plane displacement of an image by shooting an in-plane displacement calibration pattern to complete position calibration of the camera array; when the angle of the camera array is calibrated, the light source, the calibration target, the reflective optical collimator and the camera array are arranged according to a light path, and the measurement calibration module firstly adjusts an internal rotation angle before angle calibration and then performs angle calibration of the camera array. The deflection angle of the optical axis of the camera is determined by shooting the parallel light which penetrates through the displacement calibration pattern in the plane and is reflected by the reflective optical collimator, and angle calibration of the camera array is completed. The calibration precision of the camera array can be improved.

Description

Technical Field

[0001] The present application relates to the field of camera calibration, and in particular to a camera array calibration device and method. Background Art

[0002] As a typical multi-camera system, camera array has attracted much attention in recent years due to its wide application prospects in multi-image super-resolution, light field reconstruction, computational photography and other fields. The final effect of the camera array often depends on the high-precision sub-pixel displacement between the collected images, which puts extremely high requirements on the measurement and calibration of the relative displacement and rotation relationship of each camera in the array. Traditional camera array calibration methods include image feature matching method, motion constraint method and target method.

[0003] Image feature matching methods estimate the relative position and posture between cameras by detecting and matching image feature points. Typical methods include SIFT (Scale-invariant Feature Transform) algorithm and ORB (Oriented FAST and Rotated BRIEF) algorithm. Although these methods perform well in terms of flexibility and real-time performance, their sensitivity to environmental changes (such as lighting changes, occlusion or low-texture scenes) may lead to reduced calibration accuracy. Motion constraint methods calibrate through the motion trajectory or rigid body model of the camera, which has certain applicability in dynamic scenes, but are limited by motion drift and cumulative error problems. Such methods are difficult to meet the requirements of high-precision calibration. Target-based methods have become the mainstream choice for multi-camera calibration due to their high accuracy and robustness, such as checkerboard targets and AprilTag targets. However, these methods have a high dependence on the clarity of the target in the image and the pixel resolution of the camera, and are difficult to meet the accuracy requirements of the micron level or even higher.

[0004] In summary, how to achieve high-precision calibration of camera arrays has become a problem that needs to be solved urgently. Summary of the invention

[0005] The purpose of this application is to provide a camera array calibration device and method, which can improve the calibration accuracy of the camera array.

[0006] To achieve the above objectives, this application provides the following solutions:

[0007] In a first aspect, the present application provides a camera array calibration device, comprising: a sample pattern design module, a calibration target, a light source, a reflective optical collimator, and a measurement calibration module; the measurement calibration module is connected to a target camera array;

[0008] The sample pattern design module is used for:

[0009] The sample pattern is sampled by using a sampling moiré method close to the Nyquist condition to obtain an aliased signal; the sampling moiré method close to the Nyquist condition is a sampling moiré method in which the difference between the spatial frequency of the sample pattern and the sampling frequency within the sampling range is within a set frequency range; the sample pattern is a periodic pattern;

[0010] Calculate the wave number of the aliased signal according to the number of sampling points in the measurement direction within the camera sample pattern image measurement area in the target camera array and the displacement measurement magnification of the target camera array;

[0011] Calculating the wave number of the sample pattern according to the wave number of the aliased signal and the number of sampling points in the measurement direction within the image measurement area;

[0012] Determine a target pattern based on the wave number of the sample pattern; the target pattern comprises: an in-plane displacement calibration pattern and an in-plane rotation calibration pattern; the in-plane displacement calibration pattern comprises a positioning mark;

[0013] The pattern on the calibration target is the target pattern;

[0014] Before position calibration of the target camera array, the internal rotation angle adjustment is first performed before position calibration;

[0015] When adjusting the in-plane rotation angle before position calibration, the target camera array is used to shoot the in-plane rotation calibration pattern on the calibration target to obtain a first image; the measurement calibration module is used to determine the in-plane rotation angle image of the in-plane rotation calibration pattern relative to each sub-camera in the target camera array according to the first image to obtain a first in-plane rotation angle image, and calculate the in-plane rotation angle of each sub-camera relative to the calibration target according to the first in-plane rotation angle image to obtain a first in-plane rotation angle; the first in-plane rotation angle is used to adjust the in-plane rotation relationship between the calibration target and the target camera array perpendicular to the optical axis direction, so that the relative rotation angle between each sub-camera in the target camera array is within a set angle range, and the in-plane rotation angle adjustment before position calibration is completed;

[0016] When the target camera array is positionally calibrated, the target camera array is used to shoot the in-plane displacement calibration pattern on the calibration target to obtain a second image; the measurement calibration module is used to read the second image, determine the gray stripe position of the aliasing signal according to the second image to obtain the first gray stripe position, and determine the image in-plane displacement according to the first gray stripe position and the black and white pixels and positioning marks in the second image to obtain the first image in-plane displacement; the first image in-plane displacement is used to adjust the positional relationship between each sub-camera in the target camera array perpendicular to the optical axis direction to complete the position calibration of the target camera array;

[0017] Before the angle calibration of the target camera array is performed, the internal rotation angle adjustment before the angle calibration is performed; when the internal rotation angle adjustment before the angle calibration and the angle calibration of the target camera array are performed, the light source is set at the focus of the reflective optical collimator; the calibration target is set at the light outlet position of the light source; the target camera array is set on the output light path of the reflective optical collimator; the light source is used to emit transmitted light;

[0018] When adjusting the internal rotation angle before angle calibration, the transmitted light is used to image through the in-plane rotation calibration pattern on the calibration target; the reflective optical collimator is used to collimate the light passing through the in-plane rotation calibration pattern on the calibration target to obtain a first parallel light; the target camera array is used to shoot the first parallel light to obtain a third image; the measurement and calibration module is used to determine the in-plane rotation angle image of the in-plane rotation calibration pattern relative to each sub-camera in the target camera array according to the third image to obtain a second in-plane rotation angle image, and calculate the in-plane rotation angle of each sub-camera relative to the calibration target according to the second in-plane rotation angle image to obtain a second in-plane rotation angle; the second in-plane rotation angle is used to adjust the rotation relationship between the calibration target and the target camera array so that the relative rotation angle between each sub-camera in the target camera array is within a set angle range, thereby completing the internal rotation angle adjustment before angle calibration;

[0019] When the target camera array is angle-calibrated, the transmitted light is used to form an image through the in-plane displacement calibration pattern on the calibration target; the reflective optical collimator is used to collimate the light through the in-plane displacement calibration pattern on the calibration target to obtain a second parallel light; the target camera array is used to shoot the second parallel light to obtain a fourth image; the measurement calibration module is used to read the fourth image, determine the gray stripe position of the aliasing signal according to the fourth image to obtain the second gray stripe position, and determine the image in-plane displacement according to the second gray stripe position and the black and white pixels and positioning marks in the fourth image to obtain the second image in-plane displacement, calculate the camera optical axis deflection angle according to the second image in-plane displacement, and adjust the optical axis off-plane deflection angle between each sub-camera in the target camera array according to the camera optical axis deflection angle to complete the angle calibration of the target camera array.

[0020] In a second aspect, the present application provides a camera array calibration method, which is used in the above-mentioned camera array calibration device, and the camera array calibration method includes:

[0021] Before position calibration of the target camera array, the internal rotation angle adjustment is first performed before position calibration;

[0022] When adjusting the front internal rotation angle of the position calibration, a first image is read, and an in-plane rotation angle image of the in-plane rotation calibration pattern relative to each sub-camera in the target camera array is determined according to the first image to obtain a first in-plane rotation angle image; and the in-plane rotation angle of each sub-camera relative to the calibration target is calculated according to the first in-plane rotation angle image to obtain a first in-plane rotation angle; the first in-plane rotation angle is used to adjust the in-plane rotation relationship between the calibration target and the target camera array perpendicular to the optical axis direction, so that the relative rotation angle between each sub-camera in the target camera array is within a set angle range, and the front internal rotation angle adjustment of the position calibration is completed; the first image is obtained by photographing the in-plane rotation calibration pattern on the calibration target by the target camera array;

[0023] When the target camera array is positionally calibrated, the second image is read, the gray stripe position of the aliased signal is determined according to the second image to obtain the first gray stripe position, and the image plane displacement is determined according to the first gray stripe position and the black and white pixels and positioning marks in the second image to obtain the first image plane displacement; the first image plane displacement is used to adjust the positional relationship between each sub-camera in the target camera array perpendicular to the optical axis direction to complete the position calibration of the target camera array; the second image is obtained by the target camera array shooting the in-plane displacement calibration pattern on the calibration target;

[0024] Before performing angle calibration on the target camera array, first perform the internal rotation angle adjustment before the angle calibration;

[0025] When adjusting the internal rotation angle before angle calibration, read the third image, determine the in-plane rotation angle image of the in-plane rotation calibration pattern relative to each sub-camera in the target camera array according to the third image, obtain the second in-plane rotation angle image, and calculate the in-plane rotation angle of each sub-camera relative to the calibration target according to the second in-plane rotation angle image to obtain the second in-plane rotation angle; the second in-plane rotation angle is used to adjust the rotation relationship between the calibration target and the target camera array, so that the relative rotation angle between each sub-camera in the target camera array is within the set angle range, and the internal rotation angle adjustment before angle calibration is completed; the third image is obtained by shooting the first parallel light by the target camera array; the first parallel light is obtained by collimating the light passing through the in-plane rotation calibration pattern on the calibration target by a reflective optical collimator;

[0026] When the target camera array is angle-calibrated, the fourth image is read, the gray stripe position of the aliased signal is determined according to the fourth image to obtain the second gray stripe position, and the image plane displacement is determined according to the second gray stripe position and the black and white pixels and positioning marks in the fourth image to obtain the second image plane displacement, the camera optical axis deflection angle is calculated according to the second image plane displacement, and the optical axis off-plane deflection angle between each sub-camera in the target camera array is adjusted according to the camera optical axis deflection angle to complete the angle calibration of the target camera array; the fourth image is obtained by the target camera array shooting the second parallel light; the second parallel light is obtained by the reflective optical collimator collimating the light passing through the in-plane displacement calibration pattern on the calibration target;

[0027] Wherein, the pattern on the calibration target is a target pattern, and the target pattern is determined by using a sample pattern design module. The method for determining the target pattern includes:

[0028] The sample pattern is sampled by using a sampling moiré method close to the Nyquist condition to obtain an aliased signal; the sampling moiré method close to the Nyquist condition is a sampling moiré method in which the difference between the spatial frequency of the sample pattern and the sampling frequency within the sampling range is within a set frequency range; the sample pattern is a periodic pattern;

[0029] Calculate the wave number of the aliased signal according to the number of sampling points in the measurement direction within the camera sample pattern image measurement area in the target camera array and the displacement measurement magnification of the target camera array;

[0030] Calculating the wave number of the sample pattern according to the wave number of the aliased signal and the number of sampling points in the measurement direction within the image measurement area;

[0031] A target pattern is determined based on the wave number of the sample pattern; the target pattern comprises: an in-plane displacement calibration pattern and an in-plane rotation calibration pattern; and the in-plane displacement calibration pattern comprises a positioning mark.

[0032] According to the specific embodiments provided in this application, this application has the following technical effects:

[0033] The present application provides a camera array calibration device and method. The sample pattern design module adopts a sampling moiré method close to the Nyquist condition to design an in-plane displacement calibration pattern and an in-plane rotation calibration pattern on a calibration target. The sampling moiré method has great advantages in amplifying small displacements. The present application proposes a sampling moiré method close to the Nyquist condition based on the folding phenomenon of aliased signals that occurs when the Nyquist condition is close, combined with the periodicity of the sampling target. The method can simultaneously capture high-frequency sampling signals and low-frequency aliasing stripes, thereby designing a clear target pattern. When the camera array is calibrated on a calibration target having the target pattern, the measurement calibration module can directly read the pattern captured by the camera array to obtain a high-precision image in-plane displacement and a camera optical axis deflection angle, thereby improving the calibration accuracy of the position calibration and angle calibration of the camera array. In addition, when calibrating the camera array, the present application first adjusts the in-plane rotation angle, which further improves the calibration accuracy of the camera array. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0035] Figure 1 A schematic diagram of a camera array calibration device for performing position calibration on a target camera array provided in an embodiment of the present application;

[0036] Figure 2 A schematic diagram of a camera array calibration device for performing angle calibration on a target camera array provided by an embodiment of the present application;

[0037] Figure 3 A schematic diagram of an in-plane displacement calibration pattern provided in an embodiment of the present application;

[0038] Figure 4 A schematic diagram of an in-plane rotation calibration pattern provided in one embodiment of the present application. DETAILED DESCRIPTION

[0039] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0040] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0041] Compared with traditional camera array calibration methods, phase target and moiré-based sampling methods show significant advantages in high-precision displacement and attitude measurement.

[0042] A phase target is a target that achieves high-precision displacement measurement and calibration by encoding phase information. Compared with traditional targets that rely on brightness or geometric features, phase targets use phase distribution or phase difference to encode position information, which can provide higher measurement resolution and anti-interference capabilities. This makes phase targets have significant advantages in fields such as nanomanufacturing, precision alignment, displacement sensing, and multi-camera calibration. At present, researchers have proposed a method to measure the 6D posture of periodic patterns by analyzing the Fourier spectrum of the acquired image, achieving a high precision of less than 1 / 1000 pixel in translation error.

[0043] The moiré phenomenon is usually produced by the superposition of two sets of periodic patterns that meet certain conditions. Traditional active projection moiré is generated by projecting a periodic grating (sample grating) onto a target surface (reference grating), which can achieve high-precision three-dimensional shape measurement, especially under uniform lighting conditions. In order to improve the problems of short period and complex phase unwrapping processing in traditional moiré processing methods, some researchers proposed to achieve large-scale micro-alignment without phase unwrapping through dual-frequency moiré fringe heterodyning, achieving an alignment accuracy of 1.8nm within a range of 12.75μm. This type of method can achieve high-precision micro-displacement and micro-rotation amplification measurement and alignment, but the optical path is complex and relatively difficult to use.

[0044] In contrast, passive moiré is generated through the interaction between periodic stripes on the target and the camera pixel array. This method is also called the sampling moiré method. As a non-contact high-precision measurement technology, the moiré sampling method excels in the measurement of micro-strain, micro-rotation and micro-displacement. Some researchers have proposed a moiré sampling method using arbitrary repeating pattern patterns. They innovatively describe the periodic pattern through multiple frequency cosine functions and (Fourier series). Experiments have shown that when using a rectangular grating as a pattern, it is sufficient to use only the first frequency for high-precision displacement measurement. This method can achieve fast and accurate in-plane displacement distribution measurement. On this basis, some researchers have further extended this method to large-scale structure 3D displacement measurement. Compared with the traditional uncalibrated sampling moiré method, the measurement accuracy in the x-direction is improved from 280μm to 3μm, and the measurement accuracy in the z-direction is improved from 60μm to 6μm. Furthermore, the whole pixel digital image correction (DIC) method is combined with the sampling mole method to improve the in-plane displacement measurement accuracy to 1 / 350 of the length of a periodic pattern. Some researchers have also proposed the use of the grid CCD mole method to achieve low-resolution images and high-processing speed full-field vibration displacement measurement, which can achieve efficient displacement amplification measurement. The sampling mole method has become an important micro-displacement measurement calibration method due to the characteristics of the measurement calibration optical path being the same as the actual optical path and the simple system structure.

[0045] However, on the one hand, the existing sampling moiré method still requires methods such as phase unwrapping to process the collected moiré images, which poses a challenge to the convenience of camera array debugging and calibration; on the other hand, in order to achieve a satisfactory magnification, a large number of pixel sampling target patterns are often required, which makes the sampling moiré method difficult to apply to telephoto lenses or small field of view imaging systems. To this end, the present application uses the aliasing stripes presented by the aliasing signal folding phenomenon that occurs when the sampling is close to the Nyquist condition to achieve the in-plane displacement reading of the sample sampling. Specifically, in each aliasing signal cycle, a gray aliasing stripe with a length of 1 or 2 pixels can be observed, and the other positions are the black and white pairs of the sample pattern sampled. At the same time, because the length of the aliasing signal is fixed with the magnification of the sampling interval, the sub-pixel displacement can be accurately determined by observing the number of pixels moved by the gray aliasing stripes in the image, and the reading accuracy is the inverse of the magnification, and the whole pixel displacement exceeding 1 pixel is achieved by designing a triangular mark pointing to the gap between the black and white signals.

[0046] In an exemplary embodiment, a camera array calibration device is provided, comprising: a sample pattern design module, a calibration target, a light source, a reflective optical collimator, and a measurement calibration module; the measurement calibration module is connected to a target camera array.

[0047] When the position of the target camera array is calibrated in this embodiment, the camera array calibration device used includes: a sample pattern design module, a calibration target and a measurement calibration module. The calibration target and the target camera array are arranged according to the optical path. The measurement calibration module is electrically connected to the target camera array. Figure 1 shown.

[0048] When the target camera array is calibrated at an angle in this embodiment, the camera array calibration device used includes: a sample pattern design module, a calibration target, a light source, a reflective optical collimator and a measurement calibration module, wherein the light source is arranged at the focus of the reflective optical collimator; the calibration target is arranged at the light outlet of the light source; the target camera array is arranged on the outgoing light path of the reflective optical collimator, that is, the light source, the calibration target, the reflective optical collimator and the camera array are arranged according to the light path, and the measurement calibration module is electrically connected to the target camera array, such as Figure 2 shown.

[0049] The sample pattern design module is used for:

[0050] The sample pattern is sampled by a sampling moiré method close to the Nyquist condition to obtain an aliased signal; the sampling moiré method close to the Nyquist condition is a sampling moiré method in which the difference between the spatial frequency of the sample pattern and the sampling frequency within a sampling range is within a set frequency range; the sample pattern is a periodic pattern.

[0051] The wave number of the aliased signal is calculated according to the number of sampling points in the measurement direction within the camera sample pattern image measurement area in the target camera array and the displacement measurement magnification of the target camera array.

[0052] The wave number of the sample pattern is calculated according to the wave number of the aliasing signal and the number of sampling points in the measurement direction within the image measurement area.

[0053] The target pattern is determined based on the wave number of the sample pattern; the target pattern includes: an in-plane displacement calibration pattern and an in-plane rotation calibration pattern; the in-plane displacement calibration pattern includes a positioning mark. Figure 3 As shown, the in-plane rotation calibration pattern is Figure 4 shown. Figure 3 The position pointed by the triangle positioning mark in the center of the chessboard pattern is the boundary between the black and white grids of the chessboard pattern. Figure 4 The rotation calibration pattern in the mid-plane is designed as vertical black and white stripes.

[0054] The pattern on the calibration target is the target pattern, that is, the function of the sample pattern design module is to design the pattern on the calibration target.

[0055] based on Figure 1The calibration device shown in the figure first performs an internal rotation angle adjustment before position calibration of the target camera array. At this time, the calibration target is a reflective target.

[0056] When adjusting the in-plane rotation angle before position calibration, the target camera array is used to shoot the in-plane rotation calibration pattern on the calibration target to obtain a first image. The measurement calibration module is used to determine the in-plane rotation angle image of the in-plane rotation calibration pattern relative to each sub-camera in the target camera array according to the first image to obtain a first in-plane rotation angle image, and calculate the in-plane rotation angle of each sub-camera relative to the calibration target according to the first in-plane rotation angle image to obtain a first in-plane rotation angle; the first in-plane rotation angle is used to adjust the in-plane rotation relationship between the calibration target and the target camera array perpendicular to the optical axis direction, so that the relative rotation angle between each sub-camera in the target camera array is within a set angle range, and the in-plane rotation angle adjustment before position calibration is completed.

[0057] When the target camera array is positionally calibrated, the target camera array is used to shoot the in-plane displacement calibration pattern on the calibration target to obtain a second image. The measurement calibration module is used to read the second image, determine the gray stripe position of the aliasing signal according to the second image to obtain the first gray stripe position, and determine the image in-plane displacement according to the first gray stripe position and the black and white pixels and positioning marks in the second image to obtain the first image in-plane displacement; the first image in-plane displacement is used to adjust the positional relationship between each sub-camera in the target camera array perpendicular to the optical axis direction to complete the position calibration of the target camera array.

[0058] based on Figure 2 The calibration device shown in the figure first performs an internal rotation angle adjustment before performing angle calibration on the target camera array. At this time, the calibration target is a transmission target.

[0059] When adjusting the internal rotation angle before angle calibration and calibrating the angle of the target camera array, the light source is set at the focus of the reflective optical collimator; the calibration target is set at the light outlet position of the light source; the target camera array is set on the output light path of the reflective optical collimator; and the light source is used to emit transmitted light.

[0060] When adjusting the internal rotation angle before angle calibration, the transmitted light is used to image through the in-plane rotation calibration pattern on the calibration target; the reflective optical collimator is used to collimate the light passing through the in-plane rotation calibration pattern on the calibration target to obtain a first parallel light; the target camera array is used to shoot the first parallel light to obtain a third image; the measurement and calibration module is used to determine the in-plane rotation angle image of the in-plane rotation calibration pattern relative to each sub-camera in the target camera array according to the third image to obtain a second in-plane rotation angle image, and calculate the in-plane rotation angle of each sub-camera relative to the calibration target according to the second in-plane rotation angle image to obtain a second in-plane rotation angle; the second in-plane rotation angle is used to adjust the rotation relationship between the calibration target and the target camera array, so that the relative rotation angle between each sub-camera in the target camera array is within a set angle range, thereby completing the internal rotation angle adjustment before angle calibration.

[0061] When the target camera array is angle-calibrated, the transmitted light is used to form an image through the in-plane displacement calibration pattern on the calibration target; the reflective optical collimator is used to collimate the light through the in-plane displacement calibration pattern on the calibration target to obtain a second parallel light; the target camera array is used to shoot the second parallel light to obtain a fourth image; the measurement calibration module is used to read the fourth image, determine the gray stripe position of the aliasing signal according to the fourth image to obtain the second gray stripe position, and determine the image in-plane displacement according to the second gray stripe position and the black and white pixels and positioning marks in the fourth image to obtain the second image in-plane displacement, calculate the camera optical axis deflection angle according to the second image in-plane displacement, and adjust the optical axis off-plane deflection angle between each sub-camera in the target camera array according to the camera optical axis deflection angle to complete the angle calibration of the target camera array.

[0062] In another exemplary embodiment of the present application, the sampling moiré method close to the Nyquist condition adopted in the sample pattern design module is first introduced.

[0063] This embodiment uses the spatial signal wave number model of Burger W. to describe the moiré sampling principle.

[0064] The number of sampling points corresponding to the one-dimensional camera CCD is set to M, and the wave number of the sample signal corresponding to the periodic sample pattern is set to m. In the discrete case, for each component with a wave number of m, there is a mirror copy with an equal size and a wave number of -m, which is manifested as the spectral components appearing in pairs and repeating symmetrically at multiples of M. Burger W. modeled the spectral components presented after the periodic continuous signal is uniformly discretely sampled as:

[0065]

[0066] Wherein, G(.) represents the signal power spectrum, k is a positive integer, and formula (1) describes that the cyclic shift periodic power spectrum or the flip symmetric power spectrum is the same as the original signal power spectrum.

[0067] According to formula (1), the low-frequency signal is displayed as an aliased signal. First, the sampling moiré method principle in formula (1) is expanded. When the sampling is insufficient, in addition to the moiré pattern due to the aliasing phenomenon, the aliased signal folding phenomenon will also occur when the sample frequency is close to the Nyquist condition. This phenomenon is often necessary to avoid in conventional processing methods, but it has an exciting advantage when processing periodic stripes or checkerboard target patterns. Therefore, based on formula (1), the formula for generating moiré patterns by the sampling moiré method close to the Nyquist condition designed in this embodiment can be described as:

[0068] |G(m)|=|G(2|1 / 2M-m|)| (2)

[0069] Using sinusoidal signals for analysis, in order to illustrate the principle of moiré produced by the sampling moiré method close to the Nyquist condition, the simulation scene is set to sample the vertically arranged periodic black and white lines with a sampling point column of 512 pixels in image height. The number of sampling points M = 512 corresponds to 512 pixels in the vertical direction of the camera, and the sampling period is T samp , the sample signal wave number m = 255 with a length of 2π, 255 corresponds to a total of m pairs of black and white lines in the vertical field of view of the camera, and the sample period is T spec For the convenience of representation, the grayscale image signal of the original sample discrete sampling is symmetrically normalized (Symmetric Normalization) to the interval [-1,1], -1 corresponds to black with a grayscale value of 0, and +1 corresponds to white with a grayscale value of 255. The sample sampling signal x[n] is expressed as:

[0070]

[0071] Where n is the sampling point number, and the continuous signal period corresponding to formula (3) is M / m, which is close to 2. The original sampling signal is divided into odd sampling signals x according to the odd and even numbers. odd [k] and even-numbered sampled signal x even [k]:

[0072]

[0073] Where k is the sequence number of the newly extracted sampling signal. The newly extracted sequence can be understood as the original sample signal with a wave number of m, which is sampled twice by M / 2 sampling points. Corresponding to the situation described in formula (1), the wave number of the low-frequency aliasing signal fitting function corresponding to the two equations in formula (4) is The low frequency aliasing signal can be expressed as:

[0074]

[0075] t represents the actual length of the displacement calibration pattern measurement area. The odd and even sampling signals of the sample wave number m = 255 correspond to one cycle of negative sine signal and one cycle of sine signal respectively; the odd and even sampling signals of the sample wave number m = 257 correspond to one cycle of sine signal and one cycle of negative sine signal respectively. There are M / 2 sampling points for each, and the sampling period is 2T samp The aliased signal of the original sample sampling signal after the odd and even signals are recombined is a pair of positive and negative sinusoidal signals sampled alternately. Because a period of sinusoidal signal is anti-symmetric about the center, the envelope wave number corresponding to the original sample sampling signal after the odd and even signals are recombined is It is manifested as M / (2|1 / 2M-m in one aliasing signal cycle. spec |) = 256 sampling points, corresponding to the aliased signal moving magnification s = M, (2|1 / 2M-m spec |).

[0076] According to formula (3), the image normalized signal amplitude corresponding to the sampling points with serial numbers 1 and M / 2+1 is 0. When the corresponding sampling point is located exactly between a pair of black and white patterns of the sample pattern, the sample sampling appears gray, and the stripe with the amplitude closest to 0 in an aliasing cycle is named gray stripe. If the grayscale of two adjacent stripes cannot be distinguished, the average of the serial numbers of the two sampling points is taken as the current gray stripe serial number reading.

[0077] Different from the traditional method that needs to process phase information to encode position information, this embodiment only needs to read the number of sampling points of the current gray stripe offset within the aliasing period to determine the sub-pixel displacement. The reading accuracy is the inverse of the displacement measurement magnification 1 / s = (2|1 / 2M-m spec |) / M. That is, the gray stripes observed in the image are displaced by one sampling point, and the corresponding sampling point actually moves T relative to the sample pattern. samp / s.

[0078] Furthermore, the gray stripe is designed to be shifted by 5 sampling points with a length of 5×(2π / M) for simulation. The corresponding sampling points are actually moved by 5×(2π / M) / s relative to the sample pattern, that is, the sampling sequence in formula (3) is moved backward and forward by 5 / s times the sampling period to form new sampling sequences u1 and u2. The new sample sampling is:

[0079]

[0080] In summary, through the sampling Moore method close to the Nyquist sampling condition, the displacement of the image sub-pixel scale can be determined by low-frequency information (the position of the gray stripes of the aliased signal), and the displacement of the pixel scale can be determined by high-frequency information (black and white pixels obtained by sample sampling) combined with positioning marks (in this embodiment, a triangle shape pointing to the boundary of the black and white sample pattern is used). The combination of the two can directly determine the actual displacement between images by reading the pixel coordinates of the positioning marks and gray stripes of different images.

[0081] Therefore, the sample pattern can be designed according to the required relative displacement magnification and test area size. alias =2|1 / 2M-m spec |The design sample signal wave number is:

[0082]

[0083] If the displacement measurement magnification is designed to be s, and the number of pixels covering one aliasing cycle length is M, the wave number m of the sample pattern covering one aliasing cycle length can be determined. spec , thus the style design of the sample pattern can be realized according to the sampling Moore method close to the Nyquist condition.

[0084] Based on the above introduction to the sampling moiré method close to the Nyquist condition and the simulation process, the present embodiment further describes the process of designing a target pattern by the sample pattern design module.

[0085] The calculation formula of the wave number of the aliased signal is: alias =M / s; where m alias represents the wave number of the aliased signal; M represents the number of sampling points in the measurement direction within the camera sample pattern image measurement area in the target camera array; s represents the displacement measurement magnification of the target camera array.

[0086] The calculation formula of the wave number of the sample pattern is: spec =1 / 2|M±m alias |; Among them, m spec Indicates the wave number of the sample pattern.

[0087] In terms of determining the target pattern based on the wave number of the sample pattern, the sample pattern design module is specifically used to: determine the actual target pattern size based on the focal length and object distance of the target camera array; calculate the period length of the target pattern based on the actual target pattern size and the wave number of the sample pattern; and determine the target pattern based on the period length of the target pattern.

[0088] In another exemplary embodiment of the present application, in terms of determining the in-plane displacement of an image according to the position of the first gray stripe and the black and white pixels and positioning marks in the second image to obtain the in-plane displacement of the first image, the measurement and calibration module is specifically used to: determine the displacement of the image at a sub-pixel scale according to the position of the first gray stripe to obtain a first displacement; determine the displacement of the pixel scale according to the black and white pixels and positioning marks in the second image to obtain a second displacement; determine a first actual displacement according to the first displacement and the second displacement, and determine the first actual displacement as the in-plane displacement of the first image.

[0089] In another exemplary embodiment of the present application, in terms of determining the in-plane displacement of the image according to the position of the second gray stripe and the black and white pixels and positioning marks in the fourth image to obtain the in-plane displacement of the second image, the measurement and calibration module is specifically used to: determine the displacement of the image at a sub-pixel scale according to the position of the second gray stripe to obtain a third displacement; determine the displacement of the pixel scale according to the black and white pixels and positioning marks in the fourth image to obtain a fourth displacement; determine a second actual displacement according to the third displacement and the fourth displacement, and determine the second actual displacement as the in-plane displacement of the second image.

[0090] In another exemplary embodiment of the present application, in calculating the camera optical axis deflection angle according to the displacement in the second image plane, the measurement calibration module is specifically used to: according to the formula Calculate the camera optical axis deflection angle; β represents the camera optical axis deflection angle; d col represents the displacement in the second image plane; f col Indicates the focal length of the reflective optical collimator.

[0091] In another exemplary embodiment of the present application, the reflective optical collimator may be a reflective concave mirror; the light source may be an integrating sphere; and the measurement calibration module may be a computer. The integrating sphere is placed at the focus of the reflective optical collimator, the reflective optical collimator provides parallel light, and the camera array shoots the parallel light emitted by the reflective optical collimator. If the optical axes of the sub-cameras within the aperture of the reflective optical collimator are parallel, the images captured are exactly the same. The measurement experiment process of the above-mentioned camera array calibration device is given below to illustrate the effectiveness of the device.

[0092] This embodiment realizes efficient camera image in-plane displacement calibration based on the sampling moiré method close to the Nyquist condition. An experiment is designed and implemented, in which the camera array is mounted on a three-axis translation stage, and the camera array is controlled to move horizontally relative to the target to perform sub-pixel micro-displacement measurement. The designed target is a 320mm×256mm ChArUco target (occupying 640×512 pixels), and a sample pattern of 25mm×25mm in size is designed and drawn in a 30mm×30mm black checkerboard in the target (occupying 50×50 pixels). The RMS of the in-plane displacement measurement of the sample pattern experimental image is 0.009mm, which is higher than the RMS of 0.026mm measured using the overall ChArUco target. This embodiment also reprinted the camera on the stage to perform an optical axis rotation measurement experiment through a reflective optical collimator. Only a 15mm×15mm test pattern occupies 50×50 pixels to realize optical axis rotation measurement, and the RMS is 0.0001°. This embodiment has the characteristics of high magnification, strong robustness, simple system construction and operation, and provides an efficient and robust solution for high-precision calibration of sparse camera array systems. Specifically:

[0093] (1) Experiment on displacement measurement within the camera image plane.

[0094] A camera mounted on a translation stage was designed to shoot the designed target. The experiment used a Goldeye CL-033TEC1 short-wave infrared (SWIR) camera with a resolution of 640×512 pixels and a pixel size of 15μm. The focal length of the lens (KowaLM50HC-SW) was 50 mm. The horizontal field of view of the camera HFOV = 10.97°, and the vertical field of view VFOV = 8.78°. The repeatability of the translation stage is 5 microns.

[0095] The camera is mounted on the translation stage, and the camera optical axis is adjusted to be perpendicular to the lifting and horizontal movement direction of the translation stage. The target is placed toward the camera, and the target position and direction are adjusted until the image area of ​​the target with a size of 320mm×256mm is exactly within the camera field of view. At this time, the translation stage controls the camera to move 0.5mm in the horizontal direction perpendicular to the optical axis, which corresponds to a 1 pixel image movement.

[0096] (2) Design of in-plane displacement calibration pattern.

[0097] The test target is designed based on the ChArUco target. The target image area is 320mm×256mm, and the sampling period of the camera CCD back-projection onto the target is T samp=0.5mm, corresponding to an object distance of 1666.67mm. The ChArUco target image area is 300×240mm and is located in the center of the image area. The checker size is 30mm, the marker size is 22mm, and the dictionary used is AruCoDIT_7×7.

[0098] Select one of the black squares and draw the image plane displacement calibration pattern in it. Figure 3 As shown. To prevent the effect of ChArUco target from being affected, the image plane displacement calibration pattern is designed to be 25mm×25mm in size and drawn in the center of a black chessboard on the ChArUco target. This design ensures that the ChArUco target and the image plane displacement calibration pattern can be imaged at the same time, which is used for the relative in-plane displacement measurement of the camera relative to the target image.

[0099] Principle of displacement calibration pattern in the image plane: The 20mm×20mm area in the center of the sample pattern is designed to correspond to 40×40 pixels as the test area. In order to achieve the displacement measurement magnification of s=20, 40 pixels can contain two aliasing cycles. A single aliasing cycle is T alias =sT samp =10mm, including the number of sampling points M=20, according to formula (7) the sample wave number is designed to be m specS =10.5 or m specL =9.5 can achieve the designed displacement measurement magnification. specS =10.5 for test area pattern design T specS =952μm corresponds to the width of a pair of black and white grids in the test area. specL =9.5 as the pattern design of the remaining part of the verification area outside the test area, T specL =1053μm corresponds to the width of a pair of black and white grids in the calibration area, so that the designed sample pattern has the function of calibrating the relationship between focal length and object distance. 20mm can draw 21 pairs of black and white grids in the test area, and 19 pairs of black and white grids can be drawn within the corresponding length of the calibration area. Draw white triangles at the center positions of the four sides of the sample pattern and the positions corresponding to the boundaries of the test area as positioning marks to assist in determining the displacement of the sample pattern at the whole pixel scale, such as Figure 3 shown.

[0100] The imaging results of the camera under different displacement conditions relative to the sample pattern are simulated. The simulation results show that the current design only uses 50×50 pixels corresponding to the 25mm×25mm sample pattern, which can achieve 20 times the translation magnification measurement of two aliasing cycles and take into account the verification function of the object distance (focal length). The direct reading accuracy is as high as 0.05 pixels, and the focal length or object distance can be assisted by observing the number of aliasing cycles in the test area and the verification area.

[0101] The sample pattern of the sampling moiré method designed in this embodiment close to the Nyquist condition can realize high-precision image displacement and in-plane displacement calibration, and can also be used to assist in determining the relationship between object distance and focal length. And the sample pattern wave number calculated by formula (7) has nothing to do with the actual sampling period, that is, no matter how the design scale changes, as long as the sample pattern wave number and the number of sampling points are close to the Nyquist condition, the same in-plane displacement magnification will be generated. Therefore, if the design magnification remains unchanged, the current sample wave number can be directly applied to the in-plane displacement calibration pattern design in the subsequent camera optical axis calibration experiment.

[0102] However, like all sampling moiré methods, the in-plane rotation of the CCD relative to the sample pattern will also cause the image to change, which in turn affects the calibration of the image in-plane displacement. Therefore, before performing the image in-plane displacement calibration, the image in-plane rotation calibration needs to be performed first.

[0103] (3) Rotate the calibration pattern within the image plane.

[0104] The image rotation angle calibration is performed using a vertical black and white stripe pattern target. The method is as follows: after the sample stripes are projected onto the CCD, the sample stripes rotate in-plane, causing the stripe width components in the horizontal and vertical directions to change, thereby causing the aliasing period of each row and column to change. Furthermore, the aliased stripes change their position row by row and column by column relative to the rotation center, ultimately presenting aliased stripes that are amplified by the rotation angle caused by the rotation of the CCD relative to the sample pattern.

[0105] The sampling period and wave number are M and T respectively samp , the sample period is T spec , when the target rotates α relative to the CCD, the component of the sample fringe projection period in the horizontal direction is T specX =T spec ×1 / cosα, the vertical component is T specY =T spec ×1 / sinα. The corresponding sample wave number horizontal component is The vertical component is Because of design mspecX near M / 2 , so the horizontal aliasing wave number is The aliasing period is The final rotation angle of the aliased fringes is:

[0106]

[0107] The black and white stripes corresponding to the displacement calibration pattern in the plane are analyzed by formula (8). From the analysis results, it can be seen that when the rotation angle is small, the curve can be approximated as a direct proportional function, T specS and TspecL The corresponding image plane rotation angle magnifications are 10.4 times and 9.4 times respectively. specS Design an image in-plane rotation calibration pattern. According to the simulation results, the aliasing fringe rotation angle is 10.42 times when the CCD rotates relative to the sample pattern. Using an in-plane rotation calibration pattern that is easier to observe, first calibrate the in-plane rotation of the CCD relative to the target, and then calibrate and measure the in-plane displacement, which can reduce the impact of in-plane rotation on in-plane displacement measurement.

[0108] In addition to the in-plane rotation, the effect of the sample pattern out-of-plane rotation on the in-plane displacement measurement is also analyzed. When the sample pattern out-of-plane rotation angle changes by β relative to the target facing the CCD, the different distances of the near end point and the far end point relative to the CCD may cause the in-plane displacement calibration pattern to change. The near end point sample period is The far end point sample period is The horizontal field of view of the camera is HFOV = 10.97°, and the vertical field of view is VFOV = 8.78°. The field of view of the test area (40×40 pixels) of the in-plane displacement calibration pattern is 0.43°. When the pitch of the target changes by 10°, T specN =0.999T spec , T specF =1.001T spec The change is less than 3‰, which is not enough to affect the use of the in-plane displacement measurement pattern due to changes in distance.

[0109] Therefore, the displacement calibration pattern and rotation calibration pattern designed can achieve 20 times in-plane displacement magnification and 10.42 times in-plane rotation angle magnification respectively. And under the current design, the target's out-of-plane rotation change within 10° does not affect the use of the in-plane displacement calibration pattern and the in-plane rotation calibration pattern. It is relatively easy to control the target's out-of-plane rotation to less than 10°, which makes this method have good robustness. In actual target use, it is necessary to first control the target's out-of-plane rotation, then adjust the object distance and in-plane rotation to the appropriate range, and finally perform in-plane displacement calibration and measurement.

[0110] (4) Camera optical axis direction calibration experiment.

[0111] The optical axis direction calibration of sub-cameras in sparse camera array systems has important application significance. The function of the optical collimation system is to measure and calibrate the deviation between the camera optical axis and the optical axis of the collimation system itself. It can be used to calibrate the optical axis deviation between each camera in a sparse camera array. The measurement and calibration principle is: the optical axis of the camera to be calibrated has an inclination angle β relative to the optical axis of the collimation system, resulting in a camera optical axis offset d on the camera CCD. cam , corresponding to the camera optical axis pointing to the collimation system target position offset d col The relationship between the three satisfies the formula:

[0112]

[0113] Among them, f cam and f col Respectively represent the focal length of the camera and the optical collimation system (reflective optical collimator). When the angle β satisfies the small angle approximation condition, the radian value of the optical axis deflection angle and the corresponding tangent value can be considered to be approximately equal, then:

[0114]

[0115] That is, through the optical collimation system, the small-scale deflection angle of the camera optical axis (corresponding to d cam ) is converted to amplification f col / f cam times the image plane displacement d col , which can be obtained by d col Calculate the camera optical axis offset angle β.

[0116] However, the calibration accuracy of the optical collimation system is still restricted by the pixel size of the camera to be tested and the aperture of the focal light source of the optical collimator, making it difficult to achieve d col Sub-pixel calibration.

[0117] Based on the OpTest optical test system, a camera optical axis calibration experiment was designed. The pixel size of the camera to be calibrated is 15μm, the focal length design value is 150mm, and the deflection angle corresponding to a single pixel is 0.0057°. The designed calibration target is used with an integrating sphere (with a light output diameter of 40mm) to replace the original OG-1000 target generator. The camera to be tested is camera No. 4 in the sparse camera array system. The focal length of the reflective optical collimator in the system is 3000mm, and the focal length of the calibration camera lens is designed to be 150mm. The sampling interval corresponding to the back projection of the 15μm camera pixel to the target position is 0.3mm.

[0118] Therefore, new in-plane displacement calibration patterns and in-plane rotation calibration patterns are designed, and the size of both patterns is 15mm×15mm corresponding to 50×50 pixels. specS =10.5 and m specL =9.5 as the sample wave number of the test area and the sample wave number of the calibration area respectively, and the sample period of the test area of ​​the corresponding in-plane displacement calibration pattern is T specS =571μm, the sample period of the verification area is T specS = 631 μm, and the sample period of the in-plane rotation calibration pattern is also T specS =571 μm. Transmission targets use opaque triangles as positioning marks.

[0119] During the experiment, the LP-1000 off-axis optical platform with a resolution of up to 0.1 second of arc was used to control the 4th camera in the sparse camera array system to produce a horizontal optical axis deflection, with each rotation of 0.0010° and a total rotation of 0.0300°. The readings of the OpTest optical test system were used as the actual optical axis deflection values, with a total of 31 sets. The in-plane displacement calibration pattern images collected by the 4th camera were then read to obtain the in-plane displacement readings of the designed calibration target. Finally, the in-plane displacement was converted into an optical axis deflection angle using formula (10) and compared with the actual optical axis deflection value.

[0120] According to the experimental results, the aliasing period of the measurement area of ​​the measured in-plane displacement calibration pattern is 19, and the gray stripe period of the calibration area is 21. Therefore, it is estimated that the actual focal length of the camera lens should be about 150.45mm (between 150mm and 150.6mm). After the experiment, the focal length of the lens was measured by the transfer function instrument to be 150.49mm. Therefore, the magnification of the corrected in-plane displacement calibration pattern was 19 for reading measurement. According to the reading results, the RMS of 31 sets of optical axis deflection angle measurement values ​​was calculated to be 0.0001°, which broke through the limitation of the deflection angle size corresponding to a single pixel under the experimental conditions, achieved higher-precision optical axis deflection angle measurement, and provided a methodological basis for the optical axis direction calibration of high-precision sparse camera array cameras.

[0121] The sampling mole method is widely used in scenarios such as micro deformation and micro displacement measurement due to its advantages in micro displacement amplification. In this embodiment, based on the aliasing signal folding phenomenon that occurs when approaching the Nyquist condition, combined with the periodicity of the sampling target, a sampling mole method close to the Nyquist condition is proposed, which can simultaneously capture high-frequency sample sampling signals and low-frequency aliasing stripes. Under this condition, the sub-pixel displacement between the two images can be accurately determined only by reading the acquired pattern image. Therefore, the method of this embodiment can be used to directly determine the relative in-plane displacement of the camera relative to the target sample pattern, and can be applied to an optical collimator to realize the calibration of the optical axis direction of the camera. The method of this embodiment has the accuracy of achieving n times magnification using n pixel arrangements, the convenience of direct reading without post-processing, and the robustness of strong resistance to light influence. In the camera shooting target experiment, the test pattern of size 25×25mm occupies 50×50 pixels to achieve a displacement measurement RMS of 0.009mm (single pixel corresponds to a displacement of 0.5mm). In the camera optical axis measurement calibration experiment, a test pattern with a size of 15×15mm occupies 50×50 pixels, and the RMS of the optical axis direction angle measurement is 0.0001° (a single pixel corresponds to a rotation angle of 0.0057°).

[0122] Based on the same inventive concept, the embodiment of the present application also provides a camera array calibration method for implementing the camera array calibration device involved above. The implementation solution provided by the method to solve the problem is similar to the implementation solution recorded in the above device, so the specific limitations in one or more camera array calibration method embodiments provided below can refer to the limitations on the camera array calibration device above, and will not be repeated here.

[0123] In an exemplary embodiment, a camera array calibration method is provided, comprising:

[0124] (1) Before calibrating the position of the target camera array, first adjust the internal rotation angle before the position calibration.

[0125] ① When adjusting the front internal rotation angle of the position calibration, read the first image, determine the in-plane rotation angle image of the in-plane rotation calibration pattern relative to each sub-camera in the target camera array based on the first image, and obtain the first in-plane rotation angle image; calculate the in-plane rotation angle of each sub-camera relative to the calibration target based on the first in-plane rotation angle image to obtain the first in-plane rotation angle; the first in-plane rotation angle is used to adjust the in-plane rotation relationship between the calibration target and the target camera array perpendicular to the optical axis direction, so that the relative rotation angle between each sub-camera in the target camera array is within the set angle range, and the front internal rotation angle adjustment of the position calibration is completed; the first image is obtained by photographing the in-plane rotation calibration pattern on the calibration target by the target camera array.

[0126] ② When calibrating the position of the target camera array, read the second image, determine the gray stripe position of the aliasing signal according to the second image to obtain the first gray stripe position, and determine the image plane displacement according to the first gray stripe position and the black and white pixels and positioning marks in the second image to obtain the first image plane displacement; the first image plane displacement is used to adjust the position relationship between each sub-camera in the target camera array perpendicular to the optical axis direction to complete the position calibration of the target camera array; the second image is obtained by shooting the in-plane displacement calibration pattern on the calibration target by the target camera array.

[0127] (2) Before performing angle calibration on the target camera array, first perform an internal rotation angle adjustment before the angle calibration.

[0128] ① When adjusting the internal rotation angle before angle calibration, read the third image, determine the in-plane rotation angle image of the in-plane rotation calibration pattern relative to each sub-camera in the target camera array based on the third image, and obtain the second in-plane rotation angle image, and calculate the in-plane rotation angle of each sub-camera relative to the calibration target based on the second in-plane rotation angle image to obtain the second in-plane rotation angle; the second in-plane rotation angle is used to adjust the rotation relationship between the calibration target and the target camera array, so that the relative rotation angle between each sub-camera in the target camera array is within the set angle range, and the internal rotation angle adjustment before angle calibration is completed; the third image is obtained by shooting the first parallel light by the target camera array; the first parallel light is obtained by collimating the light passing through the in-plane rotation calibration pattern on the calibration target by a reflective optical collimator.

[0129] ② When the target camera array is angle-calibrated, the fourth image is read, the gray stripe position of the aliased signal is determined according to the fourth image to obtain the second gray stripe position, and the image plane displacement is determined according to the second gray stripe position and the black and white pixels and positioning marks in the fourth image to obtain the second image plane displacement, the camera optical axis deflection angle is calculated according to the second image plane displacement, and the optical axis off-plane deflection angle between each sub-camera in the target camera array is adjusted according to the camera optical axis deflection angle to complete the angle calibration of the target camera array; the fourth image is obtained by the target camera array shooting the second parallel light; the second parallel light is obtained by the reflective optical collimator to collimate the light passing through the in-plane displacement calibration pattern on the calibration target.

[0130] Wherein, the pattern on the calibration target is a target pattern, and the target pattern is determined by using a sample pattern design module. The method for determining the target pattern includes:

[0131] (1) A sample pattern is sampled using a sampling moiré method close to the Nyquist condition to obtain an aliased signal; the sampling moiré method close to the Nyquist condition is a sampling moiré method in which the difference between the spatial frequency of the sample pattern and the sampling frequency within a sampling range is within a set frequency range; and the sample pattern is a periodic pattern.

[0132] (2) Calculating the wave number of the aliased signal according to the number of sampling points in the measurement direction within the camera sample pattern image measurement area in the target camera array and the displacement measurement magnification of the target camera array.

[0133] (3) Calculating the wave number of the sample pattern according to the wave number of the aliasing signal and the number of sampling points in the measurement direction within the image measurement area.

[0134] (4) Determining a target pattern based on the wave number of the sample pattern; the target pattern comprises: an in-plane displacement calibration pattern and an in-plane rotation calibration pattern; the in-plane displacement calibration pattern comprises a positioning mark.

[0135] This application proposes a camera calibration scheme based on the sampling Moore method close to the Nyquist condition. By reading the collected sample patterns, the in-plane displacement of the image between different sample patterns is determined. Theoretically, the number of pixels in the measurement area determines the maximum displacement measurement magnification, and the scheme has a simple system construction method and strong robustness. Through experimental verification, the camera shooting target experiment achieved an in-plane displacement measurement RMS of 0.009mm. The camera optical axis measurement calibration experiment achieved an optical axis direction angle measurement RMS of 0.0001°. It is proved that this scheme can provide a convenient calibration and measurement scheme by designing target patterns, which can break through the constraints of pixel size on the accuracy of the method in conventional target calibration methods, and at the same time provide a high-precision and high-robust method basis for the high-precision calibration of sparse camera array systems.

[0136] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0137] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A camera array calibration device, characterized in that: The camera array calibration device comprises: a sample pattern design module, a calibration target, a light source, a reflective optical collimator and a measurement calibration module; the measurement calibration module is connected to the target camera array; The sample pattern design module is used for: The sample pattern is sampled by using a sampling moiré method close to the Nyquist condition to obtain an aliased signal; the sampling moiré method close to the Nyquist condition is a sampling moiré method in which the difference between the spatial frequency of the sample pattern and the sampling frequency within the sampling range is within a set frequency range; the sample pattern is a periodic pattern; Calculate the wave number of the aliased signal according to the number of sampling points in the measurement direction within the camera sample pattern image measurement area in the target camera array and the displacement measurement magnification of the target camera array; Calculating the wave number of the sample pattern according to the wave number of the aliased signal and the number of sampling points in the measurement direction within the image measurement area; Determine a target pattern based on the wave number of the sample pattern; the target pattern comprises: an in-plane displacement calibration pattern and an in-plane rotation calibration pattern; the in-plane displacement calibration pattern comprises a positioning mark; The pattern on the calibration target is the target pattern; Before position calibration of the target camera array, the internal rotation angle adjustment is first performed before position calibration; When adjusting the in-plane rotation angle before position calibration, the target camera array is used to shoot the in-plane rotation calibration pattern on the calibration target to obtain a first image; the measurement calibration module is used to determine the in-plane rotation angle image of the in-plane rotation calibration pattern relative to each sub-camera in the target camera array according to the first image to obtain a first in-plane rotation angle image, and calculate the in-plane rotation angle of each sub-camera relative to the calibration target according to the first in-plane rotation angle image to obtain a first in-plane rotation angle; the first in-plane rotation angle is used to adjust the in-plane rotation relationship between the calibration target and the target camera array perpendicular to the optical axis direction, so that the relative rotation angle between each sub-camera in the target camera array is within a set angle range, and the in-plane rotation angle adjustment before position calibration is completed; When the target camera array is positionally calibrated, the target camera array is used to shoot the in-plane displacement calibration pattern on the calibration target to obtain a second image; the measurement calibration module is used to read the second image, determine the gray stripe position of the aliasing signal according to the second image to obtain the first gray stripe position, and determine the image in-plane displacement according to the first gray stripe position and the black and white pixels and positioning marks in the second image to obtain the first image in-plane displacement; the first image in-plane displacement is used to adjust the positional relationship between each sub-camera in the target camera array perpendicular to the optical axis direction to complete the position calibration of the target camera array; Before the angle calibration of the target camera array is performed, the internal rotation angle adjustment before the angle calibration is performed; when the internal rotation angle adjustment before the angle calibration and the angle calibration of the target camera array are performed, the light source is set at the focus of the reflective optical collimator; the calibration target is set at the light outlet position of the light source; the target camera array is set on the output light path of the reflective optical collimator; the light source is used to emit transmitted light; When adjusting the internal rotation angle before angle calibration, the transmitted light is used to image through the in-plane rotation calibration pattern on the calibration target; the reflective optical collimator is used to collimate the light passing through the in-plane rotation calibration pattern on the calibration target to obtain a first parallel light; the target camera array is used to shoot the first parallel light to obtain a third image; the measurement and calibration module is used to determine the in-plane rotation angle image of the in-plane rotation calibration pattern relative to each sub-camera in the target camera array according to the third image to obtain a second in-plane rotation angle image, and calculate the in-plane rotation angle of each sub-camera relative to the calibration target according to the second in-plane rotation angle image to obtain a second in-plane rotation angle; the second in-plane rotation angle is used to adjust the rotation relationship between the calibration target and the target camera array so that the relative rotation angle between each sub-camera in the target camera array is within a set angle range, thereby completing the internal rotation angle adjustment before angle calibration; When the target camera array is angle-calibrated, the transmitted light is used to form an image through the in-plane displacement calibration pattern on the calibration target; the reflective optical collimator is used to collimate the light through the in-plane displacement calibration pattern on the calibration target to obtain a second parallel light; the target camera array is used to shoot the second parallel light to obtain a fourth image; the measurement calibration module is used to read the fourth image, determine the gray stripe position of the aliasing signal according to the fourth image to obtain the second gray stripe position, and determine the image in-plane displacement according to the second gray stripe position and the black and white pixels and positioning marks in the fourth image to obtain the second image in-plane displacement, calculate the camera optical axis deflection angle according to the second image in-plane displacement, and adjust the optical axis off-plane deflection angle between each sub-camera in the target camera array according to the camera optical axis deflection angle to complete the angle calibration of the target camera array.

2. The camera array calibration device according to claim 1, characterized in that: The calculation formula of the wave number of the aliased signal is: m alias =M / s; Among them, m alias represents the wave number of the aliased signal; M represents the number of sampling points in the measurement direction within the camera sample pattern image measurement area in the target camera array; s represents the displacement measurement magnification of the target camera array.

3. The camera array calibration device according to claim 1, characterized in that: The calculation formula of the wave number of the sample pattern is: mspec = 1 / 2M ± malias |; Where mspec represents the wave number of the sample pattern; M represents the number of sampling points in the measurement direction of the camera sample pattern image measurement area in the target camera array; m alias Indicates the wave number of the aliased signal.

4. The camera array calibration device according to claim 1, characterized in that: In terms of determining the target pattern based on the wave number of the sample pattern, the sample pattern design module is specifically used to: Determine the actual target pattern size based on the focal length and object distance of the target camera array; Calculating the period length of the target pattern according to the actual target pattern size and the wave number of the sample pattern; The target pattern is determined based on the period length of the target pattern.

5. The camera array calibration device according to claim 1, characterized in that: In terms of determining the displacement in the image plane according to the position of the first gray stripe and the black and white pixels and positioning marks in the second image to obtain the displacement in the first image plane, the measurement and calibration module is specifically used for: Determine the displacement of the image at a sub-pixel scale according to the position of the first gray stripe to obtain a first displacement; Determine a displacement amount in pixel scale according to black and white pixels and positioning marks in the second image to obtain a second displacement amount; A first actual displacement is determined according to the first displacement amount and the second displacement amount, and the first actual displacement is determined as a first image plane in-plane displacement.

6. The camera array calibration device according to claim 1, characterized in that: In terms of determining the displacement in the image plane according to the position of the second gray stripe and the black and white pixels and positioning marks in the fourth image to obtain the displacement in the second image plane, the measurement and calibration module is specifically used for: Determine the displacement of the image at a sub-pixel scale according to the position of the second gray stripe to obtain a third displacement; Determine a displacement amount of a pixel scale according to black and white pixels and the positioning mark in the fourth image to obtain a fourth displacement amount; A second actual displacement is determined according to the third displacement amount and the fourth displacement amount, and the second actual displacement is determined as a second image plane in-plane displacement.

7. The camera array calibration device according to claim 1, characterized in that: In terms of calculating the camera optical axis deflection angle according to the displacement in the second image plane, the measurement and calibration module is specifically used for: According to the formula Calculate the camera optical axis deflection angle; β represents the camera optical axis deflection angle; d col represents the displacement in the second image plane; f col Indicates the focal length of the reflective optical collimator.

8. The camera array calibration device according to claim 1, characterized in that: The reflective optical collimator is a reflective concave mirror.

9. The camera array calibration device according to claim 1, characterized in that: The light source is an integrating sphere.

10. A camera array calibration method, characterized in that: The camera array calibration method is used in the camera array calibration device according to any one of claims 1 to 9, and the camera array calibration method comprises: Before position calibration of the target camera array, the internal rotation angle adjustment is first performed before position calibration; When adjusting the front internal rotation angle of the position calibration, a first image is read, and an in-plane rotation angle image of the in-plane rotation calibration pattern relative to each sub-camera in the target camera array is determined according to the first image to obtain a first in-plane rotation angle image; and the in-plane rotation angle of each sub-camera relative to the calibration target is calculated according to the first in-plane rotation angle image to obtain a first in-plane rotation angle; the first in-plane rotation angle is used to adjust the in-plane rotation relationship between the calibration target and the target camera array perpendicular to the optical axis direction, so that the relative rotation angle between each sub-camera in the target camera array is within a set angle range, and the front internal rotation angle adjustment of the position calibration is completed; the first image is obtained by photographing the in-plane rotation calibration pattern on the calibration target by the target camera array; When the target camera array is positionally calibrated, the second image is read, the gray stripe position of the aliased signal is determined according to the second image to obtain the first gray stripe position, and the image plane displacement is determined according to the first gray stripe position and the black and white pixels and positioning marks in the second image to obtain the first image plane displacement; the first image plane displacement is used to adjust the positional relationship between each sub-camera in the target camera array perpendicular to the optical axis direction to complete the position calibration of the target camera array; the second image is obtained by the target camera array shooting the in-plane displacement calibration pattern on the calibration target; Before performing angle calibration on the target camera array, first perform the internal rotation angle adjustment before the angle calibration; When adjusting the internal rotation angle before angle calibration, read the third image, determine the in-plane rotation angle image of the in-plane rotation calibration pattern relative to each sub-camera in the target camera array according to the third image, obtain the second in-plane rotation angle image, and calculate the in-plane rotation angle of each sub-camera relative to the calibration target according to the second in-plane rotation angle image to obtain the second in-plane rotation angle; the second in-plane rotation angle is used to adjust the rotation relationship between the calibration target and the target camera array, so that the relative rotation angle between each sub-camera in the target camera array is within the set angle range, and the internal rotation angle adjustment before angle calibration is completed; the third image is obtained by shooting the first parallel light by the target camera array; the first parallel light is obtained by collimating the light passing through the in-plane rotation calibration pattern on the calibration target by a reflective optical collimator; When the target camera array is angle-calibrated, the fourth image is read, the gray stripe position of the aliased signal is determined according to the fourth image to obtain the second gray stripe position, and the image plane displacement is determined according to the second gray stripe position and the black and white pixels and positioning marks in the fourth image to obtain the second image plane displacement, the camera optical axis deflection angle is calculated according to the second image plane displacement, and the optical axis off-plane deflection angle between each sub-camera in the target camera array is adjusted according to the camera optical axis deflection angle to complete the angle calibration of the target camera array; the fourth image is obtained by the target camera array shooting the second parallel light; the second parallel light is obtained by the reflective optical collimator collimating the light passing through the in-plane displacement calibration pattern on the calibration target; Wherein, the pattern on the calibration target is a target pattern, and the target pattern is determined by using a sample pattern design module. The method for determining the target pattern includes: The sample pattern is sampled by using a sampling moiré method close to the Nyquist condition to obtain an aliased signal; the sampling moiré method close to the Nyquist condition is a sampling moiré method in which the difference between the spatial frequency of the sample pattern and the sampling frequency within the sampling range is within a set frequency range; the sample pattern is a periodic pattern; Calculate the wave number of the aliased signal according to the number of sampling points in the measurement direction within the camera sample pattern image measurement area in the target camera array and the displacement measurement magnification of the target camera array; Calculating the wave number of the sample pattern according to the wave number of the aliased signal and the number of sampling points in the measurement direction within the image measurement area; A target pattern is determined based on the wave number of the sample pattern; the target pattern comprises: an in-plane displacement calibration pattern and an in-plane rotation calibration pattern; and the in-plane displacement calibration pattern comprises a positioning mark.

Images