A large field of view camera defocus calibration method based on point spread function modeling

Abstract

The method belongs to the field of machine vision, and particularly relates to a large-view-field camera defocus calibration method based on point spread function modeling, which is used for calibrating a large-view-field imaging industrial camera under defocus condition. The method uses the gray centroid of the light transmission coefficient as the best object side matching point corresponding to the image sensor pixel point under defocus condition based on the established point spread function model, and uses an improved Zhang Zhengyou calibration method to calculate the internal and external parameters of the camera through the matched point pairs, so as to realize the calibration of the camera parameters when the camera shoots the calibration target under defocus condition. The method solves the requirement of accurately calibrating the parameters of the large-view-field imaging industrial camera under the optical three-dimensional measurement scene of large aviation structural parts, and calibrates when the camera shoots the calibration target under defocus condition. Compared with the traditional checkerboard calibration method and the camera defocus calibration method based on phase matching, the method has higher calibration precision and has practical application value.

Description

Technical Field

[0001] This invention relates to a defocus calibration method for a large field-of-view camera based on point spread function modeling. The method uses the gray-scale centroid of the light transmission coefficient as the optimal object-space matching point for the image sensor pixels, and then uses these object-image point pairs to complete the camera calibration under defocus conditions. This invention belongs to the field of machine vision. Background Technology

[0002] In camera perspective projection models, camera intrinsic parameters and distortion coefficients are crucial parameters describing the transformation relationship from the camera coordinate system to the image pixel coordinate system, forming the basis for tasks such as 3D vision measurement. The calibration accuracy of camera intrinsic parameters and distortion coefficients directly determines the accuracy of subsequent 3D reconstruction. Therefore, calibrating high-precision camera parameters under various scenarios is a fundamental and important research topic.

[0003] When calibrating industrial cameras with large field of view using traditional camera calibration methods, two conditions must be met: (1) the calibration target must cover the field of view of the large field of view camera, and (2) the camera must be able to capture a clear image of the target. However, when calibrating industrial cameras with large field of view imaging, using large calibration targets is expensive and difficult to manufacture; using a single small target, when covering the camera's field of view, often results in the camera being unable to capture a clear image of the target due to defocus. This problem makes the traditional method of calibrating large field of view cameras inaccurate. Summary of the Invention

[0004] To address the lack of a universal calibration method in the calibration process of large field-of-view imaging industrial cameras, this invention proposes a large field-of-view camera defocus calibration method based on point spread function modeling. The gray centroid of the light transmission coefficient is used as the optimal object-space matching point corresponding to the pixel of the image sensor when the camera is out of focus, thereby realizing the calibration of camera parameters when the camera is out of focus.

[0005] The basic principle of this invention is to calculate the gray-scale centroid of the light transmission coefficient as the optimal object-space matching point for an image sensor pixel. In defocus shooting, the object point and image point are no longer in a one-to-one correspondence; the light received by an image point will come from a small area of ​​the object space. In this case, not all light rays conform to the perspective projection model. Experiments have demonstrated that the gray-scale centroid of the light transmission coefficient can be used as the optimal object-space matching point corresponding to an image sensor pixel. Therefore, the centroid of the light transmission coefficient of an image point can be calculated and matched with the pixel. Using the matched point pairs, the camera's intrinsic and extrinsic parameters are calculated using an improved Zhang Zhengyou calibration algorithm, completing the defocus calibration of a large field-of-view imaging industrial camera.

[0006] The technical problem solved by this invention is the defocus calibration problem of large field-of-view imaging industrial cameras.

[0007] The technical solution of this invention is: a defocus calibration method for a large field-of-view camera based on point spread function modeling, characterized in that the process includes the following steps:

[0008] (1) Select the working distance and fix the camera. Use an LCD monitor as a calibration target, place it in different positions, and use the camera to photograph the target. At this time, the result of the camera photographing the monitor is an out-of-focus image;

[0009] (2) Perform pixel sampling on the target image captured in step (1), selecting a pixel at certain intervals, and selecting tens or hundreds of pixels;

[0010] (3) Project the fringe pattern using the multi-frequency structured light heterodyne phase unwrapping method, and calculate the display coordinates (x, y) corresponding to the pixel points (u, v) sampled in step (2). u ,y v ), completing the initial positioning;

[0011] (4) Assume that the range of the display area Ω corresponding to the light received by an image sensor pixel does not exceed m×n, and determine the display area Ω corresponding to the pixel point according to the initial value of the display coordinate point calculated in step (3);

[0012] (5) In the local region Ω determined in step (4), display the horizontal and vertical local slice stripes with length and width of m×n, take pictures of the corresponding horizontal and vertical slice stripes with the camera, and calculate the one-dimensional Fourier coefficients of the light transmission coefficient h(x,y;u,v) of the screen area corresponding to the camera pixel (u,v) in the horizontal and vertical projections.

[0013] (6) Perform a one-dimensional inverse Fourier transform on the one-dimensional Fourier coefficients obtained in step (5) to obtain the projection integral of the optical transmission coefficient on the x-axis and y-axis, and calculate the gray centroid of the optical transmission coefficient as the best object-space matching point corresponding to the image sensor pixel.

[0014] (7) Using the object-image point pairs obtained in step (6), the initial parameters of the camera are calculated by Zhang Zhengyou calibration method, and then iterative optimization is performed by removing gross error points.

[0015] In step (3), the heterodyne phase unwrapping and phase matching of multi-frequency structured light require the display of four-step phase shifting fringes on an LCD screen. In multi-frequency structured light, three-frequency is a fast and highly accurate method, and it is widely used. Therefore, three-frequency four-step phase shifting fringes are generally used for heterodyne synthesis and phase unwrapping.

[0016] In step (4), the range of the display area Ω is m×n, and generally, m = n can be taken. At this time, the starting point coordinates of the local slice stripe corresponding to (u,v) can be expressed as...

[0017]

[0018] Here, `round()` is a function that rounds the data to the nearest integer. It roughly locates the sub-pixel coordinates (x, y) of the display. u After (x, yv), the starting coordinates of the local slice stripes are obtained by rounding (x, yv). u-init， yv-init) is the precise coordinates relative to the display.

[0019] In step (5), the camera captures the corresponding horizontal and vertical stripe patterns. The responses of the camera pixels (u,v) to the horizontal and vertical stripe patterns are as follows:

[0020]

[0021] Where h(x,y;u,v) represents the light transmission coefficient of each point on the screen corresponding to the camera pixel (u,v), and R n This indicates the camera's response to ambient light.

[0022] The one-dimensional Fourier coefficients of the light transmission coefficient h(x,y; u,v) of the screen area corresponding to the camera pixel (u,v) projected horizontally and vertically, respectively, can be expressed as:

[0023]

[0024] In step (6), performing a one-dimensional inverse Fourier transform on the one-dimensional Fourier coefficients yields:

[0025] ∫h(x,y;u,v)dy=IFT[H(u,v;f x ,0)]

[0026] ∫h(u,v;u,v)dx=IFT[H(u,v;0,f y )]

[0027] Where ∫h(x,y;u,v)dy is the projection integral of the light transmission coefficient of pixel (u,v) on the x-axis, and ∫h(x,y;u,v)dx is the projection integral of the light transmission coefficient of pixel (u,v) on the y-axis. The calculated gray-level centroid is the object-space matching point of the image.

[0028] The advantages of this invention compared to the prior art are:

[0029] (1) This method can be used for calibration when shooting out of focus.

[0030] (2) This method verifies that the gray centroid of the optical transmission coefficient is the optimal object-space matching point of the image sensor pixel.

[0031] (3) Compared with the camera calibration method based on phase matching and the calibration method based on checkerboard, this method calculates the projection of the light transmission coefficient in two directions by measurement, calculates its centroid as the best object space estimation point of the image point, and calculates the initial parameters of the camera through these points, and then performs iterative optimization to solve the problem, which improves the calibration accuracy of the camera under defocus conditions. Attached Figure Description

[0032] Figure 1 This is a flowchart of the method of the present invention.

[0033] Figure 2 This is a diagram of the experimental setup used in the actual calibration experiment. The setup includes the camera to be calibrated, a gimbal, an LCD monitor, a control computer, and a tripod. Detailed Implementation

[0034] To better understand the present invention, the technical solution of the present invention will be described in detail below with reference to the accompanying drawings and examples.

[0035] See attached document Figure 1 , 2 A method for defocus calibration of a large field-of-view camera based on point spread function modeling includes the following steps:

[0036] 1. Fix the camera on a tripod and gimbal, and take pictures of the LCD screen from different positions to obtain images. Select position 9 for taking pictures.

[0037] 2. Sample the pixels of the captured image, selecting one pixel at regular intervals, specifically tens to hundreds. Use phase matching, employing multi-frequency heterodyne, specifically three-frequency four-step phase-shifting fringes, to calculate the continuous phase map after horizontal and vertical phase expansion, establishing the coordinate mapping relationship between the camera and the display.

[0038] Multi-frequency structured light heterodyne phase unwrapping and phase matching require displaying four phase shift fringes on an LCD screen:

[0039]

[0040] The corresponding camera response R i (u,v) is:

[0041]

[0042] The phase of each point in the image can be calculated as follows:

[0043]

[0044] The calculated phase varies periodically within the range (-π, π). To obtain a continuous phase map for matching, we performed heterodyne synthesis and phase unrolling using fringe patterns of various frequencies, thus obtaining a continuous phase map. use:

[0045] φ(u,v)=Φ(x,y)

[0046] It can match image points and LCD coordinate points. The formula for calculating the corresponding LCD screen coordinates from image coordinates (u,v) is as follows:

[0047]

[0048] Where λ is the spatial period of the stripes.

[0049] 3. Based on the initial object point value determined in step 2 for each sampled pixel, calculate the display area corresponding to the pixel. Apply the Fourier local slicing method to display local stripes, and obtain the pixel's response to the horizontal and vertical local stripes respectively. Then, the one-dimensional Fourier coefficients of the light transmission coefficient h(x,y; u,v) of the screen area corresponding to the camera pixel (u,v) can be obtained by projecting them horizontally and vertically.

[0050] 4. Calculate the projection integral of the optical transmission coefficient on the x and y axes using the one-dimensional Fourier coefficients obtained in step 3. This method of calculating the projection integral of the optical transmission coefficient, rather than calculating the two-dimensional distribution of the optical transmission coefficient, will greatly shorten the measurement time. Calculate the gray-level centroid based on the slice projection to obtain the object point corresponding to the camera pixel.

[0051] 5. In the calibration stage, the calibration calculation is performed using the object-image point pairs obtained in step 4, based on Zhang Zhengyou's calibration algorithm. In traditional methods, the object point coordinates are fixed on the target and the number is determined, requiring the calculation of the corresponding image point position. However, in this method, the image point coordinates can be freely selected, and the corresponding object point coordinate position is calculated; therefore, no optimization of the object point coordinates is performed.

[0052] 6. In step 5, since the object point coordinates are not optimized, based on the 3σ criterion, image points and object points with gross reprojection errors are removed, and iterative optimization is performed until no gross errors remain. Because the image sensor selected in this method has a large number of pixels, even after removing some points based on the 3σ criterion, a good calibration result can still be obtained. The benchmark for calibration optimization is to optimize the camera's intrinsic and extrinsic parameters and distortion parameters to minimize the reprojection error.

[0053]

[0054] in, The reprojected pixel coordinates are calculated using the homography matrix, mi These are the actual pixel coordinates. A is the intrinsic parameter matrix. R and t are the camera extrinsic parameters, and the rest are the camera distortion parameters.

Claims

1. A method for defocus calibration of a large field-of-view camera based on point spread function modeling, characterized in that, The process includes the following steps: (1) Select the working distance, fix the camera, use the LCD display as the calibration target, place it in different positions, and use the camera to shoot the target. At this time, the camera will shoot the display image as a defocused image. (2) Perform pixel sampling on the target image captured in step (1), selecting a pixel at certain intervals, and selecting tens to hundreds of pixels; (3) Project the fringe pattern using the multi-frequency structured light heterodyne phase unwrapping method, and calculate the display coordinates (x, y) corresponding to the pixel points (u, v) sampled in step (2). u ,y v ), completing the initial positioning; (4) Assume that the range of the display area Ω corresponding to the light received by an image sensor pixel does not exceed m×n, and determine the display area Ω corresponding to the pixel point according to the initial value of the display coordinate point calculated in step (3); (5) In the local region Ω determined in step (4), display the horizontal and vertical local slice stripes with length and width of m×n, take pictures of the corresponding horizontal and vertical slice stripes with the camera, and calculate the one-dimensional Fourier coefficients of the light transmission coefficient h(x,y;u,v) of the screen area corresponding to the camera pixel (u,v) in the horizontal and vertical projections. (6) Perform a one-dimensional inverse Fourier transform on the one-dimensional Fourier coefficients obtained in step (5) to obtain the projection integral of the optical transmission coefficient on the x-axis and y-axis, and calculate the gray centroid of the optical transmission coefficient as the best object-space matching point corresponding to the image sensor pixel. (7) Using the object-image point pairs obtained in step (6), the initial parameters of the camera are calculated by Zhang Zhengyou calibration method, and then iterative optimization is performed by removing gross error points.

2. The method for defocus calibration of a large field-of-view camera based on point spread function modeling according to claim 1, characterized in that: In (4) described above, the transverse and longitudinal local slice stripes are Where (x,y) represents the screen coordinates, (f x ,f y ) represents the transverse and longitudinal spatial frequencies of the sinusoidal base pattern, where the transverse stripes satisfy f x =i / m,i=0,1,2,…,m-1,f y =0; the vertical stripes satisfy f y =i / n,j=0,1,2,…,n-1,f x =0; m, n are small regions Ω m×n The horizontal and vertical resolution, The initial phase value of the pattern is represented by , a is the average brightness of the striped fringes, and b is the amplitude of the striped fringes; for each spatial frequency (f... x f y The phase is displayed on the screen respectively. The four-step phase-shift fringes, the range of x is [x...]. u-init ,x u-init +m-1], the range of y is [y v-init y v-init +n-1], and set the brightness value of the pattern at the remaining positions to 0.

3. The method for defocus calibration of a large field-of-view camera based on point spread function modeling according to claim 1, characterized in that: In step (6), the gray-scale centroid of the optical transmission coefficient is used as the optimal object-space matching point corresponding to the image sensor pixel. The coordinates of the gray-scale centroid are... The calculation method is as follows Where h(x,y,z;u,v) is the light transmission coefficient corresponding to an image sensor pixel, and Ω represents a small area on the display corresponding to a camera pixel.

Images