A camera calibration method and system under non-uniform illumination conditions

Abstract

The application discloses a camera calibration method and system under uneven illumination conditions, and belongs to the technical field of image processing, and comprises the following steps: S1, processing an input image by using a multi-scale Retinex algorithm; S2, performing Gaussian homomorphic filtering processing on the image obtained in the step S1; S3, processing the image obtained in the step S2 by using an Otsu algorithm; and S4, performing camera calibration on the image processed in the steps S1-S3 by using Zhang Zhengyou plane calibration method. The application firstly enhances image details by using the multi-scale Retinex algorithm and the Gaussian homomorphic filtering, then obtains a final calibration board image by combining Otsu algorithm image segmentation and bilateral filtering processing, and further realizes high-precision calibration of the camera, and is worth popularization and use.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image processing, in particular to a camera calibration method and system under non-uniform illumination conditions. BACKGROUND

[0002] Three-dimensional reconstruction has wide applications in computer-aided design, graphics, animation, computer vision, medical image reconstruction, digital multimedia, industrial reverse design, cultural relics museum and digital city, and has been a hot research topic in the field of computer vision. As one of the core and key aspects of three-dimensional reconstruction, non-uniform illumination increases the difficulty of image processing and makes the reconstruction ineffective.

[0003] In high-precision camera calibration, the surface of the calibration template is required to be smooth and flat, and the material for making the calibration template is often ceramic material, glass material or aluminum alloy material. A plurality of calibration plate images are shot from different angles and at different distances. Due to the influence of environmental light, non-uniform illumination and the smooth and flat surface of the above-mentioned materials, the actual collected calibration plate images often have non-uniform image gray scale, local gray scale is too bright or too dark, and even local highlight. These non-uniform illumination conditions will reduce the precision of feature extraction such as edge and corner, and further affect the precision of the reconstructed three-dimensional geometric model and the effect of the final texture mapping. It can be seen that non-uniform illumination is one of the most direct and challenging problems in three-dimensional reconstruction. Understanding the complex illumination conditions and improving the precision of camera calibration to achieve high-quality three-dimensional reconstruction have important theoretical significance and practical application value. Therefore, a camera calibration method and system under non-uniform illumination conditions are proposed. SUMMARY

[0004] The technical problem to be solved by the present application is how to solve the problems of non-uniform illumination and low contrast of the calibration plate image under weak light or non-uniform illumination conditions, and the problem of low camera calibration precision under complex illumination conditions. A camera calibration method under non-uniform illumination conditions is provided. The method uses a multi-scale Retinex algorithm and a Gaussian homomorphic filter to enhance image details, combines Otsu algorithm image segmentation and bilateral filter processing to obtain the final calibration plate image, and realizes high-precision camera calibration.

[0005] The present application solves the above technical problems by the following technical solutions. The present application comprises the following steps:

[0006] S1: processing the input image by using a multi-scale Retinex algorithm;

[0007] S2: performing Gaussian homomorphic filter processing on the image obtained in step S1;

[0008] S3: processing the image obtained in step S2 by using an Otsu algorithm;

[0009] S4: using Zhang Zhengyou plane calibration method to calibrate the camera after the image processed by steps S1-S3.

[0010] Further, in the step S1, the expression of the input image is:

[0011] z(x, y) = r(x, y) * i(x, y)

[0012] Converting to the logarithmic domain is:

[0013] log[r(x, y)] = log[z(x, y)] - log[i(x, y)]

[0014] Wherein, i(x, y) is the illumination component determined by the light source, and r(x, y) represents the reflection component of i(x, y).

[0015] Further, in the step S1, the multi-scale Retinex algorithm is developed on the basis of the single-scale Retinex algorithm, and different scale parameter values are used for convolution filtering of the input image, and then the weighted average of the filtering results obtained by different scale parameters is obtained. The expression of the illumination image is:

[0016]

[0017] Wherein, i∈(R, G, B), G j (x, y) represents a Gaussian filtering function, and the scale parameter is σ j , n is the number of scale parameters, W j is the weight corresponding to the jth filtering function, and satisfies

[0018] Further, the expression of the single-scale Retinex algorithm is:

[0019]

[0020] Wherein, i∈(R, G, B), G(x, y) represents a Gaussian central surrounding function, and the mathematical expression is:

[0021]

[0022] Wherein, (x, y) is the point coordinate, exp is the exponential function with natural constant e as the base, and σ is the standard deviation.

[0023] Further, in the step S2, the image r i (x, y) obtained in step S1 is expressed as the product of the illumination component i(x, y) and the reflection component r(x, y), that is:

[0024] r i (x,y) = i(x,y)*r(x,y).

[0025] Further, the step S2 comprises the following steps:

[0026] S21: taking the logarithm of both sides of the expression of the image r i (x,y) obtained in step S1, and multiplying the two components into addition, obtaining:

[0027] ln[r i (x,y)] = ln[i(x,y)*r(x,y)] = ln[i(x,y)] + ln[r(x,y)]

[0028] Taking Fourier transform of both sides:

[0029] F{ln[r i (x,y)]} = F{ln[i(x,y)]} + F{ln[r(x,y)]}

[0030] Simplify as:

[0031] R i (u,v) = I(u,v) + R(u,v);

[0032] S22: designing a homomorphic filter H(u,v) to weaken the low-frequency components and enhance the high-frequency components in the logarithmic domain, and the filtered result is expressed as:

[0033] S(u,v) = H(u,v)R i (u,v) = H(u,v)*[I(u,v) + R(u,v)]

[0034] Taking inverse Fourier transform of the filtered result S(u,v), obtaining:

[0035] s(x,y) = F -1 [S(u,v)]

[0036] Taking exponential transform of s(x,y), obtaining the homomorphic filtered image:

[0037] g(x,y) = exp[s(x,y)].

[0038] Further, in the step S22, the transfer function of the homomorphic filter H(u,v) is as follows:

[0039]

[0040] wherein γ L <1, γ H >1, and c is used to adjust the function at γL and gamma H steepness.

[0041] Further, in the step S3, in the Otsu algorithm, for a gray image I, the number of pixel points is N, the binarization threshold t is set to segment the current image, the pixel points with the gray value greater than t are foreground pixel points, the pixel points with the gray value less than t are background pixel points, the number of foreground pixel points is N0, the number of background pixel points is N1, the proportion of the foreground pixel points of the image I in the whole image is ω0, the average gray value is μ0, the proportion of the background pixel points in the whole image is ω1, the average gray value is μ1, and the inter-class variance g is calculated according to the following formula:

[0042] g = ω0ω1(μ0-μ1) 2

[0043] Then the binarization threshold is traversed from 0 to 255, the image is binarized and segmented, the binarization threshold t corresponding to the maximum g is the optimal threshold, and the image obtained in the step S2 is segmented by using the optimal threshold.

[0044] Further, in the step S4, the description of the imaging process by the linear imaging model of the camera is based on the world coordinate system, the camera coordinate system, the image physical coordinate system and the image pixel coordinate system, the homogeneous coordinates of a point P in the world coordinate system are set as [X w Y w Z w 1] T , the coordinates of the rigid body in the camera coordinate system are [X c Y c Z c ] T , the coordinates in the image imaging coordinate system after perspective projection are [x y z] T , and finally the corresponding pixel coordinates in the image are [u v]; the conversion relationship between the world coordinate system and the image coordinate system is obtained through the conversion between the four coordinate systems:

[0045]

[0046]

[0047] wherein, f x , f y , u0 and v0 are parameters of the camera intrinsic matrix H1, f x , f y are the normalized focal lengths in the X-axis and Y-axis of the pixel coordinate system respectively, u0 and v0 are the center pixel coordinates of the image, k u , k vrespectively, are scale factors on X-axis and Y-axis in pixel coordinate system;

[0048] The conversion relationship between the world coordinate system and the pixel coordinate system is represented as:

[0049]

[0050] wherein, Z c is the Z-axis coordinate value in the camera coordinate system, R and T represent rigid body transformation; H is a homography matrix, including a camera intrinsic matrix H1 and an extrinsic matrix H2.

[0051] Further, in the step S4, the pixel coordinates (u, v) of each corner point on the checkerboard calibration board in the image are obtained by using a corner point extraction algorithm, the world coordinate system is fixed on the checkerboard, the Z-axis coordinate value of the world coordinate system of any point on the checkerboard is 0, and then the coordinates of each corner point in the world coordinate system are calculated, the camera is calibrated by using the pixel coordinates of each corner point and the coordinates of each corner point in the world coordinate system, and the intrinsic and extrinsic matrices and the distortion parameters of the camera are obtained.

[0052] The application further provides a camera calibration system under non-uniform illumination conditions, which calibrates the camera by using the above method, and comprises:

[0053] The first processing module is used for processing the input image by using a multi-scale Retinex algorithm.

[0054] The second processing module is used for performing Gaussian homomorphic filtering processing on the image obtained in the step S1.

[0055] The third processing module is used for processing the image obtained in the step S2 by using an Otsu algorithm.

[0056] The calibration module is used for calibrating the camera by using Zhang Zhengyou plane calibration method on the image processed in the steps S1-S3.

[0057] The first processing module, the second processing module and the third processing module are connected with the calibration module.

[0058] Compared with the prior art, the camera calibration method and system under non-uniform illumination conditions have the following advantages: the multi-scale Retinex algorithm and the Gaussian homomorphic filtering are used to enhance the image details, then the Otsu algorithm image segmentation and the bilateral filtering processing are combined to obtain the final calibration board image, and then the high-precision calibration of the camera is realized, so that the method is worth popularizing and using. BRIEF DESCRIPTION OF DRAWINGS

[0059] Figure 1 is a flow chart of the camera calibration method under complex illumination conditions in the embodiment one of the application;

[0060] Figure 2 is the original image of the calibration board under weak light in the second embodiment of the present application;

[0061] Figure 3 is the image of the original image of the calibration board under weak light in the second embodiment of the present application processed by the multi-scale Retinex algorithm, Gaussian homomorphic filtering and Otsu algorithm;

[0062] Figure 4 is the original image of the calibration board under non-uniform light in the second embodiment of the present application;

[0063] Figure 5 is the image of the original image of the calibration board under non-uniform light in the second embodiment of the present application processed by the multi-scale Retinex algorithm, Gaussian homomorphic filtering and Otsu algorithm. DETAILED DESCRIPTION

[0064] The embodiments of the present application are described in detail below, which are implemented on the premise of the technical scheme of the present application, and detailed implementation modes and specific operation processes are given, but the protection scope of the present application is not limited to the following embodiments.

[0065] As shown in Figure 1 , the present embodiment provides a technical scheme: a camera calibration method under non-uniform light, which specifically adopts the following main steps:

[0066] S1: processing the input image by the multi-scale Retinex algorithm;

[0067] S2: processing the image obtained in step S1 by Gaussian homomorphic filtering;

[0068] S3: processing the image obtained in step S2 by the Otsu algorithm;

[0069] S4: calibrating the camera by Zhang Zhengyou plane calibration method on the image processed by steps S1-S3.

[0070] The modes adopted in each step are further described as follows:

[0071] S1, multi-scale Retinex algorithm processing

[0072] Suppose z(x, y) represents the image function, i(x, y) is the illumination component determined by the light source, and r(x, y) represents the reflection component of i(x, y), which is determined by the intrinsic properties of the imaged object. Assuming that the image seen by the human eye is z(x, y), the expression of the original image seen by the human eye is:

[0073] z(x, y) = r(x, y) * i(x, y) (1)

[0074] Converting to the logarithmic domain is:

[0075] log[r(x,y)] = log[z(x,y)] - log[i(x,y)] (2)

[0076] Single-Scale-Retinex algorithm (Single-Scale-Retinex) from the process of human eye vision, the RGB component of the original image is described respectively for the center, convolution processing of the surrounding parameters, in order to estimate the light image;

[0077] The specific expression is:

[0078]

[0079] Where, i∈(R,G,B), G(x,y) represents the Gaussian center around function, the mathematical expression is:

[0080]

[0081] Multi-Scale-Retinex algorithm (Multi-Scale-Retinex) is developed on the basis of Single-Scale-Retinex algorithm, using the only variable in the formula, that is, the scale parameter, different values are used for convolution filtering of the image, and the weighted average of the filtering results obtained by different scale parameters is obtained. The expression of the light image is:

[0082]

[0083] Where, i∈(R,G,B), G j (x,y) represents the Gaussian filter function, the scale parameter is σ j , n is the number of scale parameters, W j is the weight corresponding to the jth filter function, which satisfies

[0084] S2, Gaussian homomorphic filtering

[0085] Homomorphic filtering is a frequency domain filtering method, which can compress the overall brightness of the image, make the image light uniform, and enhance the image details. Homomorphic filtering is based on the object illumination reflection model; the image r i (x,y) can be expressed as the product of the illumination component i(x,y) and the reflection component r(x,y), that is:

[0086] r i (x,y) = i(x,y)*r(x,y) (6)

[0087] The illumination component is determined by the light source, describes the illumination of the image, and is expressed as a low-frequency component in the frequency domain after Fourier transform; the reflection component represents image information and is related to material reflectivity, and changes with the change of image details, and is expressed as a high-frequency component in the frequency domain. Therefore, the high-pass filter is designed in the application, the low-frequency information is reduced, the high-frequency information is reserved, the uneven illumination condition is eliminated, and the image details are highlighted.

[0088] First, logarithmic transformation is carried out on both sides of the above formula, multiplication of two components is changed into addition, high and low frequencies are separated in the frequency domain, that is:

[0089] ln[r i (x,y)]=ln[i(x,y)*r(x,y)]=ln[i(x,y)]+ln[r(x,y)] (7)

[0090] Fourier transform is carried out on both sides:

[0091] F{ln[r i (x,y)]}=F{ln[i(x,y)]}+F{ln[r(x,y)]} (8)

[0092] Simplified as:

[0093] R i (u,v)=I(u,v)+R(u,v) (9)

[0094] Next, a homomorphic filter H(u,v) is designed, low-frequency components are weakened and high-frequency components are enhanced in the logarithmic domain, so that the illumination component is weakened and the reflection component is enhanced, and the filtering result is expressed as follows:

[0095] S(u,v)=H(u,v)R i (u,v)=H(u,v)*[I(u,v)+R(u,v)] (10)

[0096] Fourier inverse transform is carried out on the filtering result S(u,v), and the following is obtained:

[0097] s(x,y)=F -1 [S(u,v)] (11)

[0098] Exponential transformation is carried out on s(x,y), and the homomorphic filtered image is obtained:

[0099] g(x,y)=exp[s(x,y)] (12)

[0100] The homomorphic filter function H(u,v) is essentially a high-pass filter, and the transfer function of the Gaussian homomorphic filter selected in the application is:

[0101]

[0102] wherein γ L <1, γ H >1, c is used to adjust the steepness of the function at γ L and γ H , the smaller c is, the more gently the amplitude rises.

[0103] S3, Otsu algorithm processing

[0104] The Otsu algorithm is a segmentation algorithm that can adaptively determine a binary threshold, and the basic idea is to divide the pixel points in the image into foreground and background according to the binary threshold, and the threshold corresponding to the maximum inter-class variance of the two types of pixels is the optimal threshold. The background of the calibration plate used in the present application is white, and the frame is black, and the gray value difference between the two is large, so the binary segmentation can effectively segment out the calibration plate region.

[0105] For a gray-scale image I, the number of pixel points is N, and the binary threshold t is set to segment the current image. The pixel points with a gray value greater than t are foreground pixel points, and the pixel points with a gray value less than t are background pixel points. The number of foreground pixel points is N0, the number of background pixel points is N1, the proportion of the number of foreground pixel points of the image I in the whole image is called ω0, the average gray value is μ0, the proportion of the number of background pixel points in the whole image is ω1, and the average gray value is μ1. The inter-class variance g is calculated according to the following formula:

[0106] g = ω0ω1(μ0-μ1) 2 (14)

[0107] Next, the binary threshold is traversed from 0 to 255, and the image is binary segmented, and the binary threshold t corresponding to the maximum g is the optimal threshold.

[0108] S4, camera calibration processing

[0109] The Zhang Zhengyou plane calibration method is used for camera calibration. The linear imaging model of the camera describes the imaging process based on the world coordinate system, the camera coordinate system, the image physical coordinate system and the image pixel coordinate system. Let the homogeneous coordinates of a point P in the world coordinate system be [X w Y w Z w 1] T , the coordinates of the rigid body transformation into the camera coordinate system are [X c Y c Z c ] T , and the coordinates of the perspective projection into the image imaging coordinate system are [x y z] T, and finally projected to the corresponding pixel coordinate in the image is [u v]. Through the conversion between the four coordinate systems, the conversion relationship between the world coordinate system and the image coordinate system can be obtained:

[0110]

[0111]

[0112] In the formula, f x , f y , u0, v0 are the parameters of the camera intrinsic matrix H1, f x , f y are the normalized focal lengths on the X-axis and Y-axis in the pixel coordinate system, k u , k v are the scale factors on the X-axis and Y-axis in the pixel coordinate system.

[0113] In summary, the conversion relationship between the world coordinate system and the pixel coordinate system can be expressed as:

[0114]

[0115] In the formula: Z c is the Z-axis coordinate value in the camera coordinate system, R and T represent the rigid body transformation; H is the homography matrix, including the camera intrinsic matrix H1 and the extrinsic matrix H2.

[0116] In the step S4, first, the pixel coordinates (u, v) of each corner point (the corner point is the point where the black squares on the checkerboard intersect) on the checkerboard calibration board in the image are obtained by using a corresponding corner point extraction algorithm. The world coordinate system is fixed on the checkerboard, and then the Z-axis coordinate value of the world coordinate system of any point on the checkerboard is 0. Since the world coordinate system of the calibration board is artificially defined in advance, the size of each grid on the calibration board is known, and the coordinates of each corner point in the world coordinate system can be calculated. We will use this information: the pixel coordinates of each corner point, the coordinates of each corner point in the world coordinate system to calibrate the camera, and obtain the intrinsic and extrinsic parameters of the camera and the distortion parameters.

[0117] Example Two

[0118] In this embodiment, the method in Example One is used to calibrate the original images of the calibration board under two conditions in Figure 2 and Figure 4 , and the calibration results obtained are shown in Table 1 below:

[0119] Table 1 Comparison of calibration results under two conditions

[0120]

[0121] In summary, the camera calibration method under uneven illumination conditions of the above-mentioned embodiments first uses a multi-scale Retinex algorithm and a Gaussian homomorphic filter to enhance image details, then combines Otsu algorithm image segmentation and bilateral filtering to obtain the final calibration board image, and then realizes high-precision camera calibration, which is worth popularizing.

[0122] Although the embodiments of the present application have been shown and described above, it should be understood by those skilled in the art that the above-mentioned embodiments are exemplary and cannot be understood as limiting the present application, and those skilled in the art can make changes, modifications, replacements and variations to the above-mentioned embodiments within the scope of the present application.

Claims

1. A camera calibration method under non-uniform illumination conditions, characterized in that, Includes the following steps: S1: Process the input image using the multi-scale Retinex algorithm; S2: Perform Gaussian homomorphic filtering on the image obtained in step S1; S3: Process the image obtained in step S2 using the Otsu algorithm; S4: The camera is calibrated using the Zhang Zhengyou planar calibration method on the images processed by steps S1 to S3. In step S1, the expression for the input image is: z(x,y)=r(x,y)*i(x,y) Transforming to the logarithmic field yields: log[r(x,y)]=log[z(x,y)]-log[i(x,y)] Where i(x,y) is the illumination component determined by the light source, and r(x,y) represents the reflection component of i(x,y); In step S1, the multi-scale Retinex algorithm is developed based on the single-scale Retinex algorithm. It uses different scale parameter values ​​to perform convolution filtering on the input image, and then weights the filtering results from different scale parameters to obtain a lit image, the expression of which is: Where i∈(R,G,B), G j (x, y) represents the Gaussian filter function with a scaling parameter σ. j n is the number of scale parameters, W j The weights corresponding to the j-th filter function satisfy... The expression for the single-scale Retinex algorithm is: Where i∈(R,G,B), G(x,y) represents the Gaussian center wrap function, and its mathematical expression is: Where (x,y) are the coordinates of the point, exp is the exponential function with the natural constant e as the base, and σ is the standard deviation; In step S2, the image r obtained in step S1 is... i (x,y) can be expressed as the product of the illumination component i(x,y) and the reflection component r(x,y), that is: r i (x,y)=i(x,y)*r(x,y); Step S2 includes the following steps: S21: For the image r obtained in step S1 i Taking a logarithmic transformation on both sides of the (x,y) expression, turning the multiplication of the two components into addition, we get: ln[r i (x,y)]=ln[i(x,y)*r(x,y)]=ln[i(x,y)]+ln[r(x,y)] Perform Fourier transform on both sides: F{ln[r i (x,y)]}=F{ln[i(x,y)]}+F{ln[r(x,y)]} Abbreviated as: R i (u,v)=I(u,v)+R(u,v); S22: Design a homomorphic filter H(u,v) to reduce low-frequency components and enhance high-frequency components in the logarithmic domain. The expression for the filtering result is as follows: S(u,v)=H(u,v)R i (u,v)=H(u,v)*[I(u,v)+R(u,v)] Performing an inverse Fourier transform on the filtered result S(u,v), we obtain: s(x,y)=F -1 [S(u,v)] Performing an exponential transform on s(x,y) yields the homomorphically filtered image: g(x,y)=exp[s(x,y)].

2. The camera calibration method under non-uniform illumination conditions according to claim 1, characterized in that: In step S3, in the Otsu algorithm, for a grayscale image I with N pixels, a binarization threshold t is set to segment the current image. Pixels with grayscale values ​​greater than t are foreground pixels, and pixels with grayscale values ​​less than t are background pixels. The number of foreground pixels is N0, and the number of background pixels is N1. The proportion of foreground pixels in image I to the entire image is ω0, and the average grayscale value is μ0. The proportion of background pixels to the entire image is ω1, and the average grayscale value is μ1. The inter-class variance g is calculated according to the following formula: g=ω0ω1(μ0-μ1) 2 Then, the binarization threshold is iterated from 0 to 255 to perform image binarization segmentation. The binarization threshold t corresponding to the maximum g is the optimal threshold. The image obtained in step S2 is segmented using the optimal threshold.

3. The camera calibration method under non-uniform illumination conditions according to claim 2, characterized in that: In step S4, the linear imaging model of the camera describes the imaging process based on the world coordinate system, camera coordinate system, image physical coordinate system, and image pixel coordinate system. Let the homogeneous coordinates of a point P in space in the world coordinate system be [X...]. w Y w Z w 1] T The coordinates of the rigid body transformed into the camera coordinate system are [X... c Y c Z c ] T Then, the perspective projection onto the image's imaging coordinate system yields coordinates [xyz]. T Finally, the corresponding pixel coordinates projected onto the image are [uv]; the transformation relationship between the world coordinate system and the image coordinate system is obtained through the transformation between the four coordinate systems: Among them, f x f y u0, v0 are the parameters of the camera intrinsic parameter matrix H1, f x f y These are the normalized focal lengths on the X and Y axes of the pixel coordinate system, respectively; u0 and v0 are the center pixel coordinates of the image; k u k v These are the scale factors on the X and Y axes of the pixel coordinate system, respectively; The transformation relationship between the world coordinate system and the pixel coordinate system is expressed as follows: Among them, Z c R represents the Z-axis coordinate value in the camera coordinate system, and R and T represent rigid body transformations; H is the homography matrix, which includes the camera intrinsic parameter matrix H1 and the extrinsic parameter matrix H2.

4. The camera calibration method under non-uniform illumination conditions according to claim 3, characterized in that: In step S4, the pixel coordinates (u,v) of each corner point on the checkerboard calibration board in the image are first obtained using a corner point extraction algorithm. The world coordinate system is fixed on the checkerboard, so the Z-axis coordinate of any point on the checkerboard is 0. Then, the coordinates of each corner point in the world coordinate system are calculated. The camera is calibrated using the pixel coordinates of each corner point and the coordinates of each corner point in the world coordinate system to obtain the camera's intrinsic and extrinsic parameter matrices and distortion parameters.

5. A camera calibration system under non-uniform illumination conditions, comprising calibrating the camera using the method described in any one of claims 1 to 4, including: The first processing module is used to process the input image using the multi-scale Retinex algorithm; The second processing module is used to perform Gaussian homomorphic filtering on the image obtained in step S1. The third processing module is used to process the image obtained in step S2 using the Otsu algorithm; The calibration module is used to calibrate the camera using the Zhang Zhengyou planar calibration method on the images processed by steps S1 to S3. The first processing module, the second processing module, and the third processing module are all connected to the calibration module.

Images