Glass orientation calibration method for refractive imaging systems

Abstract

The application discloses a glass orientation calibration system and method for a refractive imaging system, and the calibration system comprises a chessboard calibration plate, three plane glasses, and an imaging module. The chessboard calibration plate is an asymmetric pattern calibration plate, which is used for determining the direction during calibration, so that the feature points in the pictures of the calibration plate shot each time can be extracted in the same order. The plane glasses are used for being superposed in front of a camera to shoot the calibration plate pictures and extract different pixel coordinates of the calibration points in the same world coordinate. The imaging module is composed of an industrial camera and a camera support, and the support is used for adjusting the position and angle of the camera. When there is a refractive object in front of the camera, the device can quickly determine the orientation of the glass, i.e. the direction vector of the glass, and has the advantages of simple device, high measurement precision, fast detection rate and the like.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of refractive imaging technology, and particularly relates to a glass orientation calibration method for a refractive imaging system. BACKGROUND

[0002] Non-contact measurement using vision systems has been applied in various fields. Vision measurement has the advantages of automation, flexibility and high precision, and is therefore attractive to engineers. The main component of a vision system is a three-dimensional reconstruction model based on the path of light rays. The single view point (SVP) model is the most commonly used reconstruction model for light rays in a homogeneous medium. However, the path of a ray can contain different media; for example, various outdoor autonomous positioning based on machine vision, vision imaging of underwater equipment, measurement research in various extreme environments such as wind tunnels, etc. The existing model based on pinhole imaging in computer vision does not consider the problem of medium refraction. When the camera optical axis is perpendicular to the medium refraction surface, and the medium is relatively thin, appropriate correction can still be equivalent to the pinhole imaging model. However, when the camera pose is arbitrary relative to the medium, or the medium is relatively thick, the original pinhole imaging model is no longer applicable, that is, the influence of refraction needs to be corrected by re-modeling.

[0003] The method of refraction correction can be divided into two categories: image correction method and geometric correction method. The image correction method first removes the image distortion caused by refraction, and then uses the SVP model to perform three-dimensional reconstruction on the non-distorted image. The image correction method does not consider the refractive index, the position and direction of the refractive index interface, and even the shape of the interface. Their requirements mainly include the unrefracted image and the refracted image of the same scene, which are used to establish a transformation function. The geometric correction method tracks the path of refracted light rays, and the geometric correction method requires a large number of parameters, including the position and direction of the refractive index interface, the refractive index, the interface shape, etc. SUMMARY

[0004] The purpose of the present application is to provide a glass orientation calibration method for a refractive imaging system, which avoids the need for a large number of parameters in refraction correction compared with geometric correction, only needs the superposition of several glass ports of a single lens, and can determine the normal of the refractive index interface by straight line fitting of the photographed calibration points, and has the advantages of simple device, high measurement precision, fast detection rate, etc.

[0005] The technical scheme for achieving the purpose of the present application is as follows: a glass orientation calibration system for a refractive imaging system, mainly composed of a checkerboard calibration plate, an imaging module and a computer.

[0006] The checkerboard calibration plate is provided with asymmetric circular ring feature points at the corners.

[0007] The imaging module is used for collecting the checkerboard image of the calibration board, wherein the imaging module and the checkerboard calibration board are divided into four cases according to whether the glass is placed or not: no glass, one piece of glass, two pieces of glass, and three pieces of glass.

[0008] The computer is used for corner point extraction on the checkerboard images collected by the imaging module in the four cases, fitting of the calibration points, and calculation of the directional vector of the glass in the camera coordinate system.

[0009] A glass orientation calibration method for a refractive imaging system, comprising the following steps:

[0010] (1) Calibration process: corner point extraction on the collected calibration board image; taking the edge corner with two ring feature points as the third edge corner in the clockwise direction, extracting 8x8 feature points in a certain order; calculating the internal parameters and distortion coefficients of the camera model by Zhang Zhengyou calibration method, thereby obtaining the conversion relationship between the pixel coordinate system and the camera coordinate system;

[0011] (2) corner point extraction on the calibration board image collected when there is no glass in front of the camera lens, then corner point extraction on the calibration board image collected when there is one piece of glass in front of the camera lens, then corner point extraction on the calibration board image collected when there are two pieces of glass in front of the camera lens, and then corner point extraction on the calibration board image collected when there are three pieces of glass in front of the camera lens in order;

[0012] (3) When the captured image passes through different thickness glass windows, a combination is made of the images of the same calibration point captured in the four cases, and least square method or opencv fitline function is used for fitting, there are 64 calibration points on the calibration board, 64 sets of calibration points are fitted, and 64 straight lines intersect at a point, which is the coordinate of the normal vector of the glass projected into the pixel coordinate system, and the normal vector of the glass can be determined from the point; the four cases are no glass, one piece of glass, two pieces of glass, and three pieces of glass;

[0013] (4) For each pair of corresponding image points, let the unrefracted point and the refracted point coordinates be and , and the equation of the connecting line is:

[0014]

[0015] Assuming that there are K connecting lines in total, and the secondary principal point is the intersection point of all the connecting lines; the equation of the connecting line is obtained by extracting the calibration points from the captured calibration board image, thereby obtaining the equations of 64 sets of connecting lines, i.e. 64 sets of (k, b) values, each set of (k, b) values can be regarded as a point, and then linear fitting by least square method can obtain the coordinates of the secondary principal point at this time the normal of the glass is expressed as:

[0016]

[0017] wherein, are the internal parameters of the camera is the focal length of the camera, is the pixel size, is the image center.

[0018] The application provides a glass orientation calibration system and method for a refractive imaging system, which, in comparison with a geometric refractive correction method, does not require a large number of parameters including the position and direction of a refractive index interface, and only needs a calibration plate and three pieces of glass to determine the position and direction of the refractive index interface; because the image after refraction is expanded larger than the image before refraction; at the same time, since the light ray parallel to the normal vector of the glass and passing through the optical center of the camera does not refract, the image point of the light ray is the secondary principal point of the refracted image; the coordinates of the secondary principal point can be obtained to know the position and direction of the glass. The application has the characteristics of simplicity, rapidness, easy operation, good precision, easy understanding and the like. BRIEF DESCRIPTION OF DRAWINGS

[0019] Figure 1 It is a measurement experimental device diagram of the glass orientation calibration system and method for a refractive imaging system.

[0020] Figure 2 It is a picture of extracting calibration points in a fixed order according to a special calibration plate.

[0021] Figure 3 It is an imaging schematic diagram of the calibration points containing glass refraction.

[0022] Figure 4 It is a schematic diagram of the intersection point of the straight line fitting of the calibration points, that is, the secondary principal point of the glass direction vector. DETAILED DESCRIPTION

[0023] The application relates to a simple and rapid glass orientation calibration method for a refractive imaging system, and relates to fitting a group of calibration points by using a least square method, and the core is that a group of pictures with one-time superposition of glass and without glass are shot, then the calibration points in the pictures are fitted, and a direction vector of the glass in a camera coordinate system is obtained.

[0024] As shown in Figure 1 the application provides a glass orientation calibration system for a refractive imaging system, which comprises a checkerboard calibration plate 1, an imaging module 3, glass 2 and a computer 4.

[0025] The chessboard calibration board 1 is provided with asymmetric ring feature points at the corners, which are used to extract pixel coordinates of the calibration points and calibrate internal and external parameters of the camera.

[0026] The imaging module 3 is used to collect the chessboard image of the calibration board.

[0027] The glass 2 is used to collect the chessboard image taken when the glass has different thicknesses.

[0028] The computer 4 is used to extract the corner points of the chessboard images collected by the imaging module under four conditions, fit the calibration points, and calculate the direction vector of the glass in the camera coordinate system.

[0029] As shown in Figure 2 The chessboard calibration board is provided with five asymmetric ring feature points at the four corners, which are used to determine the direction during calibration and ensure that the feature points extracted under different angles can be arranged in the same order; the four feature points are distributed at the four corners, which can reduce the detection range to a certain range and prevent the complex environment from affecting the subsequent chessboard corner point extraction, thereby improving the detection accuracy. The fifth feature point must ensure that the symmetry of the calibration board can be broken, so that the feature points extracted in the four cameras can be arranged in the same order.

[0030] The chessboard calibration board has a thickness of 0.3 mm, and each grid in the chessboard is a black and white square grid, each small grid has a side length of 10 mm, and the number of grids is 8x8, which is used to calibrate the camera parameters and ensure that each calibration board picture is extracted in the same order.

[0031] Further, the glass is a K9 glass with a thickness of 8 mm, which is not coated and has a refractive effect on incident light, and the refractive index is 1.5.

[0032] Further, the imaging module is composed of a black and white industrial camera, a glass support and a camera support; the relative position between the camera and the glass support does not change. The pictures of the calibration board without the glass, the calibration board in front of the camera lens with a glass, the pictures of two glasses in front of the camera lens, and the pictures of three calibration boards in front of the camera lens are taken.

[0033] The imaging module is a black and white industrial camera, which uses a Daheng Image Mercury black and white camera, can collect 964x1292 gray scale images at high speed, selects a lens with a focal length of 8 mm, and is equipped with an adjustable camera support, which is provided with a groove for sliding the camera. The camera position and angle can be freely adjusted according to the actual situation to ensure complete imaging.

[0034] The present application firstly needs to shoot the pictures of the calibration board with and without glass, stack three 8mm thick glasses in front of the camera lens to shoot the pictures of the calibration board, and perform corner point extraction, take the edge corner with two circular ring feature points as the third edge corner in clockwise direction, and extract 8x8 feature points according to fixed order for each shot calibration board picture. The internal parameters and distortion coefficients of the camera model are calculated by Zhang Zhengyou calibration method, and the conversion relationship between the pixel coordinate system and the camera coordinate system is obtained.

[0035] The corner points of the calibration board pictures collected without glass in front of the camera lens are extracted and arranged in a certain order, then a glass is stacked in front of the camera lens to collect the calibration board pictures and extract the corner points, then two glasses are stacked in front of the camera lens to collect the calibration board pictures and extract the corner points, and then three glasses are stacked in front of the camera to shoot the calibration board pictures and extract the corner points in order.

[0036] When shooting images through glass windows of different thicknesses, different degrees of refraction are generated, while the pictures without glass are not affected by refraction. A combination of pictures shot in four cases is performed, and least square method or fitline function in opencv is used for fitting. Since there are 64 calibration points on the calibration board, 64 groups of calibration points are fitted, and the 64 straight lines intersect at a point, which is the coordinate of the normal vector of the glass projected into the pixel coordinate system.

[0037] The present application also provides a calibration method based on the above-mentioned glass orientation calibration system, comprising the following steps:

[0038] (1) Calibration process: the corner points of the calibration board images collected by the industrial camera are extracted; the edge corner with two circular ring feature points is taken as the third edge corner in clockwise direction, and 8x8 feature points are extracted in a certain order; the internal parameters and distortion coefficients of the camera model are calculated by Zhang Zhengyou calibration method, and the conversion relationship between the pixel coordinate system and the camera coordinate system is obtained.

[0039] (2) The corner points of the calibration board pictures collected without glass in front of the camera lens are extracted and arranged in a certain order, then a glass is stacked in front of the camera lens to collect the calibration board pictures and extract the corner points, then two glasses are stacked in front of the camera lens to collect the calibration board pictures and extract the corner points, and then three glasses are stacked in front of the camera to shoot the calibration board pictures and extract the corner points in order. Thus, the pixel coordinates of 64 groups of calibration points can be obtained, each group containing 4 calibration points, i.e. without glass, one glass, two glasses, and three glasses.

[0040] (3) As Figure 3As shown, when the image is taken through the glass window of different thickness, different degrees of refraction are generated during the period, and the picture without glass is not affected by refraction. A combination of pictures taken in four cases with the same calibration point is made, and a least square method or a fitline function in opencv is used for fitting. Since there are 64 calibration points on the calibration board, 64 sets of calibration points are fitted, and the 64 straight lines intersect at a point, which is the coordinate of the normal vector of the glass projected into the pixel coordinate system. The normal vector of the glass can be determined from the point.

[0041] (4) For each pair of corresponding image points, let the unrefracted point and the refracted point coordinates be and The equation of the connecting line is:

[0042]

[0043] Then the k, b values fitted from 64 sets of calibration points are obtained, where the k, b values can be represented as:

[0044]

[0045] Suppose there are K connecting lines, and the secondary principal point is the intersection of all connecting lines. The equation of the connecting line is obtained by extracting the calibration points from the photographed calibration board picture, so that the equation of 64 connecting lines, i.e. 64(k, b) values, can be obtained. Each(k, b) value can be regarded as a point, and then a least square linear fitting is performed, so that a(k, b) value is obtained. At this time, (-k, b) represents the coordinates of the secondary principal point. The coordinates of the secondary principal point obtained by the least square method are the coordinates of the secondary principal point, and the normal vector of the glass can be represented as:

[0046]

[0047] where represents the coordinates of the secondary principal point, are the internal parameters of the camera is the focal length of the camera, is the pixel size, is the image center.

[0048] (5) Repeat the above operation, change the attitude of the calibration board, and take multiple pictures of the calibration board in different attitudes when there is glass and no glass in front of the camera lens in the process of the last step. The coordinates of different calibration points under the same glass attitude can be obtained, and the coordinates of the calibration points are obtained by using the same steps as above. The average value of the coordinates of the secondary principal point of the glass attitude under different calibration board states is obtained.

[0049] Table 1 is the position of the sub-principal point in the image coordinate system obtained by different methods:

[0050] Table 1

[0051]

[0052] The above embodiments only express several embodiments of the present application, and the description is more specific and detailed, but it cannot be understood as a limitation on the scope of the patent of the present application. It should be noted that for ordinary skilled persons in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are within the scope of protection of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A method for calibrating the orientation of a glass in a refractive imaging system, characterized in that, The method is realized based on a glass orientation calibration system, which mainly comprises a checkerboard calibration board, an imaging module and a computer. The four corners of the checkerboard calibration board are provided with asymmetric ring feature points. The imaging module is used for collecting checkerboard images of the calibration board, wherein the imaging module and the checkerboard calibration board are divided into four cases according to whether the glass is placed or not: no glass, one piece of glass, two pieces of glass and three pieces of glass. The computer is used for extracting corner points of the checkerboard images collected by the imaging module under the four cases, fitting the calibration points and calculating the directional vector of the glass in the camera coordinate system. The glass orientation calibration method comprises the following steps: (1) Calibration process: corner points of the collected calibration board images are extracted; the corner with two ring feature points is taken as the third corner in the clockwise direction, and 8x8 feature points are extracted in sequence; the internal parameters and distortion coefficients of the camera model are calculated by Zhang Zhengyou calibration method, so as to obtain the conversion relationship between the pixel coordinate system and the camera coordinate system; (2) Corner points of the calibration board images are extracted in sequence when there is no glass in front of the camera lens, and then one piece of glass is added in front of the camera lens to collect the calibration board images and extract the corner points, then two pieces of glass are added in front of the camera lens to collect the calibration board images and extract the corner points, and then three pieces of glass are added in front of the camera to collect the calibration board images and extract the corner points in sequence; (3) When the images are taken through the glass window with different thicknesses, the same calibration point is used to take the images in four cases, and the least square method or the fitline function in opencv is used to fit them, there are 64 calibration points on the calibration board, and 64 straight lines intersect at a point, which is the coordinate perpendicular to the glass projected into the pixel coordinate system, and the normal vector of the glass can be determined from the point; the four cases are no glass, one piece of glass, two pieces of glass and three pieces of glass; (4) For each pair of corresponding image points, let the unrefracted point and the refracted point coordinates be and The equation of the connecting line is then ； Suppose there are K connecting lines in total, and the secondary principal point is the intersection of all the connecting lines; the equation of the connecting line is obtained by extracting the calibration points from the picture of the calibration board, and thus 64 sets of equations of the connecting lines, i.e. 64 sets of (k, b) values, are obtained, each set of (k, b) values is regarded as a point, and then linear fitting of least squares is performed on the points to obtain the coordinates of the secondary principal point At this time, the normal line of the glass is represented as: ； wherein, are internal parameters of the camera is the focal length of the camera, is the pixel size, is the image center.

2. The method of claim 1, wherein, The checkerboard calibration board is provided with five asymmetric ring feature points at the four corners, which are used to determine the direction during calibration and ensure that the feature points extracted at different angles are arranged in the same order; each grid in the checkerboard is a black and white square grid, and the length of each small grid is 10 mm, and the number of grids is 8x8.

3. The method of claim 2, wherein, The imaging module is composed of a black and white industrial camera, a glass support and a camera support; the relative position between the camera and the glass support does not change; the images of the calibration board without glass, the calibration board with one piece of glass in front of the camera lens, the calibration board with two pieces of glass in front of the camera lens and the calibration board with three pieces of glass in front of the camera lens are taken.

4. The method according to claim 1 or 3, characterized in that, The glass is K9 8mm thick glass with a refractive index of 1.

5.

5. The method according to claim 1 or 3, characterized in that, The black and white industrial camera uses Daheng Image Mercury black and white camera.

6. The method of claim 5, wherein, The black and white industrial camera selects a lens with a focal length of 8mm.

7. The method of claim 1, wherein, Repeat steps (1)-(4), change the posture of the calibration board, take several pictures of the calibration board in different postures with and without the glass in front of the camera lens, obtain the coordinates of different calibration points under the same glass posture, and average the coordinates of the secondary principal point of the glass posture under different calibration board states.

Images