Calibration method for combined imaging of visible light and infrared camera

Abstract

The invention discloses a visible light and infrared camera combined imaging calibration method which comprises the following steps: selecting an aluminum plate as a substrate, and drawing a black and white chessboard on the substrate to form a calibration object; the substrate is heated to a preset temperature threshold value; acquiring a plurality of visible light images and infrared images of the calibration object in different orientations by using a visible light camera and an infrared camera respectively under the condition of adjusting the orientation of the calibration object; respectively processing each visible light image and each infrared image, so as to respectively obtain the infrared image angular point coordinates of the calibration object and the visible light image angular point coordinates of the calibration object; according to the method and the device, the problems that in infrared camera imaging in the prior art, the imaging effect of angular points, colors and texture features of a two-dimensional plane target is poor, and a calibration plate of visible light binocular vision cannot be directly used for calibration are solved.

Description

Technical Field

[0001] The present invention relates to the field of camera calibration, and in particular to a calibration method for combined imaging of visible light and infrared cameras. Background Art

[0002] Among the calibration methods for visible light binocular vision, Zhang's calibration method is the most commonly used. It calibrates the camera by making a simple black and white plane chessboard as a calibration object.

[0003] However, in infrared camera imaging, the imaging effect of the corner points, color and texture features of traditional two-dimensional plane targets is poor, so it is impossible to directly use the visible light binocular vision calibration plate for calibration.

[0004] Therefore, a method for calibrating the combined imaging of visible light and infrared cameras is provided, which uses aluminum as a substrate during use and draws a black and white chessboard on the substrate as a calibration object. By heating, the camera can be assisted to obtain clearer and more stable images, thereby improving the accuracy of camera shooting and having good robustness. This is a problem that the present invention urgently needs to solve. Summary of the invention

[0005] In view of the above technical problems, the purpose of the present invention is to overcome the problem that in the prior art, the imaging effect of the corner points, color and texture features of the traditional two-dimensional plane target in infrared camera imaging is poor, and the calibration plate of the visible light binocular vision cannot be directly used for calibration. Thus, a calibration method for joint imaging of visible light and infrared cameras is provided, which uses aluminum as a substrate during use and draws a black and white chessboard on the substrate as a calibration object. By heating, the camera can be assisted to obtain clearer and more stable images, thereby improving the accuracy of camera shooting and having good robustness.

[0006] In order to achieve the above object, the present invention provides a calibration method for joint imaging of visible light and infrared cameras, the calibration method comprising:

[0007] An aluminum plate is selected as a substrate, and a black and white chessboard is drawn on the substrate to form a calibration object;

[0008] The substrate is heated to a preset temperature threshold by a heating component so that the boundary and corner points of the calibration object are clearly visible;

[0009] Using a visible light camera and an infrared camera respectively, while adjusting the orientation of the calibration object, to obtain a plurality of visible light images and infrared images of the calibration object at different orientations;

[0010] Processing each of the acquired visible light images and infrared images respectively, so as to obtain the coordinates of the corner points of the infrared image of the calibration object and the coordinates of the corner points of the visible light image of the calibration object respectively;

[0011] The infrared image corner point coordinates and the visible light image corner point coordinates corresponding to each image are transformed to obtain discrete image points of each image in the pixel coordinate system;

[0012] According to the discrete image points of each image in the pixel coordinate system, the internal and external parameters of the visible light camera and the infrared camera are calculated respectively to obtain the calibration results.

[0013] Preferably, the surface of the substrate is covered with an aluminum oxide layer, and the black and white chessboard is drawn on the aluminum oxide layer.

[0014] Preferably, the transforming of the infrared image corner point coordinates and the visible light image corner point coordinates corresponding to each image to obtain discrete image points of each image in a pixel coordinate system comprises the following steps:

[0015] The coordinates of the corner points of the infrared image of the calibration object and the coordinates of the corner points of the visible light image of the calibration object are respectively subjected to affine transformation and projection transmission to obtain discrete image points of each in a pixel coordinate system.

[0016] Preferably, the step of calculating the internal and external parameters of the visible light camera and the infrared camera respectively according to the discrete image points of each image in the pixel coordinate system to obtain the calibration result comprises the following steps:

[0017] The corner point coordinates of the infrared image of the calibration object and the corner point coordinates of the visible light image of the calibration object are respectively transformed through the transformation matrices between the world coordinate system, the camera coordinate system and the image coordinate system to obtain the pixel coordinate matrix of the infrared image of the calibration object and the pixel coordinate system matrix of the visible light image of the calibration object; wherein,

[0018] The pixel coordinate matrix of the infrared image of the calibration object and the pixel coordinate matrix of the visible light image of the calibration object are both: Among them, u and v represent the coordinates in the pixel coordinate system, s represents the scale factor, and f x 、f y , u0, v0, γ represent the camera internal parameters, R, t represent the camera external parameters, τ is the deviation of the pixel scale in the x, y direction, X w , Y w , Z w Represents the coordinates in the world coordinate system. According to Zhang's calibration method, the intrinsic parameter matrix of the camera is:

[0019] Preferably, the step of calculating the internal and external parameters of the visible light camera and the infrared camera respectively according to the discrete image points of each image in the pixel coordinate system to obtain the calibration result further comprises the following steps:

[0020] According to Zhang's calibration method, the intrinsic parameter matrix of the camera is: The external parameter matrix is:

[0021] Combining the internal parameter matrix and the external parameter matrix yields:

[0022] Homography matrix And the camera internal parameters are calculated through the homography matrix: f x 、f y , u0, v0, γ and external parameters: R, t.

[0023] Preferably, the transformation matrix from the world coordinate system to the camera coordinate system is:

[0024] The transformation matrix from the camera coordinate system to the image coordinate system is: The transformation matrix from the image coordinate system to the pixel coordinate system is: According to the above technical solution, the calibration method for joint imaging of visible light and infrared cameras provided by the present invention has the following beneficial effects when used:

[0025] (1) The advantages of the aluminum plate substrate provided by the present invention as the calibration object with the black and white chessboard drawn on it are as follows: on the one hand, since the aluminum plate has good thermal conductivity, the heating speed of the substrate can be slowly and controllably controlled in combination with the heating component, so that the heated substrate can be well controlled within the set temperature threshold, thereby making the image clearer and more stable to display, and the black and white chessboard drawn on the heated calibration object can be more clearly displayed, so as to obtain a good calibration effect for the visible light camera and the infrared camera;

[0026] On the other hand, the calibration object made of aluminum plate is easy to make, with little difficulty in making, and aluminum material is also easy to obtain. Moreover, since the calibration object has an aluminum plate as a substrate, it has good wear resistance, thereby extending the service life of the calibration object.

[0027] Therefore, an aluminum plate is used as a substrate, and a black and white chessboard is drawn on the substrate as a calibration object. The calibration object is applied to the calibration of an infrared camera after being heated, which can achieve a good imaging effect.

[0028] (2) The calibration method for joint imaging of visible light and infrared cameras provided by the present invention only requires the use of one calibration object and the taking of several groups of pictures from different directions. It is not only practical, flexible and convenient, but also has high accuracy and good robustness.

[0029] Therefore, the present invention provides a calibration method for joint imaging of visible light and infrared cameras. Aluminum is used as a substrate, and a black and white chessboard is drawn on the substrate as a calibration object. By heating, the camera can be assisted to obtain clearer and more stable images, thereby improving the accuracy of camera shooting and having good robustness.

[0030] Other features and advantages of the present invention will be described in detail in the subsequent specific implementation mode; and the parts not involved in the present invention are the same as the prior art or can be implemented by using the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the following specific embodiments, they are used to explain the present invention but do not constitute a limitation of the present invention. In the accompanying drawings:

[0032] Figure 1 It is a structural schematic diagram of a calibration method for joint imaging of visible light and infrared cameras provided in a preferred embodiment of the present invention;

[0033] Figure 2 It is a structural schematic diagram of a calibration object for drawing a black and white chessboard in a calibration method for joint imaging of visible light and infrared cameras provided in a preferred embodiment of the present invention;

[0034] Figure 3 It is a discrete image point of an image in a pixel coordinate system in a calibration method for joint imaging of visible light and infrared cameras provided in a preferred embodiment of the present invention.

[0035] Description of Reference Numerals

[0036] 1. Substrate; 2. Aluminum oxide layer. DETAILED DESCRIPTION

[0037] The specific implementation of the present invention is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described here is only used to illustrate and explain the present invention, and is not used to limit the present invention.

[0038] like Figure 1-3 As shown, the present invention provides a calibration method for joint imaging of visible light and infrared cameras, the calibration method comprising:

[0039] Step S1: Select an aluminum plate as a substrate 1, and draw a black and white chessboard on the substrate 1 to form a calibration object;

[0040] Step S2: heating the substrate 1 to a preset temperature threshold by a heating component so that the boundary and corner points of the calibration object are clearly visible;

[0041] Step S3: using a visible light camera and an infrared camera respectively to obtain a plurality of visible light images and infrared images of the calibration object at different orientations while adjusting the orientation of the calibration object;

[0042] Step S4: processing each of the obtained visible light images and infrared images respectively, so as to obtain the coordinates of the corner points of the infrared image of the calibration object and the coordinates of the corner points of the visible light image of the calibration object respectively;

[0043] Step S5: transforming the infrared image corner point coordinates and the visible light image corner point coordinates corresponding to each image to obtain discrete image points of each image in a pixel coordinate system;

[0044] Step S6: Calculate the internal and external parameters of the visible light camera and the infrared camera respectively according to the discrete image points of each image in the pixel coordinate system to obtain a calibration result.

[0045] In the above scheme, at least three visible light images and infrared images of the calibration object at different orientations are acquired by using an infrared camera and a visible light camera while adjusting the orientation of the calibration object, so as to obtain rich coordinate information.

[0046] In the above scheme, the heating component can be a conventional heating plate in the prior art to heat the substrate 1 to a preset temperature. The heating plate can be fixed on the substrate 1 facing away from the black and white chessboard. The heating component can also be a heating resistor wire, which can be fixed inside the substrate 1 with a hollow structure inside to heat the substrate 1. In the above scheme, the advantages of using the aluminum plate substrate 1 as the calibration object with a black and white chessboard drawn on it are as follows: on the one hand, since the aluminum plate has good thermal conductivity, the heating speed of the substrate 1 can be slowly and controllably controlled in conjunction with the heating component, so that the heated substrate 1 can be better controlled within the set temperature threshold, so that the image can be displayed more clearly and stably, and the black and white chessboard drawn on the heated calibration object can be more clearly displayed, so as to obtain a good calibration effect for the visible light camera and the infrared camera;

[0047] On the other hand, the calibration object made of aluminum plate is easy to make, with low difficulty in making, and aluminum material is also easy to obtain. Moreover, since the calibration object has an aluminum plate as a substrate 1, it has good wear resistance, thereby extending the service life of the calibration object.

[0048] Therefore, an aluminum plate is used as the substrate 1, and a black and white chessboard is drawn on the substrate 1 as a calibration object. When the calibration object is heated and applied to the calibration of the infrared camera, a good imaging effect can be achieved.

[0049] The calibration method for combined imaging of visible light and infrared cameras provided by the present invention only needs to use one calibration object and take several groups of pictures from different directions. It is not only practical, flexible and convenient, but also has high accuracy and good robustness.

[0050] In a preferred embodiment of the present invention, the surface of the substrate 1 is covered with an aluminum oxide layer 2 , and the black and white chessboard is drawn on the aluminum oxide layer 2 .

[0051] In the above solution, since the aluminum material has a high reflectivity to the infrared rays of the infrared camera, by covering the surface of the substrate 1 with a layer of aluminum oxide layer 2, the infrared camera can be assisted to achieve a better imaging effect.

[0052] In a preferred embodiment of the present invention, the step of transforming the infrared image corner point coordinates and the visible light image corner point coordinates corresponding to each image to obtain discrete image points of each image in a pixel coordinate system comprises the following steps:

[0053] The coordinates of the corner points of the infrared image of the calibration object and the coordinates of the corner points of the visible light image of the calibration object are respectively subjected to affine transformation and projection transmission to obtain discrete image points of each in a pixel coordinate system.

[0054] In the above scheme, the checkerboard on the calibration object is regarded as a rigid body, and the coordinates of the vertex points of each square in the world coordinate system are known. By performing a series of transformations on the rigid body in sequence, affine transformation and projection transmission are performed on it from the world coordinate system, such as Figure 3 As shown, the discrete image points in the pixel coordinate system are finally obtained, where the world coordinate system: X w , Y w , Z w . Camera coordinate system: X c , Y c , Z c . Image coordinate system: x, y. Pixel coordinate system: u, v.

[0055] In a preferred embodiment of the present invention, the step of calculating the internal and external parameters of the visible light camera and the infrared camera respectively according to the discrete image points of each image in the pixel coordinate system to obtain the calibration result comprises the following steps:

[0056] The corner point coordinates of the infrared image of the calibration object and the corner point coordinates of the visible light image of the calibration object are respectively transformed through the transformation matrices between the world coordinate system, the camera coordinate system and the image coordinate system to obtain the pixel coordinate matrix of the infrared image of the calibration object and the pixel coordinate system matrix of the visible light image of the calibration object; wherein,

[0057] The pixel coordinate matrix of the infrared image of the calibration object and the pixel coordinate matrix of the visible light image of the calibration object are both: Among them, u and v represent the coordinates in the pixel coordinate system, s represents the scale factor, and f x 、f y , u0, v0, γ represent the camera internal parameters, R, t represent the camera external parameters, τ is the deviation of the pixel scale in the x, y direction, X w , Y w , Z w Indicates that the coordinate calibration object in the world coordinate system is located in the world coordinate system Z w = 0, according to Zhang's calibration method, the intrinsic parameter matrix of the camera is:

[0058] In a preferred embodiment of the present invention, the step of calculating the internal and external parameters of the visible light camera and the infrared camera respectively according to the discrete image points of each image in the pixel coordinate system to obtain the calibration result further includes the following steps:

[0059] According to Zhang's calibration method, the intrinsic parameter matrix of the camera is: The external parameter matrix is: Combining the internal parameter matrix and the external parameter matrix yields:

[0060] Homography matrix And the camera internal parameters are calculated through the homography matrix: f x 、f y , u0, v0, γ and external parameters: R, t.

[0061] In a preferred embodiment of the present invention, the transformation matrix from the world coordinate system to the camera coordinate system is:

[0062]

[0063] The transformation matrix from the camera coordinate system to the image coordinate system is: The transformation matrix from the image coordinate system to the pixel coordinate system is: In summary, the calibration method for joint imaging of visible light and infrared cameras provided by the present invention overcomes the problem that in the prior art, the imaging effect of the corner points, color and texture features of the traditional two-dimensional plane target in infrared camera imaging is poor, and it is impossible to directly use the calibration plate of visible light binocular vision for calibration.

[0064] The preferred embodiments of the present invention are described in detail above in conjunction with the accompanying drawings. However, the present invention is not limited to the specific details in the above embodiments. Within the technical concept of the present invention, a variety of simple modifications can be made to the technical solution of the present invention, and these simple modifications all belong to the protection scope of the present invention.

[0065] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. In order to avoid unnecessary repetition, the present invention will not further describe various possible combinations.

[0066] In addition, various embodiments of the present invention may be arbitrarily combined, and as long as they do not violate the concept of the present invention, they should also be regarded as the contents disclosed by the present invention.

Claims

1. A calibration method for joint imaging of visible light and infrared cameras, characterized in that: The calibration method comprises: An aluminum plate is selected as a substrate (1), and a black and white chessboard is drawn on the substrate (1) to form a calibration object; The substrate (1) is heated to a preset temperature threshold by a heating component so that the boundaries and corners of the calibration object are clearly visible; Using a visible light camera and an infrared camera respectively, while adjusting the orientation of the calibration object, to obtain a plurality of visible light images and infrared images of the calibration object at different orientations; Processing each of the acquired visible light images and infrared images respectively, so as to obtain the coordinates of the corner points of the infrared image of the calibration object and the coordinates of the corner points of the visible light image of the calibration object respectively; The infrared image corner point coordinates and the visible light image corner point coordinates corresponding to each image are transformed to obtain discrete image points of each image in the pixel coordinate system; According to the discrete image points of each image in the pixel coordinate system, the internal and external parameters of the visible light camera and the infrared camera are calculated respectively to obtain the calibration results.

2. The calibration method for combined imaging of visible light and infrared cameras according to claim 1, characterized in that: The surface of the substrate (1) is covered with an aluminum oxide layer (2), and the black and white chessboard is drawn on the aluminum oxide layer (2).

3. The calibration method for combined imaging of visible light and infrared cameras according to claim 1, characterized in that: The step of transforming the infrared image corner point coordinates and the visible light image corner point coordinates corresponding to each image to obtain discrete image points of each image in a pixel coordinate system comprises the following steps: The coordinates of the corner points of the infrared image of the calibration object and the coordinates of the corner points of the visible light image of the calibration object are respectively subjected to affine transformation and projection transmission to obtain discrete image points of each in a pixel coordinate system.

4. The calibration method for combined imaging of visible light and infrared cameras according to claim 1, characterized in that: The step of calculating the internal and external parameters of the visible light camera and the infrared camera respectively according to the discrete image points of each image in the pixel coordinate system to obtain the calibration result includes the following steps: The corner point coordinates of the infrared image of the calibration object and the corner point coordinates of the visible light image of the calibration object are respectively transformed through the transformation matrices between the world coordinate system, the camera coordinate system and the image coordinate system to obtain the pixel coordinate matrix of the infrared image of the calibration object and the pixel coordinate system matrix of the visible light image of the calibration object; wherein, The pixel coordinate matrix of the infrared image of the calibration object and the pixel coordinate matrix of the visible light image of the calibration object are both: Among them, u and v represent the coordinates in the pixel coordinate system, s represents the scale factor, and f x 、f y , u0, v0, γ represent the camera internal parameters, R, t represent the camera external parameters, τ is the deviation of the pixel scale in the x, y direction, X w , Y w , Z w Represents the coordinates in the world coordinate system (the calibration object is located in the world coordinate system Z w = 0), according to Zhang’s calibration method, the intrinsic parameter matrix of the camera is:

5. The calibration method for combined imaging of visible light and infrared cameras according to claim 4, characterized in that: The step of calculating the internal and external parameters of the visible light camera and the infrared camera respectively according to the discrete image points of each image in the pixel coordinate system to obtain the calibration result also includes the following steps: According to Zhang's calibration method, the intrinsic parameter matrix of the camera is: The external parameter matrix is: Combining the internal parameter matrix and the external parameter matrix yields: Homography matrix And the camera internal parameters are calculated through the homography matrix: f x 、f y , u0, v0, γ and external parameters: R, t.

6. The method for calibrating the combined imaging of visible light and infrared cameras according to claim 4, characterized in that: The transformation matrix from the world coordinate system to the camera coordinate system is: The transformation matrix from the camera coordinate system to the image coordinate system is: The transformation matrix from the image coordinate system to the pixel coordinate system is:

Images