Infrared camera and visible light camera integrated calibration system

Abstract

The application discloses an infrared camera and visible light camera integrated calibration system, which is simple and reasonable in structure, convenient to install and use, adopts an infrared fluorescent ball and a checkered calibration plate, combines a three-dimensional coordinate instrument and an NDI instrument, can realize one-time data collection, and can simultaneously calibrate an infrared binocular camera and a monocular visible light camera. The calibration system has high calibration precision and good consistency, improves the calibration efficiency of camera parameter calibration, and is worth popularizing and using.

Description

Technical Field

[0001] This invention relates to the field of infrared and visible light camera calibration technology in optical positioning, and in particular to an integrated calibration system for infrared and visible light cameras. Background Technology

[0002] The calibration of intrinsic and extrinsic parameters of a stereo camera is a core factor constraining the camera's image quality and determining the positional relationship between the imaged object and the real world. Camera calibration is the process of solving for the camera's intrinsic and extrinsic parameters and distortion coefficients based on the relationship between the pixel coordinate system and the world coordinate system, using certain constraints. Camera calibration methods can be divided into two types: the first is the traditional calibration method that requires a reference object; the second is the camera self-calibration method that does not require a reference object.

[0003] Traditional calibration methods typically use a checkerboard pattern as a reference, where the size, dimensions, and number of each checkerboard grid are known. The calibration process involves establishing a correspondence between the vertices of the checkerboard grid and their corresponding points on the image, using the known information from the checkerboard grid to determine the intrinsic and extrinsic parameters and distortion coefficients of the camera model.

[0004] Since traditional checkerboard calibration boards only contain black and white grids, the black and white checkerboard calibration method is not suitable for near-infrared binocular cameras because the ordinary black and white checkerboard cannot be clearly seen in the image. The black and white checkerboard calibration board is only suitable for the calibration of visible light cameras, while near-infrared binocular cameras require the use of components such as near-infrared reflectors for auxiliary calibration.

[0005] For optical infrared binocular positioning systems, adding a visible light camera can provide additional information, which helps improve the positioning capability of the system. However, adding a visible light camera requires separate calibration of both the infrared binocular camera and the visible light camera. Since separate calibration requires transferring the entire positioning system or rearranging the auxiliary calibration equipment for the other camera after the calibration of one camera is completed, the operation is complex and the calibration process is time-consuming and labor-intensive. Summary of the Invention

[0006] In view of the above problems, the present invention provides an integrated calibration system for infrared and visible light cameras to overcome or at least partially solve the above problems.

[0007] This invention provides the following solution:

[0008] An integrated calibration system for infrared and visible light cameras includes:

[0009] The system includes a processor unit, an optical positioning unit, a calibration plate, a 3D coordinate system, and an NDI instrument. The optical positioning unit includes an infrared binocular camera and a monocular visible light camera. The calibration plate includes a checkerboard visible to the monocular visible light camera and at least three infrared fluorescent spheres positioned at the non-intersecting points of the checkerboard lines. The infrared binocular camera, the monocular visible light camera, the 3D coordinate system, and the NDI instrument are all communicatively connected to the processor unit. The calibration plate is positioned on the Z-axis of the 3D coordinate system.

[0010] The processor is used to perform the following operations:

[0011] The Z-axis is controlled to carry the calibration plate to various target positions within the field of view of the infrared dual-mode camera, and the coordinates of the Z-axis, the image of the infrared fluorescent ball acquired by the infrared binocular camera, and the image of the checkerboard acquired by the monocular visible light camera are recorded at each target position.

[0012] The infrared binocular camera is calibrated using the Z-axis coordinates and the image of the infrared fluorescent ball to obtain the distortion coefficients, internal parameters, and external parameters of the infrared binocular camera.

[0013] Using the Z-axis coordinates and the image of the checkerboard pattern, the monocular visible light camera is calibrated using the checkerboard calibration method to obtain the distortion coefficients and internal parameters of the monocular visible light camera.

[0014] Control the Z-axis to move the calibration plate into the field of view of the calibrated infrared binocular camera;

[0015] Obtain the transformation from the infrared fluorescent sphere coordinate system to the NDI coordinate system acquired by the infrared binocular camera;

[0016] The coordinates of the four corner points in the NDI coordinate system are obtained after the infrared probe of the NDI instrument clicks on the four outermost corner points of the checkerboard; the checkerboard coordinates corresponding to the four corner points are obtained by the monocular visible light camera, so as to obtain the transformation from the checkerboard coordinate system to the NDI coordinate system.

[0017] The transformation from the infrared fluorescent sphere coordinate system to the NDI coordinate system is calculated using the transformation from the infrared fluorescent sphere coordinate system to the NDI coordinate system and the transformation from the checkerboard coordinate system to the NDI coordinate system. Based on the transformation from the checkerboard coordinate system to the infrared fluorescent sphere coordinate system, the coordinate system transformation of the monocular visible light camera relative to one of the infrared binocular cameras is calculated to obtain the external parameters of the monocular visible light camera.

[0018] Preferably, the method for determining the target location includes:

[0019] The effective working range of the infrared binocular camera in the Z direction is divided into multiple planes. The coordinates of the four corners of each plane are determined using a three-dimensional coordinate instrument. M×M positions are taken from each plane as the target positions.

[0020] Preferably, M is 9.

[0021] Preferably, the method for acquiring the external parameters corresponding to the infrared binocular camera includes:

[0022] One of the infrared binocular cameras is designated as the reference camera, and the other camera is designated as the camera to be calibrated.

[0023] The coordinates of the center of the sphere in the image of the infrared fluorescent sphere at each target location acquired by the reference camera are extracted as the first sphere coordinates;

[0024] Using the Z-axis coordinates as a standard and combining them with the first small sphere coordinates, the transformation between the reference camera coordinate system and the infrared fluorescent sphere coordinate system is achieved to obtain the external parameters of the reference camera;

[0025] The coordinates of the center of the infrared fluorescent sphere in the image of each target location acquired by the camera to be calibrated are extracted as the coordinates of the second sphere.

[0026] Using the first ball coordinates as a standard and combining them with the second ball coordinates, the transformation between the coordinate system of the camera to be calibrated and the reference coordinate system is achieved to obtain the external parameters of the camera to be calibrated.

[0027] Preferably: the coordinate transformation of the checkerboard grid at each position relative to the visible light camera is obtained, and the monocular visible light camera is calibrated using the coordinate transformation to obtain the distortion coefficient and internal parameters corresponding to the monocular visible light camera.

[0028] Preferably: a ROM file suitable for the NDI instrument is obtained; the ROM file is created by measuring the positions of at least three infrared fluorescent spheres;

[0029] Based on the ROM file, the transformation from the infrared fluorescent sphere coordinate system to the NDI coordinate system is obtained.

[0030] Preferably, the transformation from the checkerboard coordinate system to the NDI coordinate system is equal to the transformation from the checkerboard coordinate system to the infrared fluorescent sphere coordinate system multiplied by the transformation from the infrared fluorescent sphere coordinate system to the NDI coordinate system.

[0031] Preferably: the transformation of the monocular visible light camera coordinate system relative to the coordinate system of one of the infrared binocular cameras includes the transformation of the monocular visible light camera coordinate system relative to the coordinate system of the left eye camera of the infrared binocular camera. The It can be expressed by the following formula:

[0032]

[0033] In the formula: N represents the number of measurements; This represents the transformation from the infrared fluorescence sphere coordinate system to the left eye camera coordinate system. This represents the transformation from the checkerboard coordinate system to the infrared fluorescent sphere coordinate system. This represents the transformation from a checkerboard coordinate system to a monocular visible light camera.

[0034] Preferably, the final coordinate transformation of the monocular visible light camera coordinate system relative to the coordinate system of the left eye camera of the infrared binocular camera is obtained by solving the least squares method.

[0035] Preferably: the Z-axis is controlled to move the calibration plate to the center of the calibrated infrared binocular camera's field of view.

[0036] According to specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0037] This application provides an integrated calibration system for infrared and visible light cameras. It features a simple and reasonable structure, is easy to install and use, and employs a calibration plate combining an infrared fluorescent ball and a checkerboard pattern. Combined with a 3D coordinate analyzer and an NDI instrument, it can simultaneously calibrate both an infrared binocular camera and a monocular visible light camera in a single data acquisition. Furthermore, it offers high calibration accuracy and good consistency, improving the efficiency of camera parameter calibration. It is worthy of widespread adoption.

[0038] Of course, any product implementing this invention does not necessarily need to achieve all of the advantages described above at the same time. Attached Figure Description

[0039] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly described below. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0040] Figure 1 This is a schematic diagram of the calibration plate provided in an embodiment of the present invention;

[0041] Figure 2 This is a schematic diagram of the connection between the calibration plate and the Z-axis provided in an embodiment of the present invention;

[0042] Figure 3 This is a schematic diagram of the coordinates of each infrared fluorescent sphere in the infrared fluorescent sphere coordinate system provided in the embodiments of the present invention. Detailed Implementation

[0043] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention are within the scope of protection of the present invention.

[0044] See Figure 1 , Figure 2 This invention provides an integrated calibration system for both infrared and visible light cameras, as shown in the embodiments of the present invention. Figure 1 , Figure 2 As shown, the system may include:

[0045] The system comprises a processor unit, an optical positioning unit, a calibration plate 1, a three-dimensional coordinate instrument, and an NDI instrument. The optical positioning unit includes an infrared binocular camera and a monocular visible light camera. The calibration plate 1 includes a checkerboard pattern 11 visible to the monocular visible light camera and at least three infrared fluorescent balls 12 positioned at non-intersecting points of the checkerboard lines. The infrared binocular camera, the monocular visible light camera, the three-dimensional coordinate instrument, and the NDI instrument are all communicatively connected to the processor unit. The calibration plate is positioned on the Z-axis 2 of the three-dimensional coordinate instrument.

[0046] The processor is used to perform the following operations:

[0047] The Z-axis is controlled to carry the calibration plate to various target positions within the field of view of the infrared dual-mode camera, and the coordinates of the Z-axis, the image of the infrared fluorescent ball acquired by the infrared binocular camera, and the image of the checkerboard acquired by the monocular visible light camera are recorded at each target position. In specific implementation, embodiments of this application may provide a method for determining the target position including:

[0048] The effective working range of the infrared binocular camera in the Z direction is equally divided into multiple planes. The coordinates of the four corners of each plane are determined using a three-dimensional coordinate instrument. M×M positions are equally selected from each plane as the target positions. Further, M is 9.

[0049] The infrared binocular camera is calibrated using the Z-axis coordinates and the image of the infrared fluorescent ball to obtain the distortion coefficients, internal parameters, and external parameters corresponding to the infrared binocular camera. It is understood that the internal parameters of the infrared binocular camera provided in this embodiment can be obtained through calculation using a mature method combining infrared fluorescent ball positioning and calibration. To ensure that the external parameters of the left and right eyes of the infrared binocular camera are unified, this embodiment provides a method for obtaining the external parameters of the infrared binocular camera, including:

[0050] One of the infrared binocular cameras is designated as the reference camera, and the other camera is designated as the camera to be calibrated.

[0051] The coordinates of the center of the sphere in the image of the infrared fluorescent sphere at each target location acquired by the reference camera are extracted as the first sphere coordinates;

[0052] Using the Z-axis coordinates as a standard and combining them with the first small sphere coordinates, the transformation between the reference camera coordinate system and the infrared fluorescent sphere coordinate system is achieved to obtain the external parameters of the reference camera;

[0053] The coordinates of the center of the infrared fluorescent sphere in the image of each target location acquired by the camera to be calibrated are extracted as the coordinates of the second sphere.

[0054] Using the first ball coordinates as a standard and combining them with the second ball coordinates, the transformation between the coordinate system of the camera to be calibrated and the reference coordinate system is achieved to obtain the external parameters of the camera to be calibrated.

[0055] The reference camera can be either the left or right eye camera. After obtaining the external parameters of the reference camera, it can be used as a benchmark to obtain the external parameters of the other camera, ensuring that the external parameters of the left and right eye cameras are consistent. The following text will use the left eye camera as the reference camera as an example for detailed explanation.

[0056] The monocular visible light camera is calibrated using the Z-axis coordinates and the image of the checkerboard pattern to obtain the distortion coefficients and internal parameters of the monocular visible light camera. In specific implementation, embodiments of this application can provide the coordinate transformation of the checkerboard pattern at each position relative to the visible light camera, and use the coordinate transformation to calibrate the monocular visible light camera to obtain the distortion coefficients and internal parameters of the monocular visible light camera.

[0057] The Z-axis is controlled to move the calibration plate to the field of view of the calibrated infrared binocular camera. For ease of calculation, this embodiment of the application can also provide control of the Z-axis to move the calibration plate to the middle of the field of view of the calibrated infrared binocular camera.

[0058] Obtain the transformation from the infrared fluorescent sphere coordinate system to the NDI coordinate system acquired by the infrared binocular camera;

[0059] The coordinates of the four corner points in the NDI coordinate system are obtained after the infrared probe of the NDI instrument clicks on the four outermost corner points of the checkerboard. The checkerboard coordinates corresponding to the four corner points are obtained by the monocular visible light camera, so as to obtain the transformation from the checkerboard coordinate system to the NDI coordinate system. In a specific implementation, this application embodiment can provide a ROM file suitable for obtaining the NDI instrument. The ROM file is made by measuring the positions of at least three infrared fluorescent balls.

[0060] Based on the ROM file, the transformation from the infrared fluorescent sphere coordinate system to the NDI coordinate system is obtained.

[0061] The transformation from the checkerboard coordinate system to the infrared fluorescence sphere coordinate system is calculated using the transformations from the infrared fluorescence sphere coordinate system to the NDI coordinate system and the transformation from the checkerboard coordinate system to the NDI coordinate system. Based on this transformation, the coordinate system transformation of the monocular visible light camera relative to one of the infrared binocular cameras is calculated to obtain the external parameters of the monocular visible light camera. Specifically, in this embodiment, the transformation from the checkerboard coordinate system to the NDI coordinate system can be equal to the transformation from the checkerboard coordinate system to the infrared fluorescence sphere coordinate system multiplied by the transformation from the infrared fluorescence sphere coordinate system to the NDI coordinate system.

[0062] The transformation of the monocular visible light camera coordinate system relative to the coordinate system of one of the infrared binocular cameras includes the transformation of the monocular visible light camera coordinate system relative to the coordinate system of the left eye camera of the infrared binocular camera. The It can be expressed by the following formula:

[0063]

[0064] In the formula: N represents the number of measurements; This represents the transformation from the infrared fluorescence sphere coordinate system to the left eye camera coordinate system. This represents the transformation from the checkerboard coordinate system to the infrared fluorescent sphere coordinate system. This represents the transformation from a checkerboard coordinate system to a monocular visible light camera.

[0065] The final coordinate transformation of the monocular visible light camera coordinate system relative to the left eye camera of the infrared binocular camera is obtained by solving the least squares method.

[0066] The integrated calibration system for infrared and visible light cameras provided in this application embodiment uses a specific calibration board and can achieve integrated calibration of infrared and visible light cameras via a 3D coordinate instrument and NDI equipment. By acquiring data from the calibration board, calibration parameters of both the binocular infrared camera and the visible light camera can be obtained simultaneously. The infrared fluorescent balls on the calibration board are fixed at non-intersection points of the checkerboard pattern. This is because the visible light camera is calibrated based on the location of the intersection points, and the distance between the intersection points is constant. If the balls were placed at the intersection points, it would negatively affect the extraction of the intersection points and the calibration calculation.

[0067] When performing calibration, the calibration plate is fixed on the Z-axis of the three-dimensional coordinate instrument. The Z-axis can be moved in space. The three-dimensional coordinate instrument's Z-axis can be moved horizontally, vertically, and horizontally to multiple target positions. At the same time, the coordinates of the three-dimensional coordinate instrument's Z-axis, the image of the infrared fluorescent ball recorded by the infrared binocular camera, and the checkerboard image recorded by the visible light camera are recorded.

[0068] Using the above data, the infrared stereo camera can first be calibrated to obtain its intrinsic and extrinsic parameters and distortion coefficients. The visible light camera can also be calibrated to obtain its intrinsic parameters and distortion coefficients. The calculation methods for the intrinsic parameters and distortion coefficients can be implemented using mature methods currently available in the technology.

[0069] Since the infrared binocular camera and the monocular visible light camera provided in this application embodiment are manufactured as an integrated structure, in order to unify the extrinsic parameters of the infrared binocular camera and the monocular visible light camera, after obtaining the intrinsic and extrinsic parameters and distortion coefficients of the infrared binocular camera and the intrinsic parameters and distortion coefficients of the visible light camera, the calibration plate is placed in the calibrated field of view of the binocular camera, and the four outermost corner points of the checkerboard are clicked with an infrared probe to obtain the coordinates of these four points in the NDI coordinate system. The checkerboard coordinates of the corresponding four points are obtained with the calibrated visible light camera, and the transformation from the checkerboard coordinate system to the NDI coordinate system can be obtained. The infrared binocular camera can obtain the transformation from the current infrared fluorescent sphere coordinate system to the NDI coordinate system, thereby obtaining the transformation from the checkerboard coordinate system to the infrared fluorescent sphere coordinate system, and finally obtaining the extrinsic parameters of the visible light camera.

[0070] The following describes in detail the usage of the system provided in this application embodiment, using the left eye camera as a reference camera for external parameter calculation.

[0071] Step S1: Design a checkerboard pattern visible to a visible light camera, containing N×N squares. Place 3 or 4 infrared fluorescent balls at the non-intersecting points of the straight lines on the checkerboard pattern; the positions of these balls are precisely determined.

[0072] Step S2: Create a ROM file suitable for NDI based on the position of the infrared fluorescent sphere. See [link / reference] Figure 3Taking three balls as an example, the relative positions of the three balls must be precisely fixed, and their coordinates must be set. Input all the coordinates into the NDI 6D Architeect software from NDI Corporation, and the corresponding ROM file can be exported.

[0073] Step S3: Fix the calibration plate onto the Z-axis of the 3D coordinate measuring machine. The 3D coordinate measuring machine moves the calibration plate to multiple target positions within the effective range of the binocular infrared camera and the visible light camera. The Z-axis of the 3D coordinate measuring machine can move precisely forward, backward, left, right, and up and down, allowing for the acquisition of precise spatial coordinates of a specified Z-axis position. By fixing the calibration plate onto the Z-axis of the coordinate measuring machine, as the Z-axis moves, precise coordinates of each intersection point and the center of the small ball on the calibration plate can be obtained at each position.

[0074] The specific method for determining the target location is as follows: the effective working range of the binocular camera in the Z-axis is equally divided into multiple planes, typically nine. A 3D coordinate instrument is used to determine the coordinates of the four corners of each plane. M×M target positions are then equally selected from each plane, with M typically being nine. The 3D coordinate instrument's Z-axis briefly pauses at these locations, usually for one second, during which data is acquired. The spatial coordinates of the 3D coordinate instrument at the stopping point are recorded. Simultaneously, the infrared binocular system acquires images of the infrared fluorescent sphere, and the visible light camera acquires checkerboard images.

[0075] Step S4: Using the data from the 3D coordinate system as a standard, the infrared binocular cameras extract the center coordinates of the sphere as the sphere coordinates based on their respective acquired infrared images. This allows for the transformation of the infrared fluorescent sphere coordinate system at each location relative to the coordinate system of the left or right eye camera. Furthermore, it enables the calibration of the binocular infrared cameras, including the distortion coefficients, internal parameters, and external parameters of each camera. The rotation of the infrared fluorescent sphere coordinate system relative to the coordinate system of the left eye camera can be written as:

[0076] N is the number of measurements.

[0077] It can be determined that the extrinsic parameters of the right-eye infrared camera are the transformation matrix of the right-eye infrared camera r to the left-eye infrared camera l, including translation and rotation matrices.

[0078] Step S5: Based on the checkerboard image obtained by the monocular visible light camera, the checkerboard calibration method can be used to calibrate the monocular visible light camera, and the transformation of the checkerboard at each position relative to the visible light camera can be obtained:

[0079] N is the number of measurements.

[0080] The internal parameters and distortion coefficients of the monocular visible light camera can be obtained.

[0081] Step S6: Place the calibration plate in the center of the calibrated binocular camera's field of view. Based on the ROM file, the transformation from the infrared fluorescence sphere coordinate system to the NDI coordinate system can be obtained.

[0082] Step S7: Use the infrared probe built into the NDI device to click on the four outermost intersections of the checkerboard to obtain their coordinates in the NDI coordinate system.

[0083] Step S8: Use a monocular visible light camera with calibrated intrinsic parameters and distortion coefficients to acquire a checkerboard image and obtain the checkerboard coordinates of the four outermost intersection points.

[0084] Step S9: Using the data obtained in S7 and S8, the transformation from the checkerboard coordinate system to the NDI coordinate system can be obtained.

[0085]

[0086] Step S10, because the transformation from the checkerboard coordinate system to the NDI coordinate system is equal to the transformation from the checkerboard coordinate system to the infrared fluorescent sphere coordinate system multiplied by the transformation from the infrared fluorescent sphere coordinate system to the NDI coordinate system, that is:

[0087]

[0088] This leads to the transformation from the checkerboard coordinate system to the infrared fluorescent sphere coordinate system:

[0089]

[0090] Step S11, the extrinsic parameters of the visible light camera, i.e., the coordinate system transformation T of the visible light camera relative to the left eye camera. l v It can be written as:

[0091] N is the number of measurements.

[0092] Finally, the final result is obtained using the least squares method. Complete the calibration of the external parameters of the monocular visible light camera.

[0093] In summary, the integrated calibration system for infrared and visible light cameras provided in this application has a simple and reasonable structure, is easy to install and use, and employs a calibration plate combining an infrared fluorescent ball and a checkerboard pattern. Combined with a 3D coordinate instrument and an NDI instrument, it can simultaneously calibrate both an infrared binocular camera and a monocular visible light camera in a single data acquisition. Furthermore, it offers high calibration accuracy and good consistency, improving the efficiency of camera parameter calibration. It is worthy of widespread adoption.

[0094] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0095] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of this application.

[0096] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for system or system embodiments, since they are basically similar to method embodiments, the description is relatively simple, and relevant parts can be referred to the descriptions in the method embodiments. The systems and system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0097] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.

Claims

1. A calibration system integrating an infrared camera and a visible light camera, characterized in that, The system includes a processor unit, an optical positioning unit, a calibration plate, a 3D coordinate system, and an NDI instrument. The optical positioning unit includes an infrared binocular camera and a monocular visible light camera. The calibration plate includes a checkerboard visible to the monocular visible light camera and at least three infrared fluorescent spheres positioned at the non-intersecting points of the checkerboard lines. The infrared binocular camera, the monocular visible light camera, the 3D coordinate system, and the NDI instrument are all communicatively connected to the processor unit. The calibration plate is positioned on the Z-axis of the 3D coordinate system. The processor is used to perform the following operations: The Z-axis is controlled to carry the calibration plate to various target positions within the field of view of the infrared binocular camera, and the coordinates of the Z-axis, the image of the infrared fluorescent ball acquired by the infrared binocular camera, and the image of the checkerboard acquired by the monocular visible light camera are recorded at each target position. The infrared binocular camera is calibrated using the Z-axis coordinates and the image of the infrared fluorescent ball to obtain the distortion coefficients, internal parameters, and external parameters of the infrared binocular camera. Using the Z-axis coordinates and the image of the checkerboard pattern, the monocular visible light camera is calibrated using the checkerboard calibration method to obtain the distortion coefficients and internal parameters of the monocular visible light camera. Control the Z-axis to move the calibration plate into the field of view of the calibrated infrared binocular camera; Obtain the transformation from the infrared fluorescent sphere coordinate system to the NDI coordinate system acquired by the infrared binocular camera; The coordinates of the four corner points in the NDI coordinate system are obtained after the infrared probe of the NDI instrument clicks on the four outermost corner points of the checkerboard; the checkerboard coordinates corresponding to the four corner points are obtained by the monocular visible light camera, so as to obtain the transformation from the checkerboard coordinate system to the NDI coordinate system. The transformation from the checkerboard coordinate system to the infrared fluorescent sphere coordinate system is calculated using the transformation from the infrared fluorescent sphere coordinate system to the NDI coordinate system and the transformation from the checkerboard coordinate system to the NDI coordinate system. The coordinate transformation of the monocular visible light camera coordinate system relative to one of the infrared binocular cameras is calculated based on the transformation from the checkerboard coordinate system to the infrared fluorescent sphere coordinate system, so as to obtain the external parameters of the monocular visible light camera.

2. The integrated calibration system for infrared and visible light cameras according to claim 1, characterized in that, The method for determining the target location includes: The effective working range of the infrared binocular camera in the Z direction is divided into multiple planes. The coordinates of the four corners of each plane are determined using a three-dimensional coordinate instrument. M×M positions are taken from each plane as the target positions.

3. The integrated calibration system for infrared and visible light cameras according to claim 2, characterized in that, The value of M is 9.

4. The integrated calibration system for infrared and visible light cameras according to claim 1, characterized in that, The method for obtaining the external parameters corresponding to the infrared binocular camera includes: One of the infrared binocular cameras is designated as the reference camera, and the other camera is designated as the camera to be calibrated. The coordinates of the center of the sphere in the image of the infrared fluorescent sphere at each target location acquired by the reference camera are extracted as the first sphere coordinates; Using the Z-axis coordinates as a standard and combining them with the first small sphere coordinates, the transformation between the reference camera coordinate system and the infrared fluorescent sphere coordinate system is achieved to obtain the external parameters of the reference camera; The coordinates of the center of the infrared fluorescent sphere in the image of each target location acquired by the camera to be calibrated are extracted as the coordinates of the second sphere. Using the first ball coordinates as a standard and combining them with the second ball coordinates, the transformation between the coordinate system of the camera to be calibrated and the reference camera coordinate system is achieved to obtain the external parameters of the camera to be calibrated.

5. The integrated calibration system for infrared and visible light cameras according to claim 1, characterized in that, Obtain the coordinate transformation of the chessboard grid at each position relative to the visible light camera, and use the coordinate transformation to calibrate the monocular visible light camera to obtain the distortion coefficient and internal parameters of the monocular visible light camera.

6. The integrated calibration system for infrared and visible light cameras according to claim 1, characterized in that, Obtain a ROM file suitable for the NDI instrument; the ROM file is created by measuring the positions of at least three infrared fluorescent spheres; Based on the ROM file, the transformation from the infrared fluorescent sphere coordinate system to the NDI coordinate system is obtained.

7. The integrated calibration system for infrared and visible light cameras according to claim 1, characterized in that, The transformation from the checkerboard coordinate system to the NDI coordinate system is equal to the transformation from the checkerboard coordinate system to the infrared fluorescent sphere coordinate system multiplied by the transformation from the infrared fluorescent sphere coordinate system to the NDI coordinate system.

8. The integrated calibration system for infrared and visible light cameras according to claim 1, characterized in that, The transformation of the monocular visible light camera coordinate system relative to the coordinate system of one of the infrared binocular cameras includes the transformation of the monocular visible light camera coordinate system relative to the coordinate system of the left eye camera of the infrared binocular camera. The It can be expressed by the following formula: In the formula: Indicates the number of measurements; This represents the transformation from the infrared fluorescence sphere coordinate system to the left eye camera coordinate system. This represents the transformation from the checkerboard coordinate system to the infrared fluorescent sphere coordinate system. This represents the transformation from a checkerboard coordinate system to a monocular visible light camera.

9. The integrated calibration system for infrared and visible light cameras according to claim 8, characterized in that, The final coordinate transformation of the monocular visible light camera coordinate system relative to the left eye camera of the infrared binocular camera is obtained by solving the least squares method. .

10. The integrated calibration system for infrared and visible light cameras according to claim 1, characterized in that, Control the Z-axis to move the calibration plate to the center of the calibrated infrared binocular camera's field of view.

Images