A method for far field condition calibration of a laser radar and a low-light infrared dual camera

Abstract

The application provides a laser radar and micro-light infrared dual-camera far-field condition calibration method, and aims at the calibration problem of the laser radar and the dual-camera in a helicopter-mounted integrated anti-collision device. The calibration among a collimator, an anti-collision device platform and the laser radar itself is realized, the accurate attitude motion of the anti-collision device is controlled, a plurality of infinite point target images are obtained, the calibration parameters of the infrared camera and the micro-light camera are calculated according to the known motion parameters of the anti-collision device platform and the images, the laser radar is converted into a visible light target through a laser conversion card, the accurate attitude motion of the laser radar is controlled, a plurality of infinite point target images are obtained, the calibration parameters of the laser radar and the micro-light camera are calculated according to the motion parameters of the laser radar and the images, and the calibration of the laser radar and the dual-camera under the infinite working condition is realized.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of dual-camera calibration, and particularly relates to a laser radar and micro-light infrared dual-camera far-field condition calibration method. BACKGROUND

[0002] The anti-collision device configured on a helicopter contains a laser radar, a micro-light camera and an infrared camera. In order to improve the anti-collision capability of the helicopter under the condition of day and night electromagnetic concealment flight, it is necessary to provide the pilot of the helicopter with a laser radar, infrared and micro-light fusion image.

[0003] Image fusion is to fuse the complementary information of different source images together to generate an image with higher quality, more information and clearer image. The premise of the fusion image is to accurately know the position relationship between different photoelectric loads and the internal parameter data of each photoelectric load.

[0004] The anti-collision device is a two-axis pod, and the laser radar, the micro-light and the infrared dual-camera are installed on the inner frame of the anti-collision device. The installation positions and postures of the photoelectric loads are different. Therefore, when the images are directly fused, the error is large, the image fusion precision is poor, and visual dizziness and fatigue are easily caused.

[0005] Camera calibration usually adopts a close distance to place a checkerboard. Different cameras image the same standard checkerboard at a close distance, and the Zhang calibration method is used to solve the calibration parameters. This calibration method is usually suitable for near-field work, and the application extension compatibility is insufficient, and it is difficult to apply to far-field or infinite work conditions. SUMMARY

[0006] (I) Technical problem to be solved

[0007] The application provides a laser radar and micro-light infrared dual-camera far-field condition calibration method to solve the technical problem of how to realize the calibration of the laser radar and the dual-camera under the far-field condition.

[0008] (II) Technical scheme

[0009] In order to solve the above technical problem, the application provides a laser radar and micro-light infrared dual-camera far-field condition calibration method, which comprises the following steps:

[0010] S1. The infrared camera is installed on the inner frame of the anti-collision device, and the anti-collision device and the calibration tool are fixed and installed. The calibration tool containing the anti-collision device is placed in the light path of the collimator, and a circular hole target is selected. The anti-collision device is normally operated, and the attitude of the anti-collision device is adjusted to make the circular hole target in the collimator be at the position (x IR-0 ,y IR-0 ) of the visual axis of the infrared camera calibration;

[0011] S2. Adjust the azimuth angle of the anti-collision device so that the circular hole targets are respectively at the left edge and the right edge of the infrared image, and collect the infrared images at the corresponding positions; according to the centroid processing algorithm, the coordinates (x IR-left ,y IR-left ) and (x IR-right ,y IR-right ) of the circular hole targets in the infrared images are calculated respectively;

[0012] S3. According to the target coordinates on the left and right sides of the infrared image, it is judged whether the circular hole targets are in the same row of the image; if not, the infrared camera row deviation angle Δα is calculated according to formula (1), and the infrared camera roll angle is fine-tuned so that y IR-left =y IR-right , so as to ensure that the azimuth axis of the anti-collision device is perpendicular to the image row of the infrared camera;

[0013]

[0014] wherein f IR is the focal length of the infrared camera;

[0015] S4. Keep the anti-collision device tool posture angle, install the low-light camera in the inner frame of the anti-collision device; the anti-collision device works normally, adjust the azimuth angle of the anti-collision device so that the circular hole targets are respectively at the left edge and the right edge of the low-light image, and collect the low-light images at the corresponding positions; the point target coordinates (x twi-left ,y twi-left ) and (x twi-right ,y twi-right ) in the low-light images are calculated respectively;

[0016] S5. According to the position coordinates of the point targets on the left and right sides of the low-light image, it is judged whether the positions of the point targets on the left and right sides of the low-light image are in the same row; if not, the low-light camera row deviation angle Δβ is calculated according to formula (2), and the low-light camera roll angle is fine-tuned so that y twi-lef =y twi-right , so as to ensure that the azimuth axis of the anti-collision device is perpendicular to the image row of the low-light camera:

[0017]

[0018] wherein f twi is the focal length of the low-light camera;

[0019] S6. Control the azimuth and pitch attitude angle of the anti-collision device, so that the point target from the collimator is located at the upper left corner of the infrared and low-light images; within the field of view of the infrared camera and the low-light camera, control the two-axis platform of the anti-collision device, so that the anti-collision device including the infrared camera and the low-light camera simultaneously performs image acquisition at equal intervals in the heading and pitch directions; wherein the anti-collision device is provided with multiple poses, and at each pose, the low-light and infrared cameras simultaneously acquire images;

[0020] S7. Install the laser radar on the inner frame of the anti-collision device; replace the circular hole target on the focal plane of the collimator with a laser conversion card; reduce the gain of the laser radar to prevent damage to the laser radar during close-range work; control the azimuth angle and scanning angle of the laser radar, so that the laser is uniformly distributed as multiple points in the field of view of the low-light camera after passing through the collimator and the laser conversion card, and record the attitude angle of the laser radar at this time and acquire the low-light image under the corresponding laser radar attitude;

[0021] Extract the centroid coordinates (x1, y1) of the circular hole target acquired by the infrared camera and the centroid coordinates (x2, y2) of the circular hole target acquired by the low-light camera; convert the infrared image coordinates to the low-light image coordinates by formula (3):

[0022]

[0023] Extract the centroid coordinates (x2 / ,y2 / ) of the laser conversion point target acquired by the low-light camera and the laser radar point cloud coordinates (x3, y3); convert the laser radar coordinates (centroid coordinates of the point target) to the low-light image coordinates by formula (4):

[0024]

[0025] wherein H 12 , H 23 are 3x3 matrices, which are the calibration conversion matrices of the infrared camera and the low-light camera and the calibration conversion matrices of the low-light camera and the laser radar, respectively;

[0026] Using the calibration conversion matrices H 12 of the infrared camera and the low-light camera and the calibration conversion matrices H 23 of the low-light camera and the laser radar, calculate the calibration conversion matrices of the infrared camera and the laser radar by formula (5):

[0027]

[0028] At this point, all data calibration is completed.

[0029] Further, in step S4, the centroid processing algorithm is used to calculate the point target coordinates in the low-light image.

[0030] Further, in step S6, the anti-collision device sets 30 poses shown in the following matrix, and in each pose, the micro-light and infrared cameras collect images simultaneously.

[0031]

[0032] Wherein, θ0 represents the initial azimuth angle of the anti-collision device when the target is at the upper left corner; represents the initial pitch angle of the anti-collision device when the target is at the upper left corner; Δθ represents the adjacent interval angle of the target in the heading direction; represents the adjacent interval angle of the target in the pitch direction.

[0033] Further, in step S7, 30 points are evenly distributed in the field of view of the micro-light camera.

[0034] (Three) beneficial effects

[0035] The present application provides a kind of laser radar and micro-light infrared dual camera far field condition calibration method, for the calibration problem of laser radar and dual light camera in helicopter-mounted integrated anti-collision device, utilizes collimator, anti-collision device platform and laser radar itself to realize the calibration between three, by controlling the accurate attitude motion of anti-collision device, obtains multiple images of infinite point target, so as to be known anti-collision device platform motion parameter and image calculates infrared and micro-light camera calibration parameter;By laser conversion card, laser radar is converted into visible light target, by controlling the accurate attitude motion of laser radar, obtains multiple images of infinite point target, so as to be known laser radar motion parameter and image calculates laser radar and micro-light camera calibration parameter, realizes the calibration of laser and dual light camera under infinite working condition. BRIEF DESCRIPTION OF DRAWINGS

[0036] Figure 1 It is the anti-collision device used in the present application;

[0037] Figure 2 It is the far field calibration light path diagram of anti-collision device;

[0038] Figure 3a It is the anti-collision device internal micro-light and infrared camera calibration point distribution schematic diagram, Figure 3b It is the laser radar and micro-light camera calibration point distribution schematic diagram. DETAILED DESCRIPTION

[0039] In order to make the purpose, content and advantages of the present application more clear, the specific embodiments of the present application are described in detail below in combination with the drawings and examples.

[0040] The present application provides a kind of laser radar and micro-light infrared dual camera far field condition calibration method, the calibration specifically includes the following steps:

[0041] S1. as Figure 1As shown, the infrared camera is installed on the inner frame of the anti-collision device, and the anti-collision device is fixedly installed with the calibration tool; as shown Figure 2 As shown, the calibration tool containing the anti-collision device is placed in the light path of the collimator, and a circular hole target is selected; the anti-collision device is normally operated, the posture of the anti-collision device is adjusted, and the circular hole target in the collimator is located at the visual axis position (x IR-0 ,y IR-0 ) of the infrared camera calibration;

[0042] S2. The azimuth angle of the anti-collision device is adjusted, the circular hole target is respectively located at the left edge and the right edge of the infrared image, and the infrared images at the corresponding positions are respectively collected; according to the centroid processing algorithm, the coordinates (x IR-left ,y IR-left ), (x IR-right ,y IR-right ) of the circular hole target in the infrared image are respectively calculated.

[0043] S3. According to the target coordinates on the left and right sides of the infrared image, it is judged whether the circular hole target is located in the same row of the image; if not, the row deviation angle Δα of the infrared camera is calculated according to formula (1), the roll angle of the infrared camera is finely adjusted, y IR-left =y IR-right is ensured, and the azimuth axis of the anti-collision device is perpendicular to the image row of the infrared camera.

[0044]

[0045] Wherein, f IR is the focal length of the infrared camera.

[0046] S4. The posture angle of the anti-collision device tool is kept, the low-light camera is installed on the inner frame of the anti-collision device; the anti-collision device is normally operated, the azimuth angle of the anti-collision device is adjusted, the circular hole target is respectively located at the left edge and the right edge of the low-light image, and the low-light images at the corresponding positions are respectively collected; according to the centroid processing algorithm, the coordinates (x twi-left ,y twi-left ), (x twi-right ,y twi-right ) of the point target in the low-light image are respectively calculated.

[0047] S5. According to the position coordinates of the point target on the left and right sides of the low-light image, it is judged whether the positions of the point target on the left and right sides of the low-light image are located in the same row; if not, the row deviation angle Δβ of the low-light camera is calculated according to formula (2), the roll angle of the low-light camera is finely adjusted, y twi-lef =y twi-right is ensured, and the azimuth axis of the anti-collision device is perpendicular to the image row of the low-light camera.

[0048]

[0049] wherein f twi is the focal length of the low-light camera.

[0050] S6. Control the azimuth and pitch attitude angle of the anti-collision device so that the point target from the collimator is at the upper left corner of the infrared and low-light images; within the field of view of the infrared camera and the low-light camera, control the two-axis platform of the anti-collision device so that the anti-collision device including the infrared camera and the low-light camera simultaneously performs image acquisition at equal intervals in the heading and pitch directions; wherein the anti-collision device is set as shown in the following matrix of 30 poses, at each pose, the low-light and infrared cameras simultaneously acquire images, and the calibration chart is shown in Figure 3a , wherein represents the pitch angle, and 1-azi represents the heading angle.

[0051]

[0052] wherein represents the initial pitch angle of the anti-collision device when the target is at the upper left corner; and represents the initial azimuth angle of the anti-collision device when the target is at the upper left corner; and

[0053] S7. Install the laser radar on the inner frame of the anti-collision device; replace the circular hole target on the focal plane of the collimator with a laser conversion card; reduce the gain of the laser radar to prevent damage to the laser radar in close-range work; control the azimuth angle and the scanning angle of the laser radar so that the laser light is uniformly distributed in 30 points in the field of view of the low-light camera after passing through the collimator and the laser conversion card, as shown in Figure 3b , record the attitude angle of the laser radar at this time and acquire the low-light image under the corresponding laser radar attitude;

[0054] extract the centroid coordinates (x1, y1) of the circular hole target acquired by the infrared camera and the centroid coordinates (x2, y2) of the circular hole target acquired by the low-light camera; convert the infrared image coordinates to the low-light image coordinates by formula (3):

[0055]

[0056] extract the centroid coordinates (x3, y3) of the laser conversion point target acquired by the low-light camera and the laser radar point cloud coordinates; convert the laser radar coordinates (centroid coordinates of the point target) to the low-light image coordinates by formula (4):

[0057]

[0058] wherein H 12 , H 23The matrix is 3*3, and is the calibration conversion matrix of the infrared camera and the low-light camera and the calibration conversion matrix of the low-light camera and the laser radar.

[0059] The calibration conversion matrix H of the infrared camera and the low-light camera is utilized 12 The calibration conversion matrix H of the low-light camera and the laser radar is utilized 23 The calibration conversion matrix H of the infrared camera and the laser radar is calculated by formula (5):

[0060]

[0061] Up to now, all data calibration is completed.

[0062] The above only describes the preferred embodiments of the present application, and it should be noted that, for those skilled in the art, without departing from the technical principles of the present application, a number of improvements and modifications can be made, and these improvements and modifications should be considered as the protection scope of the present application.

Claims

1. A method for calibrating far field conditions of a laser radar and a low-light infrared dual camera, characterized in that, The calibration method comprises the following steps: S1. The infrared camera is mounted on the frame of the anti-collision device to fix the anti-collision device and the calibration tool; the calibration tool containing the anti-collision device is placed in the light path of the collimator, and a circular hole target is selected; the anti-collision device is in normal operation, and the posture of the anti-collision device is adjusted so that the circular hole target in the collimator is in the optical axis position (x IR-0 ,y IR-0 ) of the infrared camera calibration; S2. Adjust the azimuth angle of the anti-collision device so that the circular hole targets are respectively at the left edge and the right edge of the infrared image, and infrared images of the corresponding positions are collected; according to the centroid processing algorithm, the coordinates (x IR-left ,y IR-left ) and (x IR-right ,y IR-right ) of the circular hole targets in the infrared images are respectively calculated; S3. According to the target coordinates of the left and right sides of the infrared image, it is judged whether the circular hole target is in the same row of the image; if not, according to formula (1), the infrared camera row deviation angle Δα is calculated, and the infrared camera roll angle is fine-tuned so that y IR-left = y IR-right , ensuring that the anti-collision device azimuth axis is perpendicular to the infrared camera image row. wherein f IR is the focal length of the infrared camera; S4. Keep the tool posture angle of the anti-collision device, and install the low-light camera in the inner frame of the anti-collision device; the anti-collision device works normally, the azimuth angle of the anti-collision device is adjusted, the circular hole target is respectively located at the left edge and the right edge of the low-light image, and the low-light images of the corresponding positions are collected; the point target coordinates (x twi-left ,y twi-left ) and (x twi-right ,y twi-right ) in the low-light images are respectively calculated; S5. According to the position coordinates of the point target on the left and right sides of the starlight image, it is judged whether the positions of the point target on the left and right sides of the starlight image are in the same row. If not, the starlight camera row deviation angle Δβ is calculated according to formula (2), and the starlight camera roll angle is fine-tuned so that y twi-lef = y twi-right , which ensures that the anti-collision device azimuth axis is perpendicular to the starlight camera image row. wherein f twi is the focal length of the low-light camera; S6. Control the azimuth and pitch attitude angle of the anti-collision device so that the point target from the collimator is located at the upper left corner of the infrared and low-light images; within the field of view of the infrared camera and the low-light camera, control the two-axis platform of the anti-collision device so that the anti-collision device including the infrared camera and the low-light camera simultaneously performs image acquisition at equal intervals in the heading and pitch directions; wherein the anti-collision device is provided with a plurality of poses, and at each pose, the low-light and infrared cameras simultaneously acquire images; S7. Install the laser radar on the inner frame of the anti-collision device; replace the circular hole target on the focal plane of the collimator with a laser conversion card; reduce the gain of the laser radar to prevent damage to the laser radar during close-range work; control the azimuth angle and scanning angle of the laser radar so that the laser is uniformly distributed in the field of view of the low-light camera after passing through the collimator and the laser conversion card, record the attitude angle of the laser radar at this time, and acquire the low-light image under the corresponding attitude of the laser radar; Extract the centroid coordinates (x1, y1) of the circular hole target acquired by the infrared camera and the centroid coordinates (x2, y2) of the circular hole target acquired by the low-light camera; convert the infrared image coordinates to the low-light image coordinates by formula (3): Extracting the centroid coordinates of a laser conversion point target collected by a low-light camera Lidar point cloud coordinates (x3, y3) are converted to low-light image coordinates by equation (4): wherein H 12 , H 23 are 3x3 matrices, respectively, the infrared camera and low-light camera calibration conversion matrix and the low-light camera and lidar calibration conversion matrix; Calibrating a conversion matrix H using an infrared camera and a low-light camera 12 and a low-light camera and a lidar 23 An infrared camera and a lidar calibration conversion matrix H is calculated by equation (5): At this point, all data calibration is completed.

2. The LIDAR and low-light infrared dual-camera far-field condition calibration method of claim 1, wherein, In step S4, the point target coordinates in the low-light image are calculated according to the centroid processing algorithm.

3. The LIDAR and low-light infrared dual-camera far-field condition calibration method of claim 1, wherein, In step S6, the anti-collision device is provided with 30 poses as shown in the following matrix, and at each pose, the low-light and infrared cameras simultaneously acquire images; wherein θ0 represents the initial azimuth angle of the collision avoidance device when the target is in the upper left corner; represents the initial pitch angle of the collision avoidance device when the target is in the upper left corner; Δθ represents the adjacent interval angle of the target in the heading direction; represents the adjacent interval angle of the target in the pitch direction.

4. The LIDAR and low-light infrared dual-camera far-field condition calibration method of claim 3, wherein, In step S7, 30 points are uniformly distributed in the field of view of the low-light camera.

Images