Laser radar and low-light infrared double-camera far field condition calibration method

Abstract

The invention provides a laser radar and low-light infrared dual-camera far-field condition calibration method, aiming at the calibration problem of a laser radar and a dual-light camera in helicopter-borne integrated anti-collision equipment, calibration among a collimator, an anti-collision equipment platform and the laser radar is realized by utilizing the collimator, the anti-collision equipment platform and the laser radar, and the anti-collision equipment is controlled to accurately move in an attitude, so that the anti-collision equipment is accurately calibrated. A plurality of infinite point target images are obtained, and infrared and low-light camera calibration parameters are calculated according to known anti-collision equipment platform motion parameters and images; a laser radar is converted into a visible light target through a laser conversion card, and a plurality of infinite point target images are obtained by controlling accurate attitude motion of the laser radar, so that calibration parameters of the laser radar and a low-light camera are calculated according to motion parameters and images of the laser radar, and calibration of the laser radar and the dual-light camera under the infinite working condition is realized.

Description

Technical Field

[0001] The invention belongs to the technical field of dual-light camera calibration, and in particular relates to a far-field condition calibration method for a laser radar and a low-light-level infrared dual camera. Background Art

[0002] The helicopter's anti-collision equipment includes laser radar, low-light cameras and infrared cameras. In order to improve the helicopter's anti-collision capability during electromagnetic stealth flight day and night, it is necessary to provide helicopter pilots with laser radar, infrared and low-light fusion images.

[0003] Image fusion is the process of fusing complementary information from different source images to generate a higher quality, more informative, and clearer image. The premise of image fusion is to accurately know the positional relationship between different photoelectric loads and their internal parameter data.

[0004] The anti-collision device is a two-axis pod, and the laser radar, low-light and infrared dual-light cameras are installed on the inner frame of the anti-collision device. The installation positions and postures of the various photoelectric loads are different. Therefore, the error is large when the images are directly fused, and the image fusion accuracy is poor, which can easily cause visual dizziness and fatigue.

[0005] Camera calibration usually uses a chessboard placed at a close distance. Different cameras image the standard chessboard at the same close distance, and the calibration parameters are solved by Zhang's calibration method. This calibration method is usually suitable for near-field work, but has insufficient application expansion compatibility and is difficult to apply to far-field or infinite working conditions. Summary of the invention

[0006] 1. Technical issues to be resolved

[0007] The present invention proposes a laser radar and a low-light infrared dual camera far-field condition calibration method to solve the technical problem of how to realize the laser radar and dual-light camera calibration under far-field conditions.

[0008] (II) Technical solution

[0009] In order to solve the above technical problems, the present invention proposes a far-field condition calibration method for a laser radar and a low-light infrared dual camera, the calibration method comprising the following steps:

[0010] S1. Install the infrared camera on the inner frame of the anti-collision device, and fix the anti-collision device and the calibration tooling; place the calibration tooling containing the anti-collision device in the light path of the collimator, and select a circular hole target; the anti-collision device works normally, and adjust the posture of the anti-collision device so that the circular hole target in the collimator is at the visual axis position (x IR-0 ,y IR-0 );

[0011] S2. Adjust the azimuth angle of the anti-collision device so that the circular hole target is located at the left edge and the right edge of the infrared image, and collect infrared images of the corresponding positions respectively; calculate the coordinates of the circular hole target in the infrared image (x IR-left ,y IR-left )、(x IR-right ,y IR-right );

[0012] S3. According to the target coordinates on the left and right sides of the infrared image, determine whether the circular hole target is in the same row of the image; if it is not in the same row, calculate the infrared camera row deviation angle Δα according to formula (1) and fine-tune the infrared camera roll angle so that y IR-left =y IR-right , ensure that the azimuth axis of the anti-collision device is perpendicular to the image line of the infrared camera;

[0013]

[0014] Among them, f IR is the focal length of the infrared camera;

[0015] S4. Maintain the posture angle of the anti-collision equipment tooling and install the low-light camera in the inner frame of the anti-collision equipment. When the anti-collision equipment works normally, adjust the azimuth angle of the anti-collision equipment so that the circular hole target is located at the left edge and the right edge of the low-light image respectively, and collect low-light images at the corresponding positions respectively; calculate the point target coordinates (x twi-left ,y twi-left )、(x twi-right ,y twi-right );

[0016] S5. According to the position coordinates of the point target on the left and right sides of the low-light image, determine whether the positions of the point target on the left and right sides of the low-light image are in the same row; if they are not in the same row, calculate the low-light camera row deviation angle Δβ according to formula (2), and fine-tune the low-light camera roll angle so that y twi-lef =y twi-right , ensure that the azimuth axis of the anti-collision device is perpendicular to the image line of the low-light camera:

[0017]

[0018] Among them, f twi The focal length of the low-light camera;

[0019] S6. Control the azimuth and pitch attitude angle of the anti-collision device so that the point target from the collimator is in the upper left corner of the infrared and low-light images; within the field of view of the infrared camera and the low-light camera, control the two-axis platform of the anti-collision device so that the anti-collision device, which includes both the infrared camera and the low-light camera, collects images at equal intervals in the heading and pitch directions; wherein the anti-collision device is set to multiple postures, and in each posture, the low-light and infrared images are collected simultaneously;

[0020] S7. Install the laser radar on the inner frame of the anti-collision device; replace the circular hole target on the focal plane of the collimator with a laser conversion card; reduce the laser radar gain to prevent damage to the laser radar during close-range operation; control the laser radar azimuth and scanning angle so that the laser passes through the collimator and the laser conversion card and evenly distributes multiple points in the field of view of the low-light camera, record the laser radar attitude angle at this time, and collect the low-light image under the corresponding laser radar attitude;

[0021] Extract the centroid coordinates (x1, y1) of the circular hole target collected by the infrared camera and the centroid coordinates (x2, y2) of the circular hole target collected by the low-light camera; convert the infrared image coordinates to the low-light image coordinates using formula (3):

[0022]

[0023] Extract the centroid coordinates of the laser conversion point target collected by the low-light camera (x2 / ,y2 / ), the laser radar point cloud coordinates (x3, y3), and the laser radar coordinates (the center of mass coordinates of the point target) are converted to the low-light image coordinates through formula (4):

[0024]

[0025] Among them, H 12 , H 23 is a 3×3 matrix, which is the calibration conversion matrix between infrared camera and low-light camera and the calibration conversion matrix between low-light camera and lidar respectively;

[0026] Using infrared camera and low light camera to calibrate the transformation matrix H 12 And the low-light camera and lidar calibration transformation matrix H 23 , through formula (5), the calibration conversion matrix between infrared camera and lidar is calculated:

[0027]

[0028] At this point, all data calibration is completed.

[0029] Furthermore, in step S4, the coordinates of the point targets in the low-light image are calculated and obtained according to the centroid processing algorithm.

[0030] Further, in step S6, the anti-collision device is set to 30 postures as shown in the following matrix, and in each posture, the low light and infrared images are collected simultaneously;

[0031]

[0032] Among them, θ0 represents the initial azimuth of the anti-collision device when the target is in the upper left corner; It indicates the initial pitch angle of the anti-collision device when the target is in the upper left corner; Δθ indicates the adjacent interval angle of the target in the heading direction; Indicates the interval angle between adjacent targets in the pitch direction.

[0033] Further, in step S7, 30 points are evenly distributed in the field of view of the low-light camera.

[0034] (III) Beneficial effects

[0035] The present invention proposes a laser radar and low-light-level infrared dual-camera far-field condition calibration method. Aiming at the laser radar and dual-light camera calibration problem in helicopter-borne integrated anti-collision equipment, the present invention utilizes a collimator, an anti-collision equipment platform and the laser radar itself to realize calibration among the three. By controlling the precise posture movement of the anti-collision equipment, a plurality of infinitely distant point target images are obtained, thereby calculating the infrared and low-light-level camera calibration parameters from the known anti-collision equipment platform motion parameters and images. The laser radar is converted into a visible light target through a laser conversion card, and by controlling the precise posture movement of the laser radar, a plurality of infinitely distant point target images are obtained, thereby calculating the laser radar and low-light-level camera calibration parameters from the laser radar motion parameters and images, thereby realizing the calibration of the laser radar and the dual-light camera under infinitely distant working conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 is the anti-collision device used in the present invention;

[0037] Figure 2 Calibrate the optical path diagram for the far field of the anti-collision equipment;

[0038] Figure 3a This is a schematic diagram of the calibration point distribution of the low light and infrared camera inside the anti-collision device. Figure 3b Schematic diagram of the calibration point distribution of lidar and low-light camera. DETAILED DESCRIPTION

[0039] In order to make the purpose, content and advantages of the present invention more clear, the specific implementation methods of the present invention are further described in detail below in conjunction with the drawings and examples.

[0040] This embodiment proposes a far-field condition calibration method for a laser radar and a low-light-level infrared dual camera, and the calibration specifically includes the following steps:

[0041] S1. Figure 1As shown, install the infrared camera on the inner frame of the anti-collision device, and install and fix the anti-collision device and the calibration tooling; Figure 2 As shown in the figure, the calibration fixture containing the anti-collision device is placed in the optical path of the collimator, and a circular hole target is selected; the anti-collision device works normally, and the posture of the anti-collision device is adjusted so that the circular hole target in the collimator is at the visual axis position (x IR-0 ,y IR-0 );

[0042] S2. Adjust the azimuth angle of the anti-collision device so that the circular hole target is located at the left edge and the right edge of the infrared image, and collect infrared images of the corresponding positions respectively; calculate the coordinates of the circular hole target in the infrared image (x IR-left ,y IR-left )、(x IR-right ,y IR-right );

[0043] S3. According to the target coordinates on the left and right sides of the infrared image, determine whether the circular hole target is in the same row of the image; if it is not in the same row, calculate the infrared camera row deviation angle Δα according to formula (1) and fine-tune the infrared camera roll angle so that y IR-left =y IR-right , ensure that the azimuth axis of the anti-collision device is perpendicular to the image line of the infrared camera;

[0044]

[0045] Among them, f IR is the focal length of the infrared camera.

[0046] S4. Keep the posture angle of the anti-collision equipment tooling and install the low-light camera in the inner frame of the anti-collision equipment. When the anti-collision equipment works normally, adjust the azimuth angle of the anti-collision equipment so that the circular hole target is located at the left edge and the right edge of the low-light image respectively, and collect low-light images at the corresponding positions respectively. According to the centroid processing algorithm, the coordinates of the point target in the low-light image (x twi-left ,y twi-left )、(x twi-right ,y twi-right );

[0047] S5. According to the position coordinates of the point target on the left and right sides of the low-light image, determine whether the positions of the point target on the left and right sides of the low-light image are in the same row; if they are not in the same row, calculate the low-light camera row deviation angle Δβ according to formula (2), and fine-tune the low-light camera roll angle so that y twi-lef =y twi-right , ensure that the azimuth axis of the anti-collision device is perpendicular to the image line of the low-light camera:

[0048]

[0049] Among them, f twi The focal length of the low-light camera.

[0050] S6. Control the azimuth and pitch attitude angle of the anti-collision device so that the point target from the collimator is in the upper left corner of the infrared and low-light images; within the field of view of the infrared camera and the low-light camera, control the two-axis platform of the anti-collision device so that the anti-collision device, which includes both the infrared camera and the low-light camera, collects images at equal intervals in the heading and pitch directions; the anti-collision device is set to 30 postures as shown in the following matrix. In each posture, the low-light and infrared images are collected simultaneously. The calibration diagram is shown in Figure 3a , in the figure represents the pitch angle, θ 1-azi Indicates the heading angle;

[0051]

[0052] Among them, θ0 represents the initial azimuth of the anti-collision device when the target is in the upper left corner; It indicates the initial pitch angle of the anti-collision device when the target is in the upper left corner; Δθ indicates the adjacent interval angle of the target in the heading direction; Indicates the adjacent interval angle of the target in the pitch direction;

[0053] S7. Install the laser radar on the inner frame of the anti-collision equipment; replace the circular hole target on the focal plane of the collimator with a laser conversion card; reduce the laser radar gain to prevent damage to the laser radar during close-range operation; control the laser radar azimuth and scanning angle so that the laser passes through the collimator and laser conversion card and is evenly distributed at 30 points in the field of view of the low-light camera, such as Figure 3b As shown, the laser radar attitude angle at this time is recorded and the low-light image under the corresponding laser radar attitude is collected;

[0054] Extract the centroid coordinates (x1, y1) of the circular hole target collected by the infrared camera and the centroid coordinates (x2, y2) of the circular hole target collected by the low-light camera; convert the infrared image coordinates to the low-light image coordinates using formula (3):

[0055]

[0056] Extract the centroid coordinates of the laser conversion point target collected by the low-light camera The laser radar point cloud coordinates (x3, y3) are converted to low-light image coordinates using formula (4):

[0057]

[0058] Among them, H 12 , H 23It is a 3×3 matrix, which is the calibration conversion matrix between infrared camera and low-light camera and the calibration conversion matrix between low-light camera and lidar.

[0059] Using infrared camera and low light camera to calibrate the transformation matrix H 12 And the low-light camera and lidar calibration transformation matrix H 23 , through formula (5), the calibration conversion matrix between infrared camera and lidar is calculated:

[0060]

[0061] At this point, all data calibration is completed.

[0062] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A far-field condition calibration method for a laser radar and a low-light-level infrared dual camera, characterized in that: The calibration method comprises the following steps: S1. Install the infrared camera on the inner frame of the anti-collision device, and fix the anti-collision device and the calibration tooling; place the calibration tooling containing the anti-collision device in the light path of the collimator, and select a circular hole target; the anti-collision device works normally, and adjust the posture of the anti-collision device so that the circular hole target in the collimator is at the visual axis position (x IR-0 ,y IR-0 ); S2. Adjust the azimuth angle of the anti-collision device so that the circular hole target is located at the left edge and the right edge of the infrared image, and collect infrared images of the corresponding positions respectively; calculate the coordinates of the circular hole target in the infrared image (x IR-left ,y IR-left )、(x IR-right ,y IR-right ); S3. According to the target coordinates on the left and right sides of the infrared image, determine whether the circular hole target is in the same row of the image; if it is not in the same row, calculate the infrared camera row deviation angle Δα according to formula (1) and fine-tune the infrared camera roll angle so that y IR-left =y IR-right , ensure that the azimuth axis of the anti-collision device is perpendicular to the image line of the infrared camera; Among them, f IR is the focal length of the infrared camera; S4. Maintain the posture angle of the anti-collision equipment tooling and install the low-light camera in the inner frame of the anti-collision equipment. When the anti-collision equipment works normally, adjust the azimuth angle of the anti-collision equipment so that the circular hole target is located at the left edge and the right edge of the low-light image respectively, and collect low-light images at the corresponding positions respectively; calculate the point target coordinates (x twi-left ,y twi-left )、(x twi-right ,y twi-right ); S5. According to the position coordinates of the point target on the left and right sides of the low-light image, determine whether the positions of the point target on the left and right sides of the low-light image are in the same row; if they are not in the same row, calculate the low-light camera row deviation angle Δβ according to formula (2), and fine-tune the low-light camera roll angle so that y twi-lef =y twi-right , ensure that the azimuth axis of the anti-collision device is perpendicular to the image line of the low-light camera: Among them, f twi The focal length of the low-light camera; S6. Control the azimuth and pitch attitude angle of the anti-collision device so that the point target from the collimator is in the upper left corner of the infrared and low-light images; within the field of view of the infrared camera and the low-light camera, control the two-axis platform of the anti-collision device so that the anti-collision device, which includes both the infrared camera and the low-light camera, collects images at equal intervals in the heading and pitch directions; wherein the anti-collision device is set to multiple postures, and in each posture, the low-light and infrared images are collected simultaneously; S7. Install the laser radar on the inner frame of the anti-collision device; replace the circular hole target on the focal plane of the collimator with a laser conversion card; reduce the laser radar gain to prevent damage to the laser radar during close-range operation; control the laser radar azimuth and scanning angle so that the laser passes through the collimator and the laser conversion card and evenly distributes multiple points in the field of view of the low-light camera, record the laser radar attitude angle at this time, and collect the low-light image under the corresponding laser radar attitude; Extract the centroid coordinates (x1, y1) of the circular hole target collected by the infrared camera and the centroid coordinates (x2, y2) of the circular hole target collected by the low-light camera; convert the infrared image coordinates to the low-light image coordinates using formula (3): Extract the centroid coordinates of the laser conversion point target collected by the low-light camera The laser radar point cloud coordinates (x3, y3) are converted to low-light image coordinates using formula (4): Among them, H 12 , H 23 is a 3×3 matrix, which is the calibration conversion matrix between infrared camera and low-light camera and the calibration conversion matrix between low-light camera and lidar respectively; Using infrared camera and low light camera to calibrate the transformation matrix H 12 And the low-light camera and lidar calibration transformation matrix H 23 , through formula (5), the calibration conversion matrix between infrared camera and lidar is calculated: At this point, all data calibration is completed.

2. The far-field condition calibration method of the laser radar and low-light-level infrared dual camera according to claim 1 is characterized in that: In step S4, the coordinates of the point targets in the low-light image are calculated according to the centroid processing algorithm.

3. The far-field condition calibration method of the laser radar and low-light-level infrared dual camera according to claim 1 is characterized in that: In step S6, the anti-collision device is set to 30 postures as shown in the following matrix. In each posture, the low light and infrared images are collected simultaneously; Among them, θ0 represents the initial azimuth of the anti-collision device when the target is in the upper left corner; It indicates the initial pitch angle of the anti-collision device when the target is in the upper left corner; Δθ indicates the adjacent interval angle of the target in the heading direction; Indicates the interval angle between adjacent targets in the pitch direction.

4. The far-field condition calibration method of the laser radar and low-light-level infrared dual camera as claimed in claim 3 is characterized in that: In step S7, 30 points are evenly distributed in the field of view of the low-light camera.

Images