Method for calibrating laser optical axis deviation through map matching in air

Abstract

The invention discloses a method for calibrating laser optical axis deviation through map matching in the air. Relates to the technical field of photoelectric pod error calibration, in particular to the technical field of calibration of laser optical axis deviation through map matching in the air. The method effectively solves the problems that an existing calibration method needs precise calibration equipment, and secondary deviation in subsequent long-time use is difficult to avoid, and improves the efficiency of calibrating the deviation of the laser optical axis. The method comprises the following steps: aligning data time sequences of an airplane and a pod; preparing map slices around the airport; acquiring a real shot image set and a real shot data set containing a target, and preprocessing the real shot image set; obtaining the real longitude and latitude height of the image center through map matching; and reversely deducing the deviation of the laser optical axis. The method can be applied to the field of photoelectric pod error calibration.

Description

Technical Field

[0001] The invention relates to the technical field of photoelectric pod error calibration, and in particular to the technical field of calibrating laser optical axis deviation through map matching in the air. Background Art

[0002] When the airborne optoelectronic pod irradiates the ground target with a laser, the deviation of the laser optical axis greatly reduces the irradiation accuracy, thus affecting the realization of the corresponding functions of the drone. Previous laser optical axis calibration methods require the assistance of precise calibration equipment, and generally require at least a set of instruments that can observe the laser spot. Such a calibration process is time-consuming and labor-intensive, and requires such equipment to complete the calibration work in a simple field environment; on the other hand, after the calibration is completed, as time goes by, some laser optical axes will gradually deviate due to insufficient installation. In this case, even if the calibration has been completed before, it is difficult to achieve a high irradiation accuracy during the flight. In view of these current situations, it is urgent to develop a method that can quickly calibrate the deviation of the laser optical axis without the need for additional equipment. Summary of the invention

[0003] In response to the above problems, the present invention discloses a method for calibrating the laser optical axis deviation in the air through map matching, which relates to the technical field of optoelectronic pod error calibration. It effectively solves the problem that the existing calibration method requires more precise calibration equipment and is difficult to avoid re-drift in subsequent long-term use, thereby improving the efficiency of calibrating the laser optical axis deviation.

[0004] The method comprises the following steps:

[0005] S1. Data timing alignment between aircraft and pod;

[0006] S2. Prepare map slices around the airport;

[0007] S3, obtaining a real-shot image set and a real-shot data set containing the target, and preprocessing the real-shot image set;

[0008] S4, obtaining the true latitude and longitude of the image center through map matching;

[0009] S5. Infer the deviation of the laser optical axis.

[0010] Furthermore, the data timing alignment between the aircraft and the pod is specifically as follows: the pod is installed on the aircraft, and the timing alignment between the integrated navigation device on the aircraft and each sensor inside the pod is completed.

[0011] Furthermore, the preparation of map slices around the airport is specifically as follows: taking the airport as the center point, delineating a map with a latitude and longitude range of 1°×1°, and dividing it into j×j map slices.

[0012] Furthermore, the acquisition of the real-shot image set and the real-shot data set containing the target is specifically: when the aircraft is in a stable flight state, image tracking is started on the target, and continuous laser irradiation is performed to obtain the real-shot image set and the real-shot data set; the real-shot data set specifically includes: position data of the aircraft integrated navigation device, attitude data of the aircraft integrated navigation device, attitude data of the pod encoder, and laser data.

[0013] Furthermore, the preprocessing of the real-shot image set is specifically: defogging the real-shot image set through a fast dark channel algorithm, and then enhancing the real-shot image set through a multi-detail layer sharpening and contrast-limited adaptive histogram equalization algorithm to obtain a preprocessed real-shot image set.

[0014] Furthermore, the method of obtaining the true longitude and latitude of the center of the image through map matching is specifically as follows: geometrically correcting the preprocessed real-shot image set to obtain the real-shot image set to be matched, using a feature point matching algorithm to complete the matching of the real-shot image set to be matched with the map slice, and filtering out erroneous matching points to obtain a transmission matrix, and by solving the transmission matrix, obtaining the true longitude and latitude [Bt Lt Ht] of the center of the real-shot image; the method of filtering out erroneous matching points is: a random sampling consistency algorithm.

[0015] Furthermore, the process of inferring the deviation of the laser optical axis is specifically as follows: When K converges to the minimum value, the actual pixel offset of the laser optical axis [pm pn] can be calculated; k = [1, 2, ..., n], n represents the total time of laser firing, pm is the horizontal axis offset pixel, pn is the vertical axis offset pixel, and Dl k represents the laser distance measurement value at time k, and Dt k Indicates the target distance at the current moment.

[0016] Further, the Said is the latitude and longitude of the aircraft at time k, It is the latitude and longitude of the target that the laser optical axis is actually aiming at.

[0017] The beneficial effects of the present invention are:

[0018] The present invention can be applied in the field of photoelectric pod error calibration, solving the problem that the existing calibration method requires more precise calibration equipment and is difficult to avoid re-drifting in subsequent long-term use. During the flight, it only takes a few minutes and does not require much human intervention to complete the calibration of the laser optical axis deviation, providing good technical support for the subsequent function realization of the flight process. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1The present invention is a flowchart of the method described in the embodiment of the present invention. DETAILED DESCRIPTION

[0020] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0021] This embodiment provides a method for calibrating laser optical axis deviation in the air by map matching. The flowchart of the method is as follows: Figure 1 , the method comprises the following steps:

[0022] S1. Data timing alignment between aircraft and pod.

[0023] The relevant operations of step S1 are introduced with a specific example:

[0024] Install the pod on the aircraft and complete the timing alignment between the integrated navigation equipment on the aircraft and the various sensors inside the pod.

[0025] S2. Prepare a map dataset around the airport.

[0026] The relevant operations of step S2 are introduced with a specific example:

[0027] With the airport as the center point, a map with a latitude and longitude range of 1°×1° is delineated (the default aircraft stable flight altitude range is 5000 to 10000 meters. If it is lower or higher than this altitude, the latitude and longitude range can be appropriately reduced or expanded), which is divided into 40×40 map slices, and the digital elevation model files corresponding to the map slices are obtained by cropping and merging. The map dataset includes map slices and digital elevation model files.

[0028] S3. Obtain a real-shot image set and a real-shot data set containing the target, and preprocess the real-shot image set.

[0029] The relevant operations of step S3 are introduced with a specific example:

[0030] When the aircraft is in a stable flight state, image tracking is turned on for the target, and continuous laser irradiation is performed, and a real-shot image set and a real-shot data set are obtained at the same time, which lasts for 3 to 5 minutes; the real-shot data set specifically includes: the position data of the aircraft integrated navigation device {[Ba 1 La 1 Ha 1 ],[Ba 2 La 2 ha 2 ],…,[Bak La k Ha k ],…}, attitude data of aircraft integrated navigation equipment The attitude data of the pod encoder {[α 1 β 1 ],[α 2 β 2 ],…,[α k β k ],…} and laser data {Dl 1 ,Dl 2 ,…,Dl k ,…}; among them, Ba k ,La k ,Ha k , θa k , γa k , α k , β k 、Dl k They respectively represent the aircraft latitude, aircraft longitude, aircraft altitude, aircraft heading, aircraft pitch, aircraft roll, encoder azimuth, encoder pitch and laser ranging value at time k.

[0031] The preprocessing of the real-shot image set is specifically as follows: first, the real-shot images are preprocessed such as defogging and enhancement, including defogging based on a fast dark channel algorithm and enhancement processing based on multi-detail layer sharpening and contrast limited histogram equalization algorithm CLAHE, so as to achieve the purpose of increasing effective feature points.

[0032] The fast dark channel method can perform image defogging in real time. First, a dark channel map is obtained based on the foggy image (i.e., a set of real-shot images), and then the transmittance and atmospheric light are estimated based on the dark channel map and the transmittance is refined. Finally, a clear defogging image is restored through the foggy degradation model. Specifically, assuming that the atmospheric light value A has been obtained and the transmittance t(x) remains unchanged within the filter window Ω(x), both sides of the formula are divided by A and a minimum operation is performed to obtain:

[0033]

[0034] Among them, c∈(r,g,b) represents the RGB three channels, A c is the dark channel value of the foggy image, I c (x) is the theoretical restored image, J c (x) is a foggy image;

[0035] Then use minimum filtering to get:

[0036]

[0037] According to the dark channel prior theory, the dark channel diagram can be obtained as follows:

[0038]

[0039] Among them, J c (y) is a foggy image. Combining formulas (1) to (3), we get:

[0040]

[0041] A constant parameter ω (0<ω<1) is introduced from formula (4) to obtain the final transmittance:

[0042]

[0043] In actual operation, a lower limit threshold t0 is generally set for the transmittance t(x). If the obtained transmittance is less than the threshold t0, t(x) = t0, and the final image restoration formula is:

[0044]

[0045] Among them, I(x) is the restored image, and the dehazed image can be obtained through the above steps.

[0046] The dehazed image is further processed using multi-detail layer sharpening and contrast limited histogram equalization algorithm CLAHE. Specifically, the dehazed image is processed using a 5×5 Gaussian filter to obtain base layer 1; the image is processed using a 21×21 Gaussian filter to obtain base layer 2; the original image and base layer 1 are processed using the difference method to obtain detail layer 1; the original image and base layer 2 are processed using the difference method to obtain detail layer 2; the original image and detail layers 1 and 2 are fused using the following formula to obtain the sharpened image:

[0047] I final =3*I original -G 5×5 (I original )-G 21×21 (I original ) (7) Among them, I final is the final sharpened image, I original is the original image (i.e. the image after defogging), G 5×5 (I original ) represents the base layer 1, G 21×21 (I original ) represents the base layer 2; the sharpened image is a set of real-shot images after preprocessing.

[0048] S4. Obtain the true latitude and longitude of the image center through map matching.

[0049] The relevant operations of step S4 are introduced with a specific example:

[0050] According to the position and posture data synchronized with the real-shot image and the camera calibration parameters, the pre-processed real-shot image set is geometrically corrected to obtain the real-shot image set to be matched, and the feature point matching algorithm SuperGlue is used to complete the matching of the real-shot image set to be matched with the map slice, and then the wrong matching points are screened out to obtain the transmission matrix. By solving the transmission matrix, the real longitude and latitude [Bt Lt Ht] of the center of the real-shot image is obtained; the feature point matching algorithm is specifically: using the SuperPoint algorithm to extract feature points, and then using the Sinkhorn algorithm to calculate the similarity between feature points and determine whether they are matching feature point pairs; the method for screening out wrong matching points is: random sampling consensus algorithm RANSAC; in the actual application of the algorithm, the matching success rate of the algorithm can be higher by setting a soft threshold, wherein the matching of the map slice is based on the position and posture information at the same time as the current image, plus the initial positioning result obtained by iterating the digital elevation model file.

[0051] S5. Infer the deviation of the laser optical axis.

[0052] The relevant operations of step S5 are introduced with a specific example:

[0053] Apply the passive geographic positioning algorithm based on the earth ellipsoid model. First, assume that the pixel offset is [pm pn], where pm is the horizontal axis offset pixel and pn is the vertical axis offset pixel, and substitute it into the passive positioning function: This is used to calculate the geographic coordinates of the laser optical axis at this moment. Where f is the focal length of the camera, dα k and dβ k The current aircraft position and attitude information and the true latitude and longitude of the image center obtained by map matching can be used to calculate [Bt LtHt]; Substitute into formula (8) to solve the target latitude and longitude that the laser optical axis is actually aiming at:

[0054]

[0055] The latitude and longitude of the target actually aimed at by the laser optical axis are the geographical coordinates of the target aimed at by the assumed laser optical axis in the geodetic rectangular coordinate system, where N is the radius of curvature of the earth's ellipsoid and e is the first eccentricity.

[0056] The process of inferring the deviation of the laser optical axis is specifically as follows: The deviation of the laser optical axis can be inferred. When K converges to the minimum value, the actual pixel offset of the laser optical axis [pm pn] can be calculated; k = [1, 2, ..., n], n represents the total time of laser firing, and Dt k Indicates the target distance at the current moment. The target distance at the current moment represents the distance between the ground position actually aimed at by the laser optical axis and the current position of the pod under the premise of considering the pixel offset [pm pn] of the laser optical axis. is the latitude and longitude of the aircraft at time k, The latitude and longitude of the aircraft at time k represents the geographical coordinates of the aircraft at the current time in the geodetic rectangular coordinate system; Indicates the latitude and longitude of the target that the laser optical axis is actually aiming at. The latitude and longitude of the target actually aimed at by the laser optical axis are the geographic coordinates of the target in the earth's rectangular coordinate system.

Claims

1. A method for calibrating laser optical axis deviation in the air by map matching, characterized in that: The method comprises the following steps: S1. Data timing alignment between aircraft and pod; S2. Prepare map slices around the airport; S3, obtaining a real-shot image set and a real-shot data set containing the target, and preprocessing the real-shot image set; S4, obtaining the true latitude and longitude of the image center through map matching; S5. Infer the deviation of the laser optical axis.

2. The method for calibrating laser optical axis deviation in the air by map matching according to claim 1, characterized in that: The data timing alignment between the aircraft and the pod is specifically as follows: the pod is installed on the aircraft, and the timing alignment between the integrated navigation device on the aircraft and each sensor inside the pod is completed.

3. The method for calibrating laser optical axis deviation in the air by map matching according to claim 1, characterized in that: The preparation of map slices around the airport is specifically as follows: taking the airport as the center point, delineating a map with a latitude and longitude range of 1°×1°, and dividing it into j×j map slices.

4. The method for calibrating laser optical axis deviation in the air by map matching according to claim 1, characterized in that: The method of obtaining the real-shot image set and the real-shot data set containing the target is specifically as follows: when the aircraft is in a stable flight state, image tracking is started for the target, and continuous laser irradiation is performed to obtain the real-shot image set and the real-shot data set; the real-shot data set specifically includes: position data of the aircraft integrated navigation device, attitude data of the aircraft integrated navigation device, attitude data of the pod encoder, and laser data.

5. The method for calibrating laser optical axis deviation in the air by map matching according to claim 4, characterized in that: The preprocessing of the real-shot image set is specifically: defogging the real-shot image set through a fast dark channel algorithm, and then enhancing the real-shot image set through multi-detail layer sharpening and contrast-limited histogram equalization algorithm to obtain the preprocessed real-shot image set.

6. The method for calibrating the laser optical axis deviation in the air by map matching according to claim 5, characterized in that: The method of obtaining the true longitude and latitude of the center of the image through map matching is specifically as follows: geometrically correcting the preprocessed real-shot image set to obtain the real-shot image set to be matched, using a feature point matching algorithm to complete the matching of the real-shot image set to be matched with the map slice, and filtering out erroneous matching points to obtain a transmission matrix, and by solving the transmission matrix, obtaining the true longitude and latitude [Bt Lt Ht] of the center of the real-shot image; the method of filtering out erroneous matching points is: a random sampling consistency algorithm.

7. The method for calibrating laser optical axis deviation in the air by map matching according to claim 6, characterized in that: The process of inferring the deviation of the laser optical axis is specifically as follows: The deviation of the laser optical axis can be inferred. When K converges to the minimum value, the actual pixel offset of the laser optical axis [pm pn] can be calculated; k = [1, 2, ..., n], n represents the sum of the laser firing moments, pm is the horizontal axis offset pixel, pn is the vertical axis offset pixel, and Dl k represents the laser distance measurement value at time k, and Dt k Indicates the target distance at the current moment.

8. The method for calibrating laser optical axis deviation in the air by map matching according to claim 7, characterized in that: Said Said is the latitude and longitude of the aircraft at time k, It is the latitude and longitude of the target that the laser optical axis is actually aiming at.

Images