A method for calibrating laser optical axis deviation through map matching in the air

Abstract

A method for calibrating the deviation of the laser optical axis in the air through map matching. The method relates to the technical field of error calibration of optoelectronic pods, and specifically to the technical field of calibrating the deviation of the laser optical axis in the air through map matching. It effectively solves the problem that the existing calibration method requires more precise calibration equipment and is difficult to avoid re-drifting during subsequent long-term use, thereby improving the efficiency of calibrating the deviation of the laser optical axis. The method includes the following steps: aligning the data time series of the aircraft and the pod; preparing map slices around the airport; obtaining a set of real-shot images and a set of real-shot data containing the target, and pre-processing the set of real-shot images; obtaining the true latitude and longitude of the image center through map matching; and inferring the deviation of the laser optical axis. The present invention can be applied in the field of error calibration of optoelectronic pods.

Description

Technical Field

[0001] The present invention relates to the technical field of photoelectric pod error calibration, and in particular to the technical field of calibrating laser optical axis deviation through map matching in the air. Background Art

[0002] When an airborne electro-optical pod illuminates a ground target with a laser, deviations in the laser's optical axis significantly reduce illumination accuracy, thus affecting the drone's ability to function properly. Previous methods for calibrating the laser's optical axis required the assistance of sophisticated calibration equipment, typically requiring at least a set of instruments capable of observing the laser spot. This calibration process is time-consuming and labor-intensive, requiring such equipment to be completed in a field environment with minimal operating conditions. Furthermore, after calibration, some laser axes can gradually shift over time due to insufficiently secure mounting. In these cases, even if calibration has already been completed, achieving high illumination accuracy during flight is difficult. Given these current circumstances, there is an urgent need to develop a method for calibrating laser optical axis deviation that can be performed quickly and without the need for additional equipment. Summary of the Invention

[0003] In response to the above problems, the present invention discloses a method for calibrating the laser optical axis deviation in the air through map matching, which relates to the technical field of optoelectronic pod error calibration. It effectively solves the problem that the existing calibration method requires more precise calibration equipment and is difficult to avoid re-drift during subsequent long-term use, thereby improving the efficiency of calibrating the laser optical axis deviation.

[0004] The method comprises the following steps:

[0005] S1. Data timing alignment between aircraft and pod;

[0006] S2. Prepare map slices around the airport;

[0007] S3, obtaining a real-shot image set and a real-shot data set containing the target, and preprocessing the real-shot image set;

[0008] S4. Obtain the true latitude and longitude of the image center through map matching;

[0009] S5. Determine the deviation of the laser optical axis.

[0010] Furthermore, the data timing alignment between the aircraft and the pod is specifically as follows: the pod is installed on the aircraft, and the timing alignment between the integrated navigation device on the aircraft and the various sensors inside the pod is completed.

[0011] Furthermore, the preparation of map slices around the airport is specifically as follows: taking the airport as the center point, delineating a map with a latitude and longitude range of 1°×1°, and dividing it into j×j map slices.

[0012] Furthermore, the acquisition of the real-shot image set and the real-shot data set containing the target is specifically: when the aircraft is in a stable flight state, image tracking is started on the target, and continuous laser irradiation is performed to obtain the real-shot image set and the real-shot data set; the real-shot data set specifically includes: position data of the aircraft integrated navigation device, attitude data of the aircraft integrated navigation device, attitude data of the pod encoder and laser data.

[0013] Furthermore, the preprocessing of the real-shot image set is specifically: dehazing the real-shot image set through a fast dark channel algorithm, and then enhancing the real-shot image set through a multi-detail layer sharpening and contrast-limited adaptive histogram equalization algorithm to obtain a preprocessed real-shot image set.

[0014] Furthermore, the method of obtaining the true longitude and latitude of the image center through map matching is specifically as follows: geometrically correcting the preprocessed real-shot image set to obtain a real-shot image set to be matched, using a feature point matching algorithm to complete the matching of the real-shot image set to be matched with the map slice, and filtering out erroneous matching points to obtain a transmission matrix; by solving the transmission matrix, the true longitude and latitude and height [Bt Lt Ht] of the real-shot image center are obtained; the method of filtering out erroneous matching points is: a random sampling consistency algorithm.

[0015] Furthermore, the process of inferring the deviation of the laser optical axis is specifically as follows: When K converges to the minimum value, the actual pixel offset of the laser optical axis [pm pn] can be calculated; k = [1, 2, ..., n], n represents the total time of laser firing, pm is the horizontal axis offset pixel, pn is the vertical axis offset pixel, and Dl k represents the laser distance value at time k, and Dt k Indicates the target distance at the current moment.

[0016] Further, the described is the latitude and longitude of the aircraft at time k, The latitude and longitude of the target that the laser optical axis is actually aimed at.

[0017] The beneficial effects of the present invention are:

[0018] This invention can be applied to the field of photoelectric pod error calibration, resolving the issues of existing calibration methods that require relatively sophisticated calibration equipment and are difficult to avoid re-drifting during extended use. During a single flight, laser optical axis deviation can be calibrated in just a few minutes without excessive human intervention, providing excellent technical support for subsequent functional implementation during the flight. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1The figure is a flow chart of the method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0020] The technical solution of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0021] This embodiment provides a method for calibrating laser optical axis deviation in the air by map matching. The flow chart of the method is as follows: Figure 1 , the method comprises the following steps:

[0022] S1. Data timing alignment between aircraft and pod.

[0023] The relevant operations of step S1 are introduced with a specific example:

[0024] Install the pod on the aircraft and complete the timing alignment between the integrated navigation equipment on the aircraft and the various sensors inside the pod.

[0025] S2. Prepare a map dataset around the airport.

[0026] The relevant operations of step S2 are introduced with a specific example:

[0027] With the airport as the center point, a map with a latitude and longitude range of 1°×1° is delineated (the default aircraft stable flight altitude range is 5000 to 10000 meters. If it is lower or higher than this altitude, the latitude and longitude range can be appropriately reduced or expanded). The map is divided into 40×40 map slices, and the digital elevation model files corresponding to the map slices are obtained by cropping and merging. The map dataset includes map slices and digital elevation model files.

[0028] S3. Obtain a real-shot image set and a real-shot data set containing the target, and preprocess the real-shot image set.

[0029] The relevant operations of step S3 are introduced with a specific example:

[0030] When the aircraft is in a stable flight state, image tracking is turned on for the target, and continuous laser irradiation is performed, and a real-shot image set and a real-shot data set are obtained simultaneously, which lasts for 3 to 5 minutes. The real-shot data set specifically includes: the position data of the aircraft integrated navigation device {[Ba 1 La 1 Ha 1 ],[Ba 2 La 2 ha 2 ],…,[Bak La k Ha k ],…}, attitude data of aircraft integrated navigation equipment The attitude data of the pod encoder {[α 1 β 1 ],[α 2 β 2 ],…,[α k β k ],…} and laser data {Dl 1 ,Dl 2 ,…,Dl k ,…}; Among them, Ba k 、La k 、Ha k 、 θa k , γa k , α k , β k 、Dl k They represent the aircraft latitude, aircraft longitude, aircraft altitude, aircraft heading, aircraft pitch, aircraft roll, encoder azimuth, encoder pitch and laser ranging value at time k respectively.

[0031] The preprocessing of the real-shot image set is specifically as follows: first, the real-shot images are preprocessed by defogging, enhancement, etc., including defogging based on the fast dark channel algorithm and enhancement processing based on multi-detail layer sharpening and contrast limited histogram equalization algorithm CLAHE, so as to achieve the purpose of increasing effective feature points.

[0032] The fast dark channel method can perform image dehazing in real time. First, a dark channel map is obtained from the foggy image (i.e., a set of real-time images). Then, the transmittance and atmospheric light are estimated based on the dark channel map and the transmittance is refined. Finally, a clear dehazed image is restored using a fog degradation model. Specifically, assuming that the atmospheric light value A has been obtained and the transmittance t(x) remains constant within the filter window Ω(x), dividing both sides of the formula by A and performing a minimum operation yields:

[0033]

[0034] Among them, c∈(r,g,b) represents the RGB three channels, A c is the dark channel value of the foggy image, I c (x) is the theoretical restored image, J c (x) is a foggy image;

[0035] Then use minimum filtering to get:

[0036]

[0037] According to the dark channel prior theory, the dark channel graph can be obtained as follows:

[0038]

[0039] Among them, J c (y) is a foggy image, and combining formulas (1) to (3) yields:

[0040]

[0041] By introducing a constant parameter ω (0<ω<1) from formula (4), the final transmittance is obtained as:

[0042]

[0043] In actual operation, a lower limit threshold t0 is generally set for the transmittance t(x). If the obtained transmittance is less than the threshold t0, t(x) = t0, and the final image restoration formula is:

[0044]

[0045] Among them, I(x) is the restored image, and the dehazed image can be obtained through the above steps.

[0046] The dehazed image is further processed using multi-detail layer sharpening and contrast-limited histogram equalization (CLAHE). Specifically, the dehazed image is processed using a 5×5 Gaussian filter to obtain base layer 1; the image is processed using a 21×21 Gaussian filter to obtain base layer 2; the original image and base layer 1 are processed using the difference method to obtain detail layer 1; the original image and base layer 2 are processed using the difference method to obtain detail layer 2; the original image and detail layers 1 and 2 are fused using the following formula to obtain the sharpened image:

[0047] I final =3*I original -G 5×5 (I original )-G 21×21 (I original ) (7) Among them, I final is the final sharpened image, I original is the original image (i.e. the image after defogging), G 5×5 (I original ) represents the base layer 1, G 21×21 (I original ) represents the base layer 2; the sharpened image is a set of pre-processed real-shot images.

[0048] S4. Obtain the true latitude and longitude of the image center through map matching.

[0049] The relevant operations of step S4 are introduced with a specific example:

[0050] According to the position and posture data synchronized with the real-shot images and the camera calibration parameters, the pre-processed real-shot image set is geometrically corrected to obtain the real-shot image set to be matched, and the feature point matching algorithm SuperGlue is used to complete the matching of the real-shot image set to be matched with the map slice, and then the erroneous matching points are screened out to obtain the transmission matrix. By solving the transmission matrix, the true latitude and longitude [Bt Lt Ht] of the center of the real-shot image is obtained; the feature point matching algorithm is specifically: using the SuperPoint algorithm to extract feature points, and then using the Sinkhorn algorithm to calculate the similarity between feature points and determine whether they are matching feature point pairs; the method for screening out erroneous matching points is: random sampling consensus algorithm RANSAC; in actual algorithm application, the matching success rate of the algorithm can be made higher by setting a soft threshold, wherein the matching of the map slice is based on the position and posture information at the same time as the current image, plus the initial positioning result obtained by iteratively obtaining the digital elevation model file.

[0051] S5. Determine the deviation of the laser optical axis.

[0052] The relevant operations of step S5 are introduced with a specific example:

[0053] Applying the passive geolocation algorithm based on the earth ellipsoid model, we first assume that the pixel offset is [pm pn], where pm is the horizontal axis offset pixel and pn is the vertical axis offset pixel, and substitute it into the passive positioning function: This is used to calculate the geographic coordinates of the laser optical axis at this moment Where f is the focal length of the camera, dα k and dβ k The current aircraft position and attitude information and the true latitude and longitude of the image center obtained by map matching can be used to calculate [Bt LtHt]; Substitute into formula (8) to solve the target latitude and longitude that the laser optical axis actually aims at:

[0054]

[0055] The latitude and longitude of the target actually aimed at by the laser optical axis are the geographical coordinates of the target aimed at by the assumed laser optical axis in the geodetic rectangular coordinate system, where N is the radius of curvature of the earth's ellipsoid and e is the first eccentricity.

[0056] The process of inferring the deviation of the laser optical axis is specifically as follows: The deviation of the laser optical axis can be inferred. When K converges to the minimum value, the actual pixel offset of the laser optical axis [pm pn] can be obtained; k = [1, 2, ..., n], n represents the total time of laser firing, and Dt k Indicates the target distance at the current moment, The current target distance represents the distance between the ground position actually aimed at by the laser optical axis and the current position of the pod, taking into account the pixel offset [pm pn] of the laser optical axis. is the latitude and longitude of the aircraft at time k, The latitude and longitude of the aircraft at the time k represents the geographical coordinates of the aircraft at the current time in the geodetic rectangular coordinate system; Indicates the latitude and longitude of the target that the laser optical axis is actually aiming at. The latitude and longitude of the target actually aimed at by the laser optical axis are the geographical coordinates of the target in the geodetic rectangular coordinate system.

Claims

1. A method for calibrating laser optical axis deviation by map matching in the air, characterized in that: The method comprises the following steps: S1. Data timing alignment between aircraft and pod; S2. Prepare map slices around the airport; S3, obtaining a real-shot image set and a real-shot data set containing the target, and preprocessing the real-shot image set; S4. Obtain the true latitude and longitude of the image center through map matching; The method of obtaining the true latitude, longitude, and height of the image center by map matching specifically comprises: geometrically correcting the pre-processed real-shot image set to obtain a real-shot image set to be matched; matching the real-shot image set to be matched with the map slice using a feature point matching algorithm; filtering out erroneous matching points to obtain a transmission matrix; and solving the transmission matrix to obtain the true latitude, longitude, and height [Bt Lt Ht] of the real-shot image center; the method of filtering out erroneous matching points is: a random sampling consistency algorithm; S5. Determine the deviation of the laser optical axis; Applying the passive geolocation algorithm based on the earth ellipsoid model, we first assume that the pixel offset is [pm pn], where pm is the horizontal axis offset pixel and pn is the vertical axis offset pixel, and substitute it into the passive positioning function: This is used to calculate the geographic coordinates of the laser optical axis at this moment Where f is the focal length of the camera, dα k and dβ k The current aircraft position and attitude information and the true latitude and longitude of the image center obtained by map matching can be used to calculate [Bt LtHt]; Substitute into the following formula to solve the target latitude and longitude that the laser optical axis is actually aiming at: The latitude and longitude of the target actually aimed at by the laser optical axis are the geographical coordinates of the target aimed at by the assumed laser optical axis in the geodetic rectangular coordinate system, where N is the radius of curvature of the earth's ellipsoid and e is the first eccentricity; The process of inferring the deviation of the laser optical axis is specifically as follows: The deviation of the laser optical axis can be inferred. When K converges to the minimum value, the actual pixel offset of the laser optical axis [pm pn] can be obtained; k = [1, 2, ..., n], n represents the total time of laser firing, pm is the horizontal axis offset pixel, pn is the vertical axis offset pixel, and Dl k represents the laser distance value at time k, and Dt k Indicates the target distance at the current moment; described described is the latitude and longitude of the aircraft at time k, The latitude and longitude of the target that the laser optical axis is actually aimed at.

2. The method for calibrating laser optical axis deviation by map matching in the air according to claim 1, characterized in that: The data timing alignment between the aircraft and the pod is specifically as follows: the pod is installed on the aircraft, and the timing alignment between the integrated navigation device on the aircraft and the various sensors inside the pod is completed.

3. The method for calibrating laser optical axis deviation by map matching in the air according to claim 1, characterized in that: The preparation of map slices around the airport is specifically as follows: taking the airport as the center point, demarcating a map with a latitude and longitude range of 1°×1°, and dividing it into j×j map slices.

4. The method for calibrating laser optical axis deviation by map matching in the air according to claim 1, characterized in that: The method of obtaining a real-shot image set and a real-shot data set containing the target specifically comprises: when the aircraft is in a stable flight state, starting image tracking of the target and performing continuous laser irradiation to obtain the real-shot image set and the real-shot data set; the real-shot data set specifically comprises: position data of the aircraft integrated navigation device, attitude data of the aircraft integrated navigation device, attitude data of the pod encoder, and laser data.

5. The method for calibrating laser optical axis deviation by map matching in the air according to claim 4, characterized in that: The preprocessing of the real-shot image set is specifically: defogging the real-shot image set through a fast dark channel algorithm, and then enhancing the real-shot image set through a multi-detail layer sharpening and contrast-limited histogram equalization algorithm to obtain a preprocessed real-shot image set.

Images