Target high-precision positioning method based on distortion calibration and matching DEM

Abstract

A target high-precision positioning method based on distortion calibration and matching DEM of the present application comprises S1, after the camera is installed to the unmanned aerial vehicle platform, the whole imaging system and the carrier platform are tested as a system, the preliminary statistical calibration is carried out in the room, and the distortion rate of each pixel point is determined; S2, the conversion relationship of the image coordinate system to the geographic coordinate system is established, the camera field of view angle, the focal length and the resolution content are converted and solved with the ground shooting field of view range; S3, the relationship of the unmanned aerial vehicle, the camera and the image is transformed; S4, the elevation DEM data based on the latitude and longitude information is loaded, and the elevation of the target point is determined; S5, the system error is statistically analyzed in combination with the outdoor calibration, and the latitude and longitude process is increased to solve the system error compensation quantity; S6, the high-precision position information of the target is obtained based on S3, S4 and S5. The present application can greatly reduce the target positioning difficulty, and fully utilize the camera internal parameter and DEM data to achieve the effects of reducing the gimbal load, reducing the cost and operation difficulty.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of passive target positioning, and particularly relates to a target high-precision positioning method based on distortion calibration and matching DEM. BACKGROUND

[0002] In air-to-ground imaging applications, it is crucial to accurately position ground targets. Existing air-to-ground imaging target positioning methods have the problem of insufficient precision. These methods usually adopt the methods of "image discovery + laser irradiation + platform attitude" and "feature calibration + platform attitude + pixel position". In the method of "image discovery + laser irradiation + platform attitude", a visible light / infrared camera is usually used in the image discovery link. After the target is discovered, a laser range finder is used to measure the distance, and the position is calculated in combination with the platform attitude information. This method has a complex hardware structure, and the dynamic error of the air-to-ground platform attitude is large, which further affects the calculation of the position information. In the method of "feature calibration + platform attitude + pixel position", the key ground features are set in the field, and the pixel position is calibrated through image feature matching with the assistance of the platform position and attitude. The target position is calculated through the pixel position relationship. This method mainly uses the relative position relationship of the target and the calibration object in the image for target positioning. The hardware structure is simple, but the target positioning precision is relatively poor due to the influence of camera lens distortion and the high correlation between pixel size and platform attitude. Therefore, the present application improves the target positioning model from the physical and algorithmic levels on the basis of a full analysis of the camera imaging characteristics, and innovatively proposes a monocular high-precision target positioning method, which greatly improves the target positioning precision on the basis of simplifying the hardware structure. SUMMARY

[0003] The target high-precision positioning method, device and storage medium based on distortion calibration and matching DEM proposed by the present application can at least solve one of the technical problems in the background art. It is a monocular image target high-precision positioning method based on camera distortion calibration and matching DEM elevation data. The camera imaging distortion is calibrated, then the target latitude and longitude coordinate position is calculated using the pixel position relationship, and the target is accurately positioned by matching the DEM elevation data.

[0004] To achieve the above-mentioned purpose, the present application adopts the following technical solutions:

[0005] A target high-precision positioning method based on distortion calibration and matching DEM, comprising the following steps,

[0006] S1, after the camera is installed on the unmanned aerial vehicle platform, the entire imaging system and the carrier platform are tested as a system, and preliminary statistical calibration is performed indoors to determine the distortion rate of each pixel point;

[0007] S2, based on S1, the conversion relationship between the image coordinate system and the geographic coordinate system is established, the camera field of view, focal length and resolution content are converted and calculated with the ground shooting field of view range;

[0008] S3, based on S2, the unmanned aerial vehicle, camera and image relationship are transformed;

[0009] S4, load the elevation DEM data based on latitude and longitude information, and determine the elevation of the target point;

[0010] S5, the system error is statistically analyzed in combination with outdoor calibration, and the system error compensation amount is added in the latitude and longitude process;

[0011] S6, based on S3, S4 and S5, the high-precision position information of the target is obtained.

[0012] In another aspect, the application also discloses a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to make the processor execute the steps of the above method.

[0013] In another aspect, the application also discloses a computer device, which comprises a memory and a processor, the memory stores a computer program, and the computer program is executed by the processor to make the processor execute the steps of the above method.

[0014] From the above technical solution, the target high-precision positioning method based on distortion calibration and matching DEM of the application is based on the positioning of the target by using the unmanned aerial vehicle carrying the single-mode camera, the camera and the holder are adapted and installed on the unmanned aerial vehicle. Since there are installation errors between the camera lens and the sensor, the camera and the holder and the holder and the aircraft attitude, the shooting image and the ideal image exist geometric deformation, so that the target position calculated by relying on the pixel position relationship exists larger positioning error, especially the image edge, the distortion rate is larger, the positioning error is even higher than dozens of meters, which seriously affects the target positioning accuracy. The application calibrates the camera imaging distortion, then calculates the latitude and longitude coordinate position of the target by using the pixel position relationship, and matches the DEM elevation data to complete the accurate positioning of the target.

[0015] The present application breaks through the traditional "image discovery + laser irradiation + platform posture" method in principle, reduces the overload of the unmanned aerial vehicle gimbal, and reduces the complexity of the equipment. In addition, the calibration parameters (camera internal parameters such as focal length, resolution and pixel size, etc.) of the camera when it leaves the factory are fully utilized, and a pure image pixel target high-precision positioning method is proposed. In addition, combined with the standard test chart, the distortion of the gimbal camera group is calibrated, and the positioning precision loss caused by the deviation of the image optical axis distortion error is effectively reduced, so as to determine the longitude and latitude information of the target point. Finally, the height data and the longitude and latitude of the target are highly coupled in the satellite map, and the update time is fast, and the resolution can reach meters according to different map sources, which can meet the demand of daily field operation for target height data. Through the above scheme, the target positioning difficulty can be greatly reduced, and the camera internal parameters and DEM data are fully utilized to reduce the gimbal load, reduce the cost and operation difficulty. BRIEF DESCRIPTION OF DRAWINGS

[0016] Figure 1 The camera distortion calibration test module is for

[0017] Figure 2 The unmanned aerial vehicle imaging and positioning physical model is for

[0018] Figure 3 The unmanned aerial vehicle pixel coordinate transformation schematic diagram is for

[0019] Figure 4 The image plane target position schematic diagram is for

[0020] Figure 5 The image plane and the unmanned aerial vehicle posture position relationship is for

[0021] Figure 6 The method flowchart of the present application is for DETAILED DESCRIPTION

[0022] In order to make the purpose, technical scheme and advantages of the embodiments of the present application more clear, the technical scheme in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all the embodiments.

[0023] As Figure 6 shown, the target high-precision positioning method based on distortion calibration and matching DEM described in the present embodiment includes the following steps,

[0024] S1, after the camera is installed on the unmanned aerial vehicle platform, the whole imaging system and the carrier platform are tested as a system, and the distortion rate of each pixel point is determined by preliminary statistical calibration in the room;

[0025] S2, based on S1, the conversion relationship between the image coordinate system and the geographic coordinate system is established, the camera field of view, focal length and resolution content are converted and calculated with the ground shooting field of view range;

[0026] S3, based on S2, the unmanned aerial vehicle, camera and image relationship are transformed;

[0027] S4, load the elevation DEM data based on latitude and longitude information, determine the elevation of the target point;

[0028] S5, combining outdoor calibration, statistical analysis of system error, and solving the latitude and longitude process to increase the system error compensation;

[0029] S6, based on S3, S4 and S5, the high-precision position information of the target is obtained.

[0030] Specifically, the following steps are executed,

[0031] 1. Indoor distortion calibration of imaging platform

[0032] Due to various physical installation errors and other reasons, the images taken by the camera are not ideal, generally there are radial distortion and tangential distortion, and they are nonlinear changes. After the camera is installed on the unmanned aerial vehicle platform, the entire imaging system and the carrier platform are tested as a system. Indoor preliminary statistical calibration is performed to determine the distortion rate of each pixel point;

[0033] Step 1: Adjust the aircraft attitude (hover) and gimbal attitude (horizontal zero position) to make the camera align with the chart (such as SFRplus test chart, checkerboard or dot matrix test chart), and ensure that the chart fills the field of view. Take a picture;

[0034] Step 2: Import the captured image into Imatest software, automatically detect the image feature points, and calculate the image distortion coefficient; as shown in Figure 1

[0035] Step 3: Use the distortion rate to map the distorted pixels to the ideal coordinate system through reverse transformation, calculate the displacement (Euclidean distance) of each pixel as the distortion degree, and obtain the pixel array distortion table A;

[0036] 2. Establish the conversion relationship between geographic coordinates and image coordinates

[0037] The pixel position in the image is used to calculate the position information of the target in the geographic coordinate system. The conversion relationship between the image coordinate system and the geographic coordinate system needs to be established. For ideal shooting conditions, the imaging schematic diagram is as shown in Figure 2 、 Figure 3

[0038] ​​Through the above relationship between the air-to-ground imaging image coordinate system, the camera field of view angle, focal length, and resolution can be converted and calculated with the ground shooting field of view range.

[0039] 3. UAV, camera, image relationship transformation

[0040] Assuming that the UAV and the camera are rigidly connected, the distance deviation between the camera and the UAV can be ignored, and the UAV's own attitude position information (Ux lon , Uy lat , Uz h ) can be used instead of the camera position information (where Ux lon is the longitude coordinate, Uy lat is the latitude coordinate, and Uz h is the elevation data), and the aircraft orientation is 0° north and α° deviation (yaw angle), and the camera pitch angle is β° (pitch angle). The image center and the field of view center position relationship can be calculated:

[0041] The camera projection distance from the field of view center in the geodetic coordinate system is D C :

[0042]

[0043] According to the aircraft orientation, the longitude and latitude of the field of view center point can be determined:

[0044]

[0045] In the formula, R is the average radius of the earth, about 6371.393 km; Cx lon is the longitude coordinate of the field of view center point, Cy lat is the latitude coordinate of the field of view center point, α° is the yaw angle, and D C is the camera projection distance from the field of view center in the geodetic coordinate system.

[0046] After the above position relationship is determined, the target area image can be shot at this attitude, and the image center point longitude and latitude coordinates are known. At this time, the target point M in the image coordinate system is determined in the image coordinate system. The pixel coordinates of the target point M are (M i , M j ): as shown in the following formula: Figure 4

[0047] Assuming that the image pixel is (W x H), where W is the number of image row pixel points, and H is the number of image column pixel points; the pixel coordinate position of the M point in the image is (i, j), and the pixel size is (W pix , H pix ), where W pix is the row single pixel size, and H pix ​is the column pixel size; f is the focal length of the sensor, and δ is the field of view angle W ,δ H , where δ W is the field of view angle of the camera in the row direction, and δ H is the field of view angle of the camera in the column direction, then the angles of the field of view deviating from the optical axis in the i and j directions are:

[0048]

[0049] where δ Wi is the offset angle of the target in the row direction, and δ Hj is the offset angle of the camera in the column direction.

[0050] The distances of W and H from the center are:

[0051]

[0052] where,

[0053] For a detector, the pixel size is generally difficult to measure, but the field of view angle and the focal length of the camera are generally known. Therefore, the pixel size can be calculated using the field of view angle and the focal length. Assuming that the image pixel is (W x H), the field of view angle in the row direction is δ W , the field of view angle in the column direction is δ H , and f is the focal length of the camera, then the size of each pixel is:

[0054]

[0055] The distance deviation of the target point from the center of the field of view is D MC :

[0056]

[0057] where D Wi and D Hj are the pixel distances of the target point to the row and column coordinate axes, respectively, and then converted to the geodetic coordinate system, as shown in Figure 5

[0058] Therefore, the longitude and latitude of the target point M converted to the geodetic coordinate system are:

[0059]

[0060] where α is the yaw angle of the unmanned aerial vehicle gimbal.

[0061] 4. Elevation DEM data loading based on longitude and latitude information

[0062] ​In some areas, the elevation data is generally kept relatively stable, for this, download the elevation data DEM of some area, index the elevation data according to the longitude and latitude information of the target point, and thus determine the elevation of the target point, and finally the position information of the target point is: (M lon ,M lat ,M DEM )

[0063] 5. Outdoor multi-pose compensation quantity statistical analysis and correction

[0064] The above model is solved on the premise of theoretical analysis, and there is a certain system error when the distance is converted into longitude and latitude coordinates, therefore, the system error is statistically analyzed in combination with outdoor calibration, and the system error compensation quantity is added for solving the longitude and latitude. In a known region (the longitude and latitude information of each point in the region is known) as an outdoor calibration environment, five landmark objects are set in the region at a height of 100m, 200m, and the pan-tilt attitude angle is 90°, 60°, 45°, and the target region image (including the landmark) is shot under the condition, the longitude and latitude information of each landmark is solved by using the above solving process, and is compared with the actual longitude and latitude information, and the longitude compensation quantity γ lon and the latitude compensation quantity γ lat (an average distance in each direction) are determined respectively.

[0065] 6. High-precision target positioning

[0066] According to the above method, the high-precision position information of the target can be obtained as follows:

[0067]

[0068] As described above, the embodiment of the present application breaks through the traditional method of “image discovery + laser irradiation + platform attitude” in principle, reduces the overload of the unmanned aerial vehicle pan-tilt, and reduces the complexity of the equipment. In addition, the calibration parameters (camera intrinsic parameters such as focal length, resolution and pixel size) of the camera when it leaves the factory are fully utilized, and a pure image pixel target high-precision positioning method is proposed; in addition, the distortion of the pan-tilt camera group is calibrated in combination with the standard test chart, and the positioning precision loss caused by the deviation of the image optical axis distortion error is effectively reduced, and thus the longitude and latitude information of the target point is determined. Finally, the elevation data and the longitude and latitude of the target are highly coupled in the satellite map, and the update time is fast, and the resolution can reach meters according to different map sources, which can meet the demand of target elevation data in daily field operation. Through the above scheme, the target positioning difficulty can be greatly reduced, and the camera intrinsic parameters and DEM data are fully utilized to achieve the effect of reducing the load of the pan-tilt, reducing the cost and operation difficulty.

[0069] In another aspect, the present application also discloses a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to make the processor execute the steps of the above method.

[0070] In still another aspect, the present application also discloses a computer device comprising a memory and a processor, wherein the memory stores a computer program, and the computer program is executed by the processor to make the processor execute the steps of the above method.

[0071] In yet another embodiment provided in the present application, a computer program product containing instructions is also provided, which, when executed on a computer, causes the computer to execute the target high-precision positioning method based on distortion calibration and matching DEM in any of the above embodiments.

[0072] It can be understood that the system, device and storage medium provided by the embodiments of the present application correspond to the method provided by the embodiments of the present application, and the explanation, examples and beneficial effects of the related content can refer to the corresponding part in the above method.

[0073] In the above embodiments, the system, device and storage medium provided by the embodiments of the present application can be realized by software, hardware, firmware or any combination thereof, in whole or in part. When realized by software, it can be realized in the form of a computer program product in whole or in part. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the flow or function described in the embodiments of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center through wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) mode. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. integrated with one or more available media. The available medium can be a magnetic medium (for example, floppy disk, hard disk, magnetic tape), optical medium (for example, DVD), or semiconductor medium (for example, solid state disk (SSD)) and the like.

[0074] It is to be noted that, in the present text, the relative terms such as first and second, and the like are used merely to differentiate one entity or operation from another entity or operation, without necessarily requiring or implying any such actual relationship or order between such entities or operations. Moreover, the terms "comprising", "including", or any other variant thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the statement "comprising a" does not exclude the existence of additional identical elements in the process, method, article, or apparatus that includes the element.

[0075] Each of the embodiments in the present specification is described in a relevant manner, and the same or similar parts among the embodiments can be referred to each other. Each of the embodiments focuses on the difference from other embodiments. In particular, for the system embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the part of the description of the method embodiments.

[0076] The above embodiments are only used to illustrate the technical solutions of the present application, rather than limiting them. Although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacements for some technical features. Such modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A target high-precision positioning method based on distortion calibration and matching DEM, characterized in that, It comprises the following steps, S1, after the camera is installed to the unmanned aerial vehicle platform, the whole imaging system is tested with the carrier platform as a system, the preliminary statistical calibration is carried out in the room, and the distortion rate of each pixel point is determined; S2, the conversion relationship from the image coordinate system to the geographic coordinate system is established based on S1, the field angle, the focal length and the resolution content of the camera are converted and solved with the ground shooting field range in the coordinate; S3, the unmanned aerial vehicle, the camera and the image relationship are transformed based on S2; S4, the DEM data based on the latitude and longitude information is loaded, and the elevation of the target point is determined; S5, the system error is statistically analyzed in combination with the outdoor calibration, and the system error compensation quantity is solved in the latitude and longitude process; S6, the high-precision position information of the target is obtained based on S3, S4 and S5; Step S1 comprises the following steps, Step 1: adjust the aircraft attitude and the gimbal attitude, make the camera align with the picture card, ensure that the picture card fills the field of view, and shoot the image; Step 2: import the shot image into Imatest software, automatically detect the image feature points, and calculate the image distortion coefficient; Step 3: using the distortion rate, the distorted pixels are mapped to the ideal coordinate system through reverse transformation, the displacement of each pixel is calculated as the distortion degree, and the pixel array distortion table is obtained; Step S3 specifically comprises, Assuming the drone and camera remain rigidly connected, the distance between the camera and the drone is negligible, and the camera's position information is replaced by the drone's own attitude position information where, is the longitude coordinate, is the latitude coordinate, is the elevation data, and the plane heading is 0° with true north, the yaw angle °, the camera's pitch angle °, the camera's roll angle, the image center position relationship with the field of view center can be calculated as follows, then the camera projects in the geodetic coordinate system a distance from the center of the field of view : According to the aircraft orientation, the latitude and longitude of the field center point can be determined: where R is the mean radius of the earth; is the longitude coordinate of the center point of the field of view, is the latitude coordinate of the center point of the field of view, is the yaw angle, is the projection distance of the camera from the center of the field of view in the geodetic coordinate system; After the above position relationship is determined, the target region image is captured in this posture, and the longitude and latitude coordinates of the image center point are known. At this time, the coordinate position of the target point M in the pixel point of the image coordinate system is determined in the image ; Assume that the image pixels are WxH, wherein W is the number of image row pixel points, and H is the number of image column pixel points; the pixel coordinate position of the M point in the image is (i, j), and the pixel size is (W pix , H pix ), wherein W pix is the row single pixel size, and H pix is the column pixel size; the sensor focal length is f, and the field of view angle is , wherein is the row direction camera field of view angle, is the column direction camera field of view angle, and the angles of the field of view deviating from the optical axis in the i and j directions are as follows: wherein is an offset angle of the camera in the row direction, is an offset angle of the camera in the column direction; Then the distance of W and H from the center is respectively: Wherein, For the detector, the size of the pixel is calculated by the field of view angle and the focal length. Assuming the image pixel is W x H, the field of view angle in the row direction is , the field of view angle in the column direction is , and f is the focal length of the camera, then the size of each pixel is: Then the distance deviation of the target point from the center point of the field of view is : where D Wi and D Hj are the pixel distances of the target point to the row and column axes, respectively, converted to the geodetic coordinate system. Then, the latitude and longitude of the target point M converted to the geodetic coordinate are: wherein is the yaw angle of the UAV gimbal.

2. The target high-precision positioning method based on distortion calibration and matching DEM according to claim 1, characterized in that: S5 specifically comprises, With a known region, i.e. the latitude and longitude information of each point in the region is known as an outdoor calibration environment, five landmark features are set in the region, respectively at a height of 100 meters, 200 meters, and the gimbal attitude angle is 90°, 60°, 45°. The target region image containing the markers is shot under the condition, the latitude and longitude information of each marker is solved, and compared with the actual latitude and longitude information, respectively, to determine the longitude compensation amount and the latitude compensation amount .

3. The method according to claim 2, wherein: The high-precision position information of the target obtained in S6 is: 。

Images