A method for calibrating unmanned aerial vehicle hyperspectral camera mounting parameters

Abstract

The application discloses a calibration method based on unmanned aerial vehicle hyperspectral camera installation parameters, and relates to the field of hyperspectral cameras. The method comprises the following steps: acquiring ground point clouds of a calibration area, flight altitude of an unmanned aerial vehicle and state data of a hyperspectral camera; based on the flight altitude, camera position, attitude data and initial installation parameters, the hyperspectral image is point clouded to generate image point clouds; based on linear feature views in the ground point clouds and corresponding linear feature views in the image point clouds, the matching error of the linear feature calibration object is calculated; and based on the matching error, the initial installation parameters are dynamically adjusted until the matching error is less than an error threshold, so that the optimal installation parameters of the hyperspectral camera are obtained. The application can improve the precision of parameter calibration.

Description

Technical Field

[0001] This application relates to the field of hyperspectral camera technology, and in particular to a calibration method based on the installation parameters of a hyperspectral camera on a UAV. Background Technology

[0002] With the development of drone technology, the method of using drones as platforms to carry hyperspectral sensors for low-altitude data acquisition has been widely adopted, offering advantages such as lower cost, more flexible operation, lower flight altitude, and higher data resolution. Drone hyperspectral cameras have become one of the important low-altitude data acquisition devices.

[0003] The UAV hyperspectral camera system mainly consists of a linear array camera and an integrated navigation satellite system. During the integration of the navigation satellite system, there are certain coordinate and attitude eccentricities (i.e., camera mounting parameters) between the hyperspectral camera and the inertial measurement unit (IMU) within the system. Therefore, calibrating the UAV hyperspectral camera mounting parameters is crucial for obtaining high-precision hyperspectral imagery. In many cases, most mounting parameter calibration methods involve acquiring coordinate information from imagery and corresponding points in the field, constructing equations, and performing calculations. However, since linear and angular eccentricities have a certain correlation with the results, the location and distribution of control points can influence parameter calculations and even lead to incorrect parameters. The data acquisition workload for corresponding control points during calibration is substantial, resulting in low parameter calibration accuracy. Summary of the Invention

[0004] The purpose of this application is to provide a calibration method based on the installation parameters of a UAV hyperspectral camera, which can improve the accuracy of installation parameter calibration.

[0005] To achieve the above objectives, this application provides the following solution.

[0006] This application provides a calibration method based on the installation parameters of a UAV hyperspectral camera, including: Acquire ground point cloud data of the calibration area, UAV flight altitude, and status data of the hyperspectral camera; the status data includes: hyperspectral images and corresponding camera position and attitude data; Based on the flight altitude, the camera position, the attitude data, and the initial placement parameters, the hyperspectral image is converted into a point cloud to generate an image point cloud; the initial placement parameters are set values. Based on the view of the linear feature site in the ground point cloud and the corresponding view of the linear feature site in the image point cloud, the matching error of the linear feature marker is calculated. The initial placement parameters are dynamically adjusted based on the matching error until the matching error is less than the error threshold, thus obtaining the optimal placement parameters for the hyperspectral camera.

[0007] According to the specific embodiments provided in this application, this application has the following technical effects: This application, based on flight altitude, camera position, attitude data, and initial placement parameters, generates an image point cloud by converting a hyperspectral image into a point cloud. This integrates the exterior orientation elements of the hyperspectral image into each pixel data, achieving a unified format and providing data support for subsequent matching error calculation accuracy. Then, by comparing the views of linear feature sites in the ground point cloud of the calibration area with the views of the corresponding linear feature sites in the image point cloud, the matching error of the linear feature sites is calculated. This dynamically adjusts the initial placement parameters until the matching error is less than the error threshold, thus obtaining the optimal placement parameters for the hyperspectral camera. Ultimately, this achieves the best fit between the ground point cloud and the image point cloud in the calibration area, improving the accuracy of the hyperspectral camera's placement parameters. This provides precise placement parameters for the subsequent generation of orthophoto maps, ensuring the accuracy of orthophoto map generation. Attached Figure Description

[0008] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0009] Figure 1 This is a flowchart illustrating a calibration method based on the installation parameters of a UAV hyperspectral camera, as provided in this application.

[0010] Figure 2 A schematic diagram of a hyperspectral image from a hyperspectral camera provided in this application.

[0011] Figure 3 A schematic diagram of the cross-shaped reciprocating flight trajectory provided in this application.

[0012] Figure 4 A schematic diagram of the initial image point cloud provided for this application.

[0013] Figure 5 A schematic diagram of the image point cloud provided in this application.

[0014] Figure 6 A schematic diagram of the ground point cloud for the calibration area provided in this application.

[0015] Figure 7 A schematic diagram of the image point cloud corresponding to the ground point cloud of the calibration area provided in this application.

[0016] Figure 8 This is a schematic diagram of the point cloud map of the superimposed state provided in this application.

[0017] Figure 9 A schematic diagram of the interface for adjusting the mounting parameters of the hyperspectral camera provided in this application.

[0018] Figure 10 A schematic diagram of the point cloud in the superimposed state of the ground point cloud and image point cloud of the calibration area under the optimal placement parameters provided in this application.

[0019] Figure 11 A schematic diagram of the overall orthophoto provided in this application. Detailed Implementation

[0020] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0021] Because each line of hyperspectral image data acquired by a hyperspectral camera contains corresponding position and orientation information, processing each line as a separate image would be extremely complex. Furthermore, the distance between the image plane and the actual point cloud used for calibration is significant, and the variations in features between adjacent lines are substantial, making practical operation quite challenging.

[0022] This application proposes a point cloud representation of hyperspectral images, termed image point cloud. All points in the image point cloud have consistent structure and meaning, facilitating unified management and processing. Taking the exterior orientation elements of one row of images as an example, the viewpoint is placed at the principal point position of this row of images, and the line-of-sight axis direction and rotation angle (i.e., camera attitude matrix) are set according to the attitude angle of this row of images, thus restoring the actual position and attitude of this row of images at the time of capture. The image point cloud and ground point cloud feature information of this row should perfectly match. If there is a deviation, the hyperspectral camera's mounting parameters are adjusted, and the image point cloud is recalculated until the two perfectly match, completing the parameter calibration.

[0023] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0024] In one exemplary embodiment, such as Figure 1 As shown, a method for calibrating the placement parameters of a UAV hyperspectral camera is provided. This method is executed by a computer device, specifically by a terminal or server alone, or by both a terminal and a server. In this embodiment, the method is described using a server as an example, and includes the following steps S1 to S4.

[0025] Step S1: Acquire ground point cloud data of the calibration area, UAV flight altitude, and status data from the hyperspectral camera; status data includes: hyperspectral image (see...). Figure 2 (as shown) and the corresponding camera position and attitude data.

[0026] As an feasible method, obtaining the ground point cloud of the calibration area specifically includes: selecting two linear feature sites that are parallel and perpendicular to the flight trajectory of the UAV; determining calibration data based on the linear feature sites; and generating the ground point cloud based on the calibration data.

[0027] Specifically, the selection requirements for the calibration area are: linear ground features parallel and perpendicular to the flight path (i.e., point features at both ends of similar line images within the strip), with a certain elevation difference between the feature ground features. When determining the calibration data, at least two calibration data points from two round-trip flights at different flight altitudes should be used. The round-trip flight path is as follows: Figure 3 As shown.

[0028] Step S2: Based on flight altitude, camera position, attitude data and initial placement parameters, the hyperspectral image is converted into a point cloud to generate an image point cloud; the initial placement parameters are set values.

[0029] As one feasible approach, step S2 specifically includes steps S21 to S4: Step S21: Calculate the virtual projection coordinates of pixel cloudification based on flight altitude and hyperspectral images.

[0030] As an feasible approach, the formula for calculating the virtual projection coordinates of pixel cloudification is: .

[0031] .

[0032] in, Virtual projection for pixel cloudification The coordinates of the axis; The flight altitude of the drone; The x-coordinate of the hyperspectral image; The camera principal distance of the hyperspectral camera; Virtual projection for pixel cloudification The coordinates of the axis; Virtual projection for pixel cloudification The coordinates of the axis; The angle of the ray corresponding to the hyperspectral image.

[0033] Step S22: Based on camera position and attitude data, determine the coordinates and attitude matrix of the line image in the hyperspectral image.

[0034] Step S23: Calculate the projection center coordinates of the line image based on the coordinates and pose matrix of the line image in the hyperspectral image.

[0035] As an feasible approach, the formula for calculating the coordinates of the projection center of the line image is: .

[0036] in, The projection center of the line image The coordinates of the axis; The projection center of the line image The coordinates of the axis; The projection center of the line image The coordinates of the axis; For hyperspectral images in The coordinates of the axis; For hyperspectral images in line image The coordinates of the axis; For hyperspectral images in The coordinates of the axis; This is the attitude matrix; For hyperspectral cameras in Camera line eccentricity of the axis; For hyperspectral cameras in Camera line eccentricity of the axis; For hyperspectral cameras in Camera line eccentricity of the axis.

[0037] Step S23: Generate the camera eccentricity element attitude matrix based on the initial placement parameters.

[0038] Step S24: Based on the virtual projection coordinates of pixel point cloudification, the projection center coordinates of the line image, the attitude matrix, and the attitude matrix of the camera eccentricity element, the hyperspectral image is point cloudified to generate an image point cloud.

[0039] As an feasible approach, the expression for point cloudification is: .

[0040] in, In image point clouds point The coordinates of the axis; In image point clouds point The coordinates of the axis; In image point clouds point The coordinates of the axis; The projection center of the line image The coordinates of the axis; The projection center of the line image The coordinates of the axis; The projection center of the line image The coordinates of the axis; This is the attitude matrix; For the camera's off-center angle element pose matrix; Virtual projection for pixel cloudification The coordinates of the axis; Virtual projection for pixel cloudification The coordinates of the axis; Virtual projection for pixel cloudification The coordinates of the axis.

[0041] Specifically, the initial image point cloud, such as Figure 4 As shown, the image point cloud after initial attitude correction exhibits significant boundary variations and even layering. After attitude correction of the initial point cloud, an image of the image point cloud from the orthophoto direction is obtained (see...). Figure 5 After attitude correction, the image point cloud feature information has been restored to normal and can be used for parameter calibration.

[0042] Step S3: Calculate the matching error of the linear feature site based on the view of the linear feature site in the ground point cloud and the corresponding view of the linear feature site in the image point cloud.

[0043] Specifically, first, the ground point cloud of the calibrated area (see...) Figure 6 ) and the corresponding image point cloud (see Figure 7 The points are superimposed to obtain the superimposed point cloud map (see...). Figure 8 Then, based on the view of the linear feature site in the ground point cloud of the calibration area and the view of the linear feature site in the corresponding image point cloud, the matching error of the linear feature site is calculated.

[0044] Step S4: Dynamically adjust the initial setup parameters based on the matching error until the matching error is less than the error threshold, thus obtaining the optimal setup parameters for the hyperspectral camera and completing the calibration of the setup parameters.

[0045] Specifically, the error threshold is 1 pixel. The interface for adjusting the hyperspectral camera's mounting parameters is as follows: Figure 9 As shown, based on the direction and magnitude of the deviation between the corresponding features in the image point cloud view and the ground point cloud view, the strongly correlated placement parameters are fine-tuned and recalculated until the matching error of the corresponding features is less than the error threshold, thus obtaining the optimal placement parameters; the superposition state of the ground point cloud and image point cloud under the optimal placement parameters is as follows. Figure 10 As shown.

[0046] As an feasible approach, after obtaining the optimal placement parameters of the hyperspectral camera, step S5 is also included: verifying the optimal placement parameters, specifically including steps S51 to S56: Step S51: Obtain DEM data of the target area and the actual flight altitude of the UAV.

[0047] Step S52: Based on the optimal placement parameters, obtain the actual hyperspectral image of the hyperspectral camera.

[0048] Step S53: Based on DEM data and actual flight altitude, perform hole interpolation on the actual hyperspectral image to obtain an orthophoto.

[0049] Specifically, orthophotos are generated using forward calculation. The process involves calculating the corresponding point in the DEM data for each pixel in the actual hyperspectral image, filling the hyperspectral image, and finally interpolating the holes to form a complete orthophoto.

[0050] The specific process is as follows: 1) Using the actual flight altitude as the initial value, calculate the coordinates of the ground point corresponding to each pixel in the actual hyperspectral image; 2) Find the DEM elevation value of the corresponding ground point coordinates; 3) Calculate the corresponding elevation difference based on the DEM data and DEM elevation values, adjust the initial value, repeat the calculation until the elevation difference is less than the threshold, perform hole interpolation, and form a complete orthophoto.

[0051] Step S54: Extract the junctions of land features and geomorphic features in the orthophoto image to obtain the pixel coordinates of the junctions.

[0052] Step S55: Calculate the pixel matching degree based on the pixel coordinates at the intersection of landform features.

[0053] Step S56: Determine whether the pixel matching degree is less than the pixel matching threshold; if so, verify that the optimal placement parameters meet the accuracy requirements.

[0054] Specifically, in verifying that the optimal placement parameters meet the accuracy requirements, a certain survey area is taken as an example. This survey area has 10 flight paths, a maximum elevation difference of approximately 30 meters, a relative flight altitude of 100 meters, and a DOM resolution of 0.2 meters. Figure 11 The overall orthophoto image is shown. Then, the accuracy of the edges where there are obvious terrain features is analyzed. The results show that there is no significant misalignment between all flight strips, and the edge accuracy is better than 1 pixel. This method can also be used to calibrate the installation parameters of other line scan cameras.

[0055] The beneficial effects of the calibration method based on the installation parameters of a UAV hyperspectral camera proposed in this application are mainly reflected in the following aspects: 1. This application can accurately and in real time recover the exterior orientation elements of hyperspectral images, perform comprehensive feature registration with known calibration platform point clouds, calibrate the hyperspectral camera installation parameters, and use relevant data from the ground point cloud in the calibration area to achieve high-precision determination of the hyperspectral camera installation parameters.

[0056] 2. The optimal mounting parameters were applied to the production of orthophotos based on a hyperspectral camera in an engineering project, verifying the correctness and stability of the parameters. The edge-to-edge accuracy between flight strips was better than one pixel. This method can also be used to calibrate the mounting parameters of other line scan cameras.

[0057] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0058] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0059] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0060] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0061] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A calibration method based on the installation parameters of a UAV hyperspectral camera, characterized in that, The calibration method based on the installation parameters of the UAV hyperspectral camera includes: Acquire ground point cloud data of the calibration area, UAV flight altitude, and status data of the hyperspectral camera; the status data includes: hyperspectral images and corresponding camera position and attitude data; Based on the flight altitude, the camera position, the attitude data, and the initial placement parameters, the hyperspectral image is converted into a point cloud to generate an image point cloud; the initial placement parameters are set values. Based on the view of the linear feature site in the ground point cloud and the corresponding view of the linear feature site in the image point cloud, the matching error of the linear feature site is calculated. The initial placement parameters are dynamically adjusted based on the matching error until the matching error is less than the error threshold, thus obtaining the optimal placement parameters for the hyperspectral camera.

2. The calibration method based on the installation parameters of a UAV hyperspectral camera according to claim 1, characterized in that, The acquisition of the ground point cloud of the calibration area specifically includes: Two types of linear feature sites were selected, one parallel and one perpendicular to the flight path of the UAV; Based on the linear feature site, calibration data is determined; Based on the calibration data, a ground point cloud is generated.

3. The calibration method based on the installation parameters of a UAV hyperspectral camera according to claim 1, characterized in that, Based on the flight altitude, camera position, attitude data, and initial placement parameters, the hyperspectral image is converted into a point cloud to generate an image point cloud, specifically including: Based on the flight altitude and the hyperspectral image, calculate the virtual projection coordinates of pixel cloudification; Based on the camera position and the attitude data, the coordinates and attitude matrix of the line image in the hyperspectral image are determined; Based on the coordinates and pose matrix of the line image in the hyperspectral image, calculate the coordinates of the projection center of the line image; Based on the initial placement parameters, generate a camera eccentricity angle element attitude matrix; Based on the virtual projection coordinates of the pixel point cloudification, the projection center coordinates of the line image, the attitude matrix, and the camera eccentricity element attitude matrix, the hyperspectral image is point cloudified to generate an image point cloud.

4. The calibration method based on the installation parameters of a UAV hyperspectral camera according to claim 3, characterized in that, The formula for calculating the virtual projection coordinates of the pixel cloudification is as follows: ； ； in, Virtual projection for pixel cloudification The coordinates of the axis; The flight altitude of the drone; The x-coordinate of the hyperspectral image; The camera principal distance of the hyperspectral camera; Virtual projection for pixel cloudification The coordinates of the axis; Virtual projection for pixel cloudification The coordinates of the axis; The angle of the ray corresponding to the hyperspectral image.

5. The calibration method based on the installation parameters of a UAV hyperspectral camera according to claim 3, characterized in that, The formula for calculating the projection center coordinates of the line image is: ； in, The projection center of the line image The coordinates of the axis; The projection center of the line image The coordinates of the axis; The projection center of the line image The coordinates of the axis; For hyperspectral images in The coordinates of the axis; For hyperspectral images in line image The coordinates of the axis; For hyperspectral images in The coordinates of the axis; This is the attitude matrix; For hyperspectral cameras in Camera line eccentricity of the axis; For hyperspectral cameras in Camera line eccentricity of the axis; For hyperspectral cameras in Camera line eccentricity of the axis.

6. The calibration method based on the installation parameters of a UAV hyperspectral camera according to claim 3, characterized in that, The expression for point cloudification is: ； in, In image point clouds point The coordinates of the axis; In image point clouds point The coordinates of the axis; In image point clouds point The coordinates of the axis; The projection center of the line image The coordinates of the axis; The projection center of the line image The coordinates of the axis; The projection center of the line image The coordinates of the axis; This is the attitude matrix; For the camera's off-center angle element pose matrix; Virtual projection for pixel cloudification The coordinates of the axis; Virtual projection for pixel cloudification The coordinates of the axis; Virtual projection for pixel cloudification The coordinates of the axis.

7. The calibration method based on the installation parameters of a UAV hyperspectral camera according to claim 1, characterized in that, After obtaining the optimal installation parameters for the hyperspectral camera, the process also includes verifying these optimal installation parameters, specifically including: Acquire DEM data of the target area and the actual flight altitude of the UAV; Based on the optimal placement parameters, the actual hyperspectral image of the hyperspectral camera is obtained; Based on the DEM data and the actual flight altitude, the actual hyperspectral image is subjected to hole interpolation to obtain an orthophoto. The intersections of land features and geomorphic features in the orthophoto are extracted to obtain the pixel coordinates of the intersections. Pixel matching degree is calculated based on the pixel coordinates at the intersection of the aforementioned landform features; Determine whether the pixel matching degree is less than the pixel matching threshold; If so, then verify that the optimal placement parameters meet the accuracy requirements.

Images