Airborne camera homography mapping chain calibration method for planar structure
By utilizing natural feature points and homography mapping chains for calibration, the problem of insufficient adaptability to camera pose changes in UAV remote sensing is solved, achieving efficient and accurate airborne camera calibration that adapts to the flexibility and efficiency of UAV remote sensing.
Patent Information
- Application Number
- CN202511836268.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-08
- Publication Date
- 2026-03-17
AI Technical Summary
Traditional camera calibration methods are difficult to adapt to changes in camera pose in UAV remote sensing, and the frequent deployment of calibration boards contradicts the flexibility and convenience of UAV remote sensing, resulting in a complex and inaccurate calibration process.
A calibration method based on natural feature points and homography mapping chains is adopted. By selecting a visible light or infrared thermal imaging camera, a feature point detection algorithm is used to identify natural markers or calibration boards. Combined with image preprocessing and homography matrix multiplication, adaptive calibration based on camera pose changes is achieved.
It improves the accuracy and efficiency of airborne camera calibration, reduces dependence on calibration boards, adapts to the high efficiency and flexibility of UAV remote sensing, reduces error accumulation, and saves computing resources.
Smart Images
Figure CN121685661A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of aerial remote sensing and UAV data processing technology, specifically relating to a homography mapping chain calibration method for airborne cameras with planar structures. Background Technology
[0002] Camera calibration is an essential step in determining the camera's intrinsic and extrinsic parameters and converting pixel coordinates into physical space coordinates. With the rapid development and application of UAV remote sensing technology, the richness of the image information acquired has been significantly improved. In this process, airborne camera calibration, as a key technical step to ensure the geometric accuracy of images, provides the foundation for the accurate positioning of feature points in aerial images.
[0003] Traditional camera calibration methods often require the camera to be stationary to ensure accuracy and reliability. While active vision calibration methods can utilize the camera's spatial movement, their motion trajectory is singular and requires precise acquisition of specific information such as camera angle and direction of movement, resulting in insufficient applicability in complex scenarios. The pose of UAV-borne cameras changes with waypoints, requiring calibration at each waypoint. However, frequently deploying calibration boards at each waypoint contradicts the flexibility and convenience of UAV remote sensing. Therefore, airborne camera calibration requires an adaptive method that changes camera pose to suit the operability and applicability of UAV remote sensing technology. Summary of the Invention
[0004] To overcome the shortcomings of existing technologies, this invention utilizes natural feature points, homography mapping chains, etc., to form an adaptive camera pose change-oriented calibration method for planar structures, so as to meet the requirements of airborne camera calibration.
[0005] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows: A method for calibrating the homography mapping chain of an airborne camera for planar structures includes the following steps: (1) Based on the illumination characteristics of the target scene, select a visible light camera or an infrared thermal imaging camera; determine the flight altitude by combining existing parameters such as camera sensor pixel size, optical focal length, and ground sampling distance; plan the waypoint spacing and layout according to requirements such as image overlap rate, and design the flight route.
[0006] (2) Test flight in the middle section of the preset route, collect images and analyze whether there are natural markers of known size and obvious features for initial calibration; if no effective natural markers are detected, then set up an artificial calibration board; the natural markers or calibration board are located in the middle section of the route to reduce the cumulative effect of errors and improve the calibration accuracy of the airborne camera.
[0007] (3) Visible light or infrared thermal images are collected sequentially according to the route planning. The collected images are preprocessed such as distortion correction, grayscale conversion and image enhancement. The lens distortion coefficient is determined in advance through indoor calibration test.
[0008] (4) Select the image of the natural marker or calibration board located in the center of the field of view as the calibration frame image. Based on the direction of flight, use the calibration frame image as the reference point to divide all aerial image sequences into the forward acquisition sequence and the rear acquisition sequence.
[0009] (5) For calibration frame images, if natural markers are used, at least four geometrically significant and easily measurable feature points should be selected from the markers as calibration feature points; if a calibration board is used, at least four corner points should be selected as calibration feature points; and the pixel coordinates of the calibration feature points should be obtained. When the number of feature points is insufficient, the SuperPoint feature point recognition algorithm should be used to increase the number of feature points.
[0010] (6) Establish the world coordinate system of the calibration frame image. If it is a natural marker, construct the world coordinate system by measuring its physical size; if it is a calibration plate, establish the world coordinate system based on the known size of the calibration plate. Then the world coordinates of the calibration feature points in step (5) can be determined.
[0011] (7) Combine steps (5) and (6) to calibrate the pixel coordinates and world coordinates of the feature points, and solve the projection transformation matrix between the image coordinate system and the world coordinate system; by substituting the pixel coordinates, world coordinates and projection matrix of the feature points into the camera imaging model, the scaling factor is calculated, and the calibration frame image calibration is completed.
[0012] (8) For the preceding acquisition sequence, select images sequentially, identify corresponding feature points through feature point detection algorithm, and match them with adjacent previous frame images. Based on the matching feature point pairs in the overlapping area of the images, establish pixel coordinate mapping relationship to obtain the homography matrix between adjacent frame images in the preceding sequence.
[0013] (9) For the subsequent acquisition sequence, select images sequentially, identify the corresponding feature points through the feature point detection algorithm, and match them with the adjacent next frame image. Based on the matching feature point pairs in the overlapping area of the images, establish the pixel coordinate mapping relationship to obtain the homography matrix between adjacent frames of the subsequent sequence.
[0014] (10) Using the mapping relationship of pixel coordinates of two adjacent frames obtained in step (8) or (9), construct a homography matrix mapping chain of pixel coordinates of arbitrary frame image and calibration frame image through matrix multiplication; substitute the projection matrix and scaling coefficient of calibration frame image in step (7) into the homography matrix mapping chain, solve the world coordinates of feature points of arbitrary frame image, and complete the calibration of each frame image.
[0015] Furthermore, in step (1), the selection of the airborne camera needs to comprehensively consider the illumination conditions of the observation scene. For high illumination scenes (>10000 Lux), a high-resolution visible light camera is preferred, while in low illumination or nighttime environments (<100 Lux), an infrared thermal imaging camera is selected. Based on the existing camera sensor pixel size, focal length, and ground sampling distance requirements, the UAV flight altitude is calculated and determined. Taking into account parameters such as image overlap and camera gimbal tilt angle, waypoint spacing and layout are planned accordingly, and a reasonable flight path is planned, which can improve the efficiency and accuracy of calibration.
[0016] Furthermore, in step (2), a test flight is conducted at a waypoint in the middle of the preset route to check whether the UAV has incomplete coverage or abnormal flight altitude, and the route and flight parameters are readjusted according to step (1) to ensure the completion of the task. Since the selection of natural markers is closely related to the calibration accuracy, natural markers with actual size that is easy to measure and with unique shape, texture or color should be selected. If there are no effective natural markers, artificial calibration boards can be arranged. Natural markers or calibration boards should be located in the middle area of the route to reduce the calibration path and reduce the accumulation of homography mapping chain errors.
[0017] Further, in step (3), visible light or infrared thermal images are collected according to the preset flight path, and the collected images are preprocessed to improve image quality; firstly, distortion correction is performed on the images collected by the camera. Since the distortion characteristics of the same lens at different focal lengths are negligible, multi-angle calibration board images can be obtained through indoor calibration tests before the flight mission to predetermine lens distortion parameters and perform distortion correction accordingly. Secondly, the aerial images are grayscale processed to reduce computational complexity while retaining key details. Finally, image enhancement technology is used to optimize image quality, and the robustness of the feature point detection algorithm is improved by enhancing local contrast so that the feature point detection algorithm can identify more and more accurate feature points for subsequent selection.
[0018] Furthermore, in step (4), because the angle of incidence of light in the edge region of the image is large, the focus deviates from the central light, which easily leads to image blurring and distortion; the central region is less affected by perspective distortion and has less distortion, so the image of the natural marker or calibration plate located in the center of the field of view should be selected as the calibration frame image. The calibration frame image is used as the calibration reference point, thereby dividing all acquired image sequences into the front acquisition sequence and the rear acquisition sequence.
[0019] Further, in step (5), for the calibration frame image, if natural markers are used, at least four geometrically significant and easily measurable feature points should be selected as calibration feature points; if a calibration board is used, at least four corner points should be selected as calibration feature points; and their pixel coordinates should be obtained. If feature points are scarce, the SuperPoint feature point recognition algorithm can be used: input the grayscale image into the SuperPoint network, and by adjusting the recognition parameters, lowering the confidence threshold or widening the non-maximum suppression (NMS) range, more feature points can be detected; if the number of feature points is still insufficient, the image can be scaled at different scales, the recognition parameters can be readjusted, and finally the feature points at different scales can be merged and duplicate feature points removed. This method can effectively increase the number of feature points while ensuring the quality of the detection results.
[0020] Furthermore, in step (6), a world coordinate system for the calibration frame image is established. For natural landmarks, the actual physical dimensions of the natural landmarks can be obtained using precision equipment such as a total station, and the world coordinate system can be constructed accordingly. For artificial calibration plates, the world coordinate system is established based on the known size parameters of the calibration plate. The world coordinates of the calibration feature points in step (5) can be determined using this method.
[0021] Furthermore, in step (7), the correspondence between the pixel coordinates of the calibration frame image and the world coordinates is as follows: (1) in, This is the proportionality coefficient. To calibrate the pixel coordinates of the frame image, To calibrate the homography matrix of the frame image, For corresponding The world coordinates; combining the pixel coordinates and world coordinates of the feature points calibrated in steps (5) and (6), and combining existing calibration methods (such as Zhang Zhengyou calibration method, etc.), the projection matrix of the calibration frame image is calculated. Then, the projection matrix is applied to the world coordinates of the feature point to obtain its pixel coordinate projection vector. The scaling factor is obtained by using the dot product of the pixel coordinate vector and the projection coordinate vector as the numerator and the dot product of the pixel coordinate vector and itself as the denominator. : (3) Due to the number of calibrated feature points, there will be multiple sets of different pixel coordinates and world coordinates, resulting in different scaling factors in the calculated solution. It is not unique. Therefore, it is necessary to select the most suitable scaling factor from among them. Used for subsequent calibration. When selecting a method, the needs of the actual application and the characteristics of the available data should be considered. If the data sample is large and the value range is concentrated, the average or median method is more effective. However, when the data amplitude varies greatly or outliers exist, the minimum error method or other optimization algorithms, such as the least squares method, should be used, and it may even be necessary to re-acquire images to ensure calibration accuracy. This step completes the conversion between pixel coordinates of the calibration frame image and world coordinates.
[0022] Further, in step (8), for the preceding sequence of images, the feature point detection algorithm is used to identify and match the calibration feature points of the calibration frame image and its adjacent previous frame image. The feature point pairs of the overlapping area of the calibration frame image and its adjacent previous frame image are used to calculate the homography matrix describing the pixel coordinate transformation relationship between the two frames. The calculation result is further optimized by the 3σ criterion and the RANSAC algorithm.
[0023] By calibrating the homography matrix between the calibrated frame image and the adjacent preceding frame image, the pixel coordinate transformation relationship between the two frames can be obtained as follows: (11) in These are the pixel coordinates of adjacent frames in the preceding sequence. To calibrate the homography matrix between the calibrated frame image and the adjacent preceding frame image, repeat this step to obtain the homography matrix between two adjacent frames in the preceding sequence.
[0024] Further, in step (9), for the subsequent sequence images, the feature point detection algorithm is used to identify and match the calibration feature points of the calibration frame image and its adjacent next frame image. The feature point pairs of the overlapping area of the calibration frame image and its adjacent next frame image are used to calculate the homography matrix describing the pixel coordinate transformation relationship between the two frames. The calculation result is further optimized by the 3σ criterion and the RANSAC algorithm.
[0025] By calibrating the homography matrix between the calibrated frame image and the adjacent subsequent frame image, the pixel coordinate transformation relationship between the two frames can be obtained as follows: (12) in These are the pixel coordinates of adjacent frames in the subsequent sequence. To determine the homography matrix between the calibrated frame image and the adjacent subsequent frame image, repeat this step to obtain the homography matrix between two adjacent frames in the subsequent sequence.
[0026] Furthermore, in step (10), using the transformation relationship of pixel coordinates between two adjacent frames in step (8) or (9), a transformation relationship between pixel coordinates of any frame image and pixel coordinates of the calibration frame image is constructed through matrix multiplication, i.e., a homography matrix mapping chain: (9) in, For the preceding sequence number Pixel coordinates of a frame image For the subsequent sequence number The pixel coordinates of the frame image. Since the world coordinates of a target are only related to its objective position in the physical world under the same world coordinate system, and are not related to the mapping relationship of the camera image, the world coordinates of the same stationary target are consistent in each frame image. Substitute the projection matrix and scaling coefficient of the frame image calibrated in step (7) into the homography matrix mapping chain to solve for the world coordinates of feature points of any frame image: (10) in, For the preceding sequence number Pixel coordinates of a frame image For corresponding World coordinates For the subsequent sequence number Pixel coordinates of a frame image For corresponding The world coordinates; by following the steps above, the conversion between pixel coordinates and world coordinates for each frame of the image can be completed.
[0027] Compared with the prior art, the present invention has the following beneficial effects: (1) This invention uses a feature point detection algorithm to identify natural feature points in the initial frame as calibration feature points, which overcomes the problem that pixel points may not coincide with natural markers. It is applicable to the calibration of airborne infrared thermal imaging cameras and airborne visible light cameras, and improves the application range and measurement accuracy.
[0028] (2) The present invention performs initial calibration by selecting a suitable natural marker or manually placing a calibration board at a waypoint in the middle of the route, without having to place a calibration board at each waypoint, thereby reducing the dependence on traditional calibration boards and perfectly matching the efficiency and flexibility of UAV operations; and calibrating from the middle area of the route to both sides reduces the length of the calibration path, which can reduce the accumulation of homography mapping chain errors and make the calibration process more efficient and accurate.
[0029] (3) This invention does not require decomposition of the homography matrix, but directly uses the chain multiplication of the homography matrix to construct the homography mapping chain to realize camera calibration, which effectively saves computing resources and significantly improves computing efficiency. Attached Figure Description
[0030] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation
[0031] The technical solution of the present invention will be further described below with reference to the accompanying drawings.
[0032] A method for calibrating the homography mapping chain of an airborne camera for planar structures includes the following steps: (1) The type of airborne camera, such as a visible light camera or an infrared thermal imaging camera, should be selected according to the illumination conditions of the observation scene and the different shooting tasks. Based on the existing parameters such as the pixel size, focal length and ground sampling distance of the airborne camera sensor, the flight altitude of the UAV is determined; according to the requirements such as the image overlap rate, the waypoint spacing and layout are planned, and a reasonable flight route is planned; during the subsequent flight, the semi-automatic flight mode is switched: the parameters such as the flight route of the UAV and the tilt angle of the camera gimbal are dynamically adjusted, and the target image is captured in real time through oblique photography to ensure the efficiency of calibration and the accuracy of data.
[0033] (2) Conduct on-site test flights at midway points of the preset flight path to check whether the UAV deviates from the flight path, has incomplete coverage, or has abnormal flight altitude. Based on (1), re-optimize the flight path and flight parameters to improve mission efficiency and ensure the safety and reliability of UAV flight. Collect and analyze images through test flights, and select natural markers that are easy to measure in actual size and have unique shapes, textures, or colors. If there are no effective natural markers, calibration plates can be placed manually. The natural markers or manually placed calibration plates selected in this patent should be located in the middle area of the flight path. Use this as the calibration reference point to calibrate to both sides. This can reduce the calibration path, thereby reducing the accumulation of homography mapping chain errors and improving the calibration accuracy of the airborne camera.
[0034] (3) After the images are acquired according to the preset flight path, the aerial images undergo three preprocessing steps: distortion correction, grayscale conversion, and image enhancement. First, to address the common distortion phenomenon in camera lenses, distortion correction measures are used to restore the image distortion caused by physical defects in the lens, thereby eliminating the impact of distortion on calibration accuracy. Since the distortion difference of the same lens at different focal lengths is very small, before the flight mission, an indoor calibration test can be conducted to take images of the calibration board from multiple angles to predetermine the lens distortion coefficient, and distortion correction can be performed accordingly. Second, the aerial images are grayscale converted to reduce the interference of color and lighting changes on key structural textures, thereby reducing the amount of processing computation while preserving the detailed information in the image. Finally, image enhancement technology is used to enhance the contrast between different object features in the image, optimize the visual effect of the image, and highlight small features so that the feature point detection algorithm can identify more and more accurate feature points.
[0035] (4) Because the angle of incidence of light in the edge area of the image is large, the focus deviates from the principal optical axis, which easily causes image blurring and distortion; while the central area of the field of view is less affected by lens distortion and the degree of distortion is lower. Therefore, the image of the natural marker or calibration plate located in the center of the field of view should be selected as the calibration frame image. The calibration frame image is used as the calibration reference point, and all acquired image sequences are divided into front acquisition sequence and rear acquisition sequence.
[0036] (5) For calibration frame images, if natural markers are used, at least four geometrically significant and easily measurable feature points should be selected as calibration feature points; if a calibration board is used, at least four corner points should be selected as calibration feature points, and their pixel coordinates should be obtained. If there are fewer than four feature points, the SuperPoint feature point detection algorithm is used. By optimizing detection parameters and employing a multi-scale strategy, the number of feature points can be significantly increased while maintaining detection quality. The specific method is as follows: First, the grayscale image is input into the SuperPoint network, and its convolutional neural network structure is used to extract initial feature points. Adjusting the detection parameters can increase the number of feature points; lowering the confidence threshold or widening the non-maximum suppression (NMS) range can enhance the response sensitivity in low-texture regions. If the number of feature points still does not meet the requirements, a multi-scale feature enhancement strategy is implemented: the original grayscale image is scaled at different scales (e.g., 0.5x, 1.0x, and 2.0x), and the detection parameters are readjusted to extract feature points at each scale. Subsequently, the feature points at different scales are mapped to the original image coordinate system, all feature points are merged, and duplicate points are removed to ensure the uniqueness of the feature points. This method ensures feature point detection quality and significantly increases the number of feature points.
[0037] (6) Establish the world coordinate system of the calibration frame image. For natural markers, the world coordinate system is constructed by directly measuring the physical dimensions of their key geometric features; for artificial calibration plates, the world coordinate system is established based on the manufacturing specifications and dimensions of the calibration plate. This method can be used to establish the world coordinate system and determine the world coordinates of the calibration feature points in step (5).
[0038] (7) The transformation relationship between pixel coordinates and world coordinates of the calibration frame image is as follows: (1) in, This is the proportionality coefficient. To calibrate the pixel coordinates of the frame image, To calibrate the projection matrix of the frame image, For corresponding World coordinates; Since both pixel coordinates and world coordinates are homogeneous, the scaling factor... Does not affect the projection matrix Therefore, the projection matrix should be solved first. Then calculate the proportionality coefficient. .
[0039] Furthermore, since all feature points on the surface of the observed target lie in the same plane, the XOY plane of the world coordinate system can be established on the plane of the observed target, thus allowing each feature point to be... Therefore, the coordinate transformation equation (1) above can be simplified to: (2) By combining steps (5) and (6) to calibrate the pixel coordinates and world coordinates of the feature points, the projection matrix is calculated. By rearranging terms in relation (2), we obtain the proportionality coefficient. The expression is: (3) The proportionality coefficient can be calculated from this formula. This enables the conversion between pixel coordinates of the calibration frame image and world coordinates, thus completing the calibration of the calibration frame image.
[0040] (8) Using the calibration frame image as the calibration reference point, for the preceding sequence of images, the feature point detection algorithm is used to identify and match the calibration feature points of the calibration frame image and its adjacent previous frame image. Using the feature point pairs of the overlapping area of the calibration frame image and its adjacent previous frame image, the homography matrix describing the pixel coordinate transformation relationship between the two frames is calculated: (4) in These are the pixel coordinates of adjacent frames in the preceding sequence. To determine the homography matrix between the calibrated frame image and the adjacent frame images in the preceding sequence, repeat this step to obtain the homography matrix between two adjacent frames in the preceding sequence.
[0041] (9) Using the calibration frame image as the calibration reference point, for the subsequent sequence of images, the feature point detection algorithm is used to identify and match the calibration feature points of the calibration frame image and its adjacent subsequent frame image. Using the feature point pairs of the overlapping area of the calibration frame image and its adjacent subsequent frame image, the homography matrix describing the pixel coordinate transformation relationship between the two frames is calculated: (5) in These are the pixel coordinates of adjacent frames in the subsequent sequence. To determine the homography matrix between the calibrated frame image and adjacent frames in the subsequent sequence, repeat this step to obtain the homography matrix between two adjacent frames in the subsequent sequence.
[0042] Given that some pairs of feature points between two images may be mismatched, thus affecting the homography matrix... For accuracy issues, the solution results need to be further optimized using the 3σ criterion and the RANSAC algorithm.
[0043] (10) Using the 3σ criterion, we can initially screen for obviously abnormal matching point pairs. For each matching point pair in two frames of images, we can calculate the projection matrix. Residual under transformation ( For example, the formula for calculating the i-th point pair is: (6) (7) in Let i be the residual of the i-th point pair. To calibrate the coordinates of point i in the projection of the frame image onto the adjacent frame images, Let i be the coordinates of point i in the adjacent frame image. The mean of the residual is calculated using these coordinates. ) and standard deviation ( The Z-score of the residuals can be calculated using the mean and standard deviation. The formula is as follows: (8) If the absolute value of the Z-score of the residual of a point pair is greater than 3, the matching point pair is considered abnormal and is removed.
[0044] Next, the RANSAC algorithm is used to randomly select four pairs of feature points from the remaining matching points. The projection matrix of these four pairs of feature points is calculated. Then, all feature point pairs are substituted into the calculated projection matrix to calculate the reprojection error. If the calculated error is less than a set threshold, the feature point pair is considered an interior point; otherwise, it is considered an exterior point. By repeating the above steps, the homography matrix with the most interior points is selected as the final calculation result. During the repeated iterations of the RANSAC algorithm, the 3σ criterion is used to further eliminate outliers, ensuring that the calculated projection matrix is more accurate each time.
[0045] (11) Using the transformation relationship of pixel coordinates between two adjacent frames in step (8) or (9), construct a homography matrix mapping chain that reflects the transformation relationship between pixel coordinates of any frame and pixel coordinates of the calibration frame through matrix multiplication: (9) in, For the preceding sequence number Pixel coordinates of a frame image For the subsequent sequence number The pixel coordinates of the frame image. Since the world coordinates of a target are only related to its objective position in the physical world under the same world coordinate system, and are not related to the mapping relationship of the camera image, the world coordinates of the same stationary target are consistent in each frame image. Substitute the projection matrix and scaling coefficient of the frame image calibrated in step (7) into the homography matrix mapping chain to solve for the world coordinates of feature points of any frame image: (10) in, For the preceding sequence number Pixel coordinates of a frame image For corresponding World coordinates For the subsequent sequence number Pixel coordinates of a frame image For corresponding The world coordinates; by following the above steps, the conversion between the pixel coordinates of each frame of the image and the world coordinates can be completed.
[0046] The above embodiments are merely illustrative of specific implementations of the present invention, and their descriptions are intended to aid in understanding the technical solutions of the present invention, rather than constituting a limitation on the scope of protection of the present invention. Those skilled in the art should understand that any equivalent substitutions, technical improvements, or adaptive modifications based on the principles of the present invention, without departing from the core concept of the present invention, should be considered to fall within the scope of protection of the claims of the present invention.
Claims
1. A method for calibrating a homography mapping chain of an airborne camera facing a planar structure, comprising the following steps: (1) selecting a visible light camera or an infrared thermal imaging camera based on the illumination characteristics of the target scene; determining the flight height by comprehensively considering the existing parameters of the camera sensor pixel size, optical focal length, and ground sampling distance; planning the distance and layout between waypoints according to the image overlap rate requirement, and designing the flight route; (2) conducting a test flight in the middle section of the preset route, collecting images, and analyzing whether there are natural markers with known size and obvious features for initial calibration; if no effective natural markers are detected, artificial calibration boards are arranged; the natural markers or calibration boards are located in the middle section of the route to reduce the cumulative error effect and improve the calibration accuracy of the airborne camera; (3) collecting visible light or infrared thermal images in sequence according to the route path planning, and performing preprocessing such as distortion correction, grayscale, and image enhancement on the collected images, wherein the lens distortion coefficient is determined through indoor calibration test; (4) selecting an image with the natural marker or calibration board located in the center of the field of view as the calibration frame image, and dividing all aerial image sequences into front and rear sequences based on the calibration frame image as the reference point according to the route direction; (5) for the calibration frame image, if the natural marker is used, at least four feature points with significant geometric features and easy to measure are selected as calibration feature points; if the calibration board is used, at least four corner points thereon are selected as calibration feature points; and the pixel coordinates of the calibration feature points are obtained; when the number of feature points is insufficient, the SuperPoint feature point recognition algorithm is used to increase the number of feature points; (6) establishing a world coordinate system for the calibration frame image; if it is a natural marker, the world coordinate system is constructed by measuring its physical size; if it is a calibration board, the world coordinate system is established according to the known size of the calibration board; and then the world coordinates of the calibration feature points in step (5) can be determined; (7) combining the pixel coordinates and world coordinates of the calibration feature points in steps (5) and (6) to solve the projection transformation matrix of the image coordinate system and the world coordinate system; by substituting the pixel coordinates, world coordinates, and projection matrix of the feature points into the camera imaging model, the scale factor is calculated, and the calibration of the calibration frame image is completed; (8) for the front sequence, images are selected in sequence, the corresponding feature points are recognized by the feature point detection algorithm, and matching is performed with the adjacent previous frame image; the pixel coordinate mapping relationship is established based on the matching feature point pairs in the image overlap area, and the homography matrix between the adjacent frame images in the front sequence is obtained; (9) for the rear sequence, images are selected in sequence, the corresponding feature points are recognized by the feature point detection algorithm, and matching is performed with the adjacent next frame image; the pixel coordinate mapping relationship is established based on the matching feature point pairs in the image overlap area, and the homography matrix between the adjacent frame images in the rear sequence is obtained. (10) using the mapping relationship of the pixel coordinates of the two adjacent images obtained in step (8) or (9), a homography matrix mapping chain of the pixel coordinates of any frame image and the calibration frame image is constructed by matrix multiplication; the projection matrix and the scale factor of the calibration frame image in step (7) are substituted into the homography matrix mapping chain to solve the world coordinates of the feature points of the any frame image, and the calibration of each frame image is completed.
2. The plane-structure-oriented on-board camera homography mapping chain calibration method according to claim 1, characterized in that, In step (1), the selection of the airborne camera needs to comprehensively consider the illumination conditions of the observed scene. For high-illumination scenes (>10000Lux), a high-resolution visible light camera is preferred, while for low-illumination or night-time environments (<100Lux), an infrared thermal imaging camera is selected; According to the existing camera sensor pixel size, focal length and ground sampling distance requirements, the UAV flight altitude is calculated and determined, and the image overlap and camera gimbal angle parameters are integrated to plan the flight path and layout, and a reasonable flight route is planned, which can improve the efficiency and accuracy of calibration; In step (2), a test flight is performed at the middle point of the preset route to check whether the UAV has incomplete coverage and abnormal flight height, and the route and flight parameters are adjusted according to step (1) to provide support for completing the task; since the selection of natural markers is closely related to the calibration accuracy, natural markers with actual size, unique shape, texture or color should be selected, and if there are no effective natural markers, artificial calibration boards can be arranged; the natural markers or calibration boards should be located in the middle area of the route, which can reduce the calibration path and reduce the error accumulation of the homography mapping chain.
3. The plane-structure-oriented on-board camera homography mapping chain calibration method of claim 1, wherein, In step (3), visible light or infrared thermal images are collected according to the preset route and the collected images are preprocessed to improve image quality; first, the images collected by the camera are corrected for distortion; since the distortion characteristics of the same lens at different focal lengths can be ignored, indoor calibration tests can be performed before the flight task to obtain multi-angle calibration board images, and the lens distortion parameters can be determined in advance and used for distortion correction; secondly, the aerial images are processed to reduce the computational complexity while preserving key details; finally, image enhancement techniques are used to optimize image quality, and the robustness of the feature point detection algorithm is improved by enhancing local contrast to identify more and more accurate feature points for subsequent selection.
4. The plane-structure-oriented on-board camera homography mapping chain calibration method of claim 1, wherein, In step (4), since the light in the edge area of the image has a large angle of incidence, the focus deviates from the central light, which easily causes image blur and distortion; the central area is less affected by perspective distortion and has less distortion, so the image with the natural marker or calibration board located in the center of the field of view should be selected as the calibration frame image; and the calibration frame image is used as the calibration reference point, and all collected image sequences are divided into front and rear collection sequences.
5. The plane-structure-oriented on-board camera homography mapping chain calibration method of claim 1, wherein, In step (5), for the calibration frame image, if a natural marker is used, at least four feature points with significant geometric features and easy to measure are selected as calibration feature points; if a calibration board is used, at least four corner points are selected as calibration feature points; and obtain their pixel coordinates; If the feature points are insufficient, the SuperPoint feature point recognition algorithm is used: the gray image is input into the SuperPoint network, and by adjusting the recognition parameters, reducing the confidence threshold or relaxing the Non-Maximum Suppression (NMS) range, more feature points can be detected; If the number of feature points is still insufficient, the image can be scaled at different scales, the recognition parameters are adjusted again, and finally the feature points of different scales are merged and the repeated feature points are removed; through this method, the number of feature points can be effectively increased, while ensuring the quality of the detection result.
6. The plane-structure-oriented on-board camera homography mapping chain calibration method of claim 1, wherein, In step (6), the world coordinate system of the calibration frame image is established, for natural markers, the actual physical size of the natural markers can be obtained by using precise equipment such as total station, and the world coordinate system is constructed according to the known size parameters of the calibration board; through this method, the world coordinates of the calibration feature points in step (5) can be determined.
7. The plane-structure-oriented on-board camera homography mapping chain calibration method of claim 1, wherein, In step (7), the corresponding relationship between the pixel coordinates of the calibration frame image and the world coordinates is as follows: （1） wherein, is a proportionality coefficient, is a pixel coordinate of the calibration frame image, is a homography matrix of the calibration frame image, is a world coordinate corresponding to The pixel coordinate and the world coordinate of the calibration feature point are calibrated in the comprehensive step (5) and the step (6), and the projection matrix of the calibration frame image is obtained by combining the existing calibration method (such as Zhang Zhengyou calibration method, etc.) After that, the projection coordinate vector of the pixel coordinate of the feature point is obtained by using the projection matrix on the world coordinate of the feature point, and the dot product of the pixel coordinate vector and the projection coordinate vector is taken as the numerator, and the dot product of the pixel coordinate vector and itself is taken as the denominator, so that the proportionality coefficient is obtained. （3） The number of characteristic points will affect the number of different pixel coordinates and world coordinates, resulting in different scale factors Not unique; therefore, the most appropriate scale factor needs to be selected from them For subsequent calibration; when selecting, the actual application requirements and the characteristics of the available data should be considered; if the data sample is large and the value range is concentrated, the average or median method is more effective, while when the data amplitude changes greatly or there are outliers, the minimum error method or other optimization algorithms such as least squares method should be used, and even the image needs to be re-collected to ensure the accuracy of the calibration; through this step, the conversion of the pixel coordinates and world coordinates of the calibration frame image can be completed.
8. The plane-structure-oriented on-board camera homography mapping chain calibration method of claim 1, wherein, In step (8), for the front sequence image, the calibration feature points of the calibration frame image and its adjacent previous frame image are recognized and matched using the feature point detection algorithm, and the homography matrix describing the pixel coordinate conversion relationship between the two images is obtained by using the feature point pairs in the overlapping area of the calibration frame image and its adjacent previous frame image, and the 3σ criterion and RANSAC algorithm are used to further optimize the calculation result; The homography matrix between the calibration frame image and the adjacent previous frame image can obtain the pixel coordinate conversion relationship between the two images as follows: (11) wherein is the pixel coordinate of the previous frame image, is the homography matrix between the current frame image and the previous frame image; repeat the step to obtain the homography matrix between the two previous frame images.
9. The plane-structure-oriented on-board camera homography mapping chain calibration method of claim 1, wherein, In step (9), for the rear sequence image, the calibration feature points of the calibration frame image and its adjacent next frame image are recognized and matched using the feature point detection algorithm, and the homography matrix describing the pixel coordinate conversion relationship between the two images is obtained by using the feature point pairs in the overlapping area of the calibration frame image and its adjacent next frame image, and the 3σ criterion and RANSAC algorithm are used to further optimize the calculation result; The homography matrix between the calibration frame image and the adjacent next frame image can obtain the pixel coordinate conversion relationship between the two images as follows: (12) wherein is the pixel coordinate of the rear sequence adjacent frame image, is the homography matrix between the calibration frame image and the adjacent next frame image; repeat the step to obtain the homography matrix between the two adjacent frame images of the rear sequence.
10. The plane-structure-oriented on-board camera homography mapping chain calibration method of claim 1, wherein, In step (10), the conversion relationship of the pixel coordinates of the adjacent two frames of images in steps (8) or (9) is used to construct the conversion relationship of the pixel coordinates of any frame image and the pixel coordinates of the calibration frame image through matrix multiplication, that is, the homography matrix mapping chain: (9) wherein, is the first frame image pixel coordinate of the front sequence is the first frame image pixel coordinate of the back sequence is the first frame image pixel coordinate of the back sequence is the first frame image pixel coordinate of the back sequence; since in the same world coordinate system, the world coordinate of the target is only related to its objective position in the physical world, and is irrelevant to the mapping relationship of the camera image; therefore, the world coordinate of the same stationary target in each frame image is consistent; the projection matrix and the scale factor of the calibration frame image in step (7) are substituted into the homography matrix mapping chain to solve the world coordinate of the feature point of any frame image: (10) wherein, is the pixel coordinate of the first frame image of the front sequence, is the pixel coordinate of the first frame image of the back sequence, is the world coordinate corresponding to the pixel coordinate of the first frame image of the front sequence, is the world coordinate corresponding to the pixel coordinate of the first frame image of the back sequence, is the pixel coordinate of the last frame image of the front sequence, is the pixel coordinate of the last frame image of the back sequence, is the world coordinate corresponding to the pixel coordinate of the last frame image of the front sequence, is the world coordinate corresponding to the pixel coordinate of the last frame image of the back sequence; the conversion between the pixel coordinate and the world coordinate of each frame image is completed through the above steps.