A method for comparing the utilization rate of drone flight path images

Abstract

A method for comparing the utilization rate of UAV flight path images belongs to the field of remote sensing data processing technology. The method involves: labeling images acquired by the UAV to select target areas; classifying images based on these target areas; after filtering, images are divided into three categories: irrelevant images without target information, valid images containing target information, and invalid images where target information is occluded; defining image utilization evaluation indicators, including total utilization (TU), effective utilization (EU), and occlusion rate (OR); TU represents the image utilization rate without considering occlusion, and EU represents the image utilization rate considering occlusion; and calculating the image utilization rate. This invention can effectively quantify the utilization rate of images captured by UAVs, providing technical support for further flight path planning, improving modeling accuracy, and reducing operational costs.

Description

Technical Field

[0001] This invention belongs to the field of remote sensing data processing technology, specifically relating to a method for comparing the utilization rate of images from different UAV flight paths. Background Technology

[0002] With the continuous increase in the global population, the demand for food is also growing year by year. How to utilize modern technology to improve crop yield and quality and ensure food supply has become an urgent problem to be solved. In recent years, the widespread application of drone platforms has provided new solutions for the modernization of agricultural production. By analyzing and processing remote sensing images acquired by drones, agricultural producers can perform operations such as precision fertilization, precision irrigation, precision pesticide application, and crop growth monitoring. It is worth noting that in the actual application of drones, different flight path settings determine the image accuracy, number of images, and operating costs, but only a portion of the images can provide effective information about the target area. However, at present, an efficient evaluation method for drone image utilization rate (i.e., the proportion of images containing effective information about the target area to the total number of images) has not yet been found. Summary of the Invention

[0003] To address the lack of methods for evaluating the utilization rate of UAV remote sensing images, this invention proposes a method for comparing the utilization rate of UAV flight route images, aiming to provide a scientific basis for the selection of different flight routes.

[0004] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0005] A method for comparing the utilization rate of UAV flight path images, the method being:

[0006] Step 1: The UAV acquires image tags. The purpose of image tagging is to select target areas and classify images based on these target areas. After filtering, the images are divided into three categories: irrelevant images without target information, valid images containing target information, and invalid images where the target information is occluded.

[0007] Step 2: Define image utilization evaluation metrics, including total utilization (TU), effective utilization (EU), and occlusion rate (OR). Total utilization (TU) represents image utilization without considering occlusion, while effective utilization (EU) represents image utilization considering occlusion. The specific formulas for these three metrics are as follows:

[0008]

[0009]

[0010]

[0011] Where n is the total number of test cells, and N is the total number of captured images. Total It is the sum of valid and invalid images in a test cell; N Valid N is the number of valid images in a test cell; InValid N / n represents the number of invalid images in a test cell; N / n represents the number of images allocated to a single test cell.

[0012] Step 3: Calculate image utilization, specifically:

[0013] (1) Obtain the three-dimensional coordinates of the four vertices of the effective region of the target image;

[0014] (2) The image set (including valid images and invalid images) corresponding to the effective information area is calculated according to the backtracking algorithm of the three-dimensional point cloud reconstruction of the cell; the four vertices correspond to four image sets respectively; the intersection of the four image sets is the backtracking image set of the effective information area;

[0015] (3) Distinguish between valid and invalid images, calculate the minimum bounding rectangle of the four backtracking points on the image; rotate the boundary rectangle to the horizontal direction, and calculate the ratio of white pixels to the total pixels of the rectangle; if the proportion of pixels in the valid information area is greater than or equal to the threshold D, the image is considered valid; otherwise, it is considered invalid.

[0016] (4) Calculate the image utilization rate using the formula in step two.

[0017] Further, in step three (2), the purpose of the backtracking algorithm is to backtrack the reconstructed 3D point P to multiple 2D images containing the information of that point; the 3D point P derived from the 3D point cloud is located in the UTM 49N coordinate system, and the point P is converted from the UTM 49N coordinate system to the geocentric coordinate system. According to the scaling factor, rotation matrix, and translation matrix, the point P is further converted to the internal coordinate system; after 3D reconstruction, the pose (camera intrinsic and extrinsic parameters) and distortion coefficients of each image are obtained from Metashape. The camera extrinsic parameter matrix is ​​used to transform the point P into the camera coordinate system of each image. The specific calculation formula is as follows:

[0018]

[0019] Where Xc is the x-coordinate in the camera coordinate system; Yc is the y-coordinate in the camera coordinate system; Zc is the y-coordinate in the camera coordinate system; R is a 3x3 rotation matrix; T is a 3x1 translation matrix; Xw is the x-coordinate in the internal coordinate system; Yw is the y-coordinate in the internal coordinate system; and Zw is the y-coordinate in the internal coordinate system.

[0020] After z-coordinate normalization, the coordinates of point P are corrected according to the tangential and radial distortion coefficients, and the specific calculation formula is as follows:

[0021]

[0022]

[0023]

[0024] Where X' is the distortion-corrected x-coordinate, X is the normalized camera x-coordinate, k1 is the first-order radial distortion coefficient, k2 is the second-order radial distortion coefficient, k3 is the third-order radial distortion coefficient, and r 2 =X 2 +Y 2 p1 is the first-order tangential distortion coefficient, p2 is the second-order tangential distortion coefficient, Y is the normalized camera ordinate, the distance to the camera imaging plane is 1, Y' is the y-coordinate after distortion correction, u is the pixel x-coordinate, v is the pixel y-coordinate, the focal length f is the distance from the camera focus to the imaging plane, dx represents the actual size of the pixel in the x-direction, dy represents the actual size of the pixel in the y-direction, cx is the x-position of the optical center, and cy is the y-position of the optical center.

[0025] Finally, the pixel coordinates of point P are obtained using the camera's intrinsic parameters. If the pixel coordinates of point P are within the range of the original image, then the image is considered a backtracking image of point P. After traversing all images, the total number of images satisfying the backtracking condition is N. Total .

[0026] Furthermore, in step three (3), since valid images with a D value less than 0.5 contain less information, in order to be more objective, the D value is set to increase from 0.5 to 0.9, with an increment of 0.1.

[0027] The advantages of this invention compared to existing technologies are as follows: This invention proposes a method for comparing the utilization rate of UAV flight path images, using a cross-circling flight path and a traditional five-directional flight path as comparisons. The method proposed in this invention calculates the image utilization rate. The average total utilization (TU) and effective utilization (EU) of the cross-circling flight path are 20% and 124% higher, respectively, than those of the five-directional flight path, while the average obstruction rate (OR) of the cross-circling flight path is 11.7% lower. This is because the cross-circling flight path obtains effective images from more observation angles of the target blade. In summary, this invention can effectively quantify the utilization rate of images captured by UAVs, providing technical support for further flight path planning, improving modeling accuracy, and reducing operating costs. Attached Figure Description

[0028] Figure 1 During the preprocessing of UAV images, marked leaf point clouds are selected as the target area map;

[0029] Figure 2This is a classification image of three types of leaf point clouds during the preprocessing of UAV images;

[0030] Figure 3 This is a flowchart for calculating the utilization rate of images obtained by drones;

[0031] Figure 4 This is a schematic diagram of the image backtracking process. Detailed Implementation

[0032] To make the technical solution and advantages of this invention more intuitive and clear, the following specific implementation examples further illustrate the application of this invention. It should be noted that the implementation examples used herein are only for better explanation of this invention and do not limit the invention.

[0033] Example 1:

[0034] Step 1: The UAV acquires image tags. The purpose of these image tags is to identify selected target regions. Further image classification is then performed based on these target regions. See details below. Figure 1 , Figure 2 After screening, the images were divided into three categories: irrelevant images that do not contain target information, valid images that contain target information, and invalid images where the target information is occluded.

[0035] Step 2: Define image utilization evaluation metrics, including Total Utilization (TU), Effective Utilization (EU), and Occlusion Rate (OR). Total Utilization (TU) represents image utilization without considering occlusion, and Effective Utilization (EU) represents image utilization considering occlusion. The specific formulas for the three evaluation metrics are as follows:

[0036]

[0037]

[0038]

[0039] Where n is the total number of test cells, and N is the total number of captured images. Total It is the sum of valid and invalid images in a test cell; N Valid N is the number of valid images in a test cell; InValid N / n represents the number of invalid images in a test cell; N / n represents the number of images allocated to a single test cell.

[0040] Step 3: Calculate image utilization rate. The algorithm for calculating image utilization rate is as follows: Figure 3 As shown; the algorithm comprises four parts:

[0041] (1) Obtain the three-dimensional coordinates of the four vertices of the effective region of the target image; the coordinate system involved is the same as that of the reconstructed three-dimensional point cloud;

[0042] (2) The image set (including valid images and invalid images) corresponding to the effective information region is calculated according to the backtracking algorithm; the four vertices correspond to four image sets respectively; the intersection of the four image sets is the backtracking image set of the effective information region;

[0043] (3) Distinguish between valid and invalid images, and calculate the minimum bounding rectangle of the four backtracking points on the image. Rotate the boundary rectangle to the horizontal direction and calculate the ratio of white pixels to the total pixels of the rectangle; if the proportion of pixels in the valid information area is greater than or equal to the threshold D, the image is considered valid; otherwise, it is considered invalid. Because valid images with a D value less than 0.5 contain less information, in order to be more objective, the D value is set to increase from 0.5 to 0.9, with an increment of 0.1.

[0044] (4) Calculation of image utilization rate;

[0045] Step 4: The purpose of the backtracking algorithm proposed in Step 3 is to backtrack the reconstructed 3D point P to multiple 2D images containing information about that point. See [link / reference] for details. Figure 4 In the aforementioned implementation example, the derived 3D point P is located in the UTM 49N coordinate system. Point P is then converted from the UTM 49N coordinate system to the geocentric coordinate system, and further converted to the internal coordinate system based on the scaling factor, rotation matrix, and translation matrix. After further 3D reconstruction, the pose (camera intrinsic and extrinsic parameters) and distortion coefficients of each image can be obtained from Metashape. The camera extrinsic parameter matrix is ​​used to transform point P into the camera coordinate system of each image. The specific calculation formula is as follows:

[0046]

[0047] Where Xc is the x-coordinate in the camera coordinate system; Yc is the y-coordinate in the camera coordinate system; Zc is the y-coordinate in the camera coordinate system; R is a 3x3 rotation matrix; T is a 3x1 translation matrix; Xw is the x-coordinate in the internal coordinate system; Yw is the y-coordinate in the internal coordinate system; and Zw is the y-coordinate in the internal coordinate system.

[0048] After z-coordinate normalization, the coordinates of point P are corrected according to the tangential and radial distortion coefficients, and the specific calculation formula is as follows:

[0049]

[0050]

[0051]

[0052] Where X' is the distortion-corrected x-coordinate, X is the normalized camera x-coordinate, k1 is the first-order radial distortion coefficient, k2 is the second-order radial distortion coefficient, k3 is the third-order radial distortion coefficient, and r 2 =X 2 +Y 2 p1 is the first-order tangential distortion coefficient, p2 is the second-order tangential distortion coefficient, Y is the normalized camera ordinate, the distance to the camera imaging plane is 1, Y' is the y-coordinate after distortion correction, u is the pixel x-coordinate, v is the pixel y-coordinate, the focal length f is the distance from the camera focus to the imaging plane, dx represents the actual size of the pixel in the x-direction, dy represents the actual size of the pixel in the y-direction, cx is the x-position of the optical center, and cy is the y-position of the optical center.

[0053] Finally, the pixel coordinates of point P are obtained using the camera's intrinsic parameters. If the pixel coordinates of point P are within the range of the original image, then the image is considered a backtracking image of point P. After traversing all images, the total number of images satisfying the backtracking condition is N. Total .

[0054] It should be understood that the above is only one of the implementation examples of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for comparing the utilization rate of UAV flight path images, characterized in that: The method is as follows: Step 1: The UAV acquires image tags. The purpose of image tagging is to select target areas and classify images based on target areas. After filtering, the images are divided into three categories: irrelevant images without target information, valid images containing target information, and invalid images where the target information is occluded. Step 2: Define image utilization evaluation metrics, including total utilization (TU), effective utilization (EU), and occlusion rate (OR). Total utilization (TU) represents image utilization without considering occlusion, while effective utilization (EU) represents image utilization considering occlusion. The specific formulas for these three metrics are as follows: Where n is the total number of test cells, and N is the total number of captured images. It is the sum of valid and invalid images in a test cell; It is the number of valid images in a test cell; N / n represents the number of invalid images in a test cell; N / n represents the number of images allocated to a single test cell. Step 3: Calculate image utilization rate, specifically: (1) Obtain the three-dimensional coordinates of the four vertices of the effective region of the target image; (2) The image set corresponding to the effective information area is calculated according to the backtracking algorithm of the three-dimensional point cloud reconstruction of the community, including effective images and invalid images; the four vertices correspond to four image sets respectively; the intersection of the four image sets is the backtracking image set of the effective information area; (3) Distinguish between valid and invalid images, calculate the minimum bounding rectangle of the four backtracking points on the image; rotate the boundary rectangle to the horizontal direction, and calculate the ratio of white pixels to the total pixels of the rectangle; if the proportion of pixels in the valid information area is greater than or equal to the threshold D, the image is considered valid; otherwise, it is considered invalid. (4) Calculate the image utilization rate using the formula in step two.

2. The method for comparing the utilization rate of UAV flight path images according to claim 1, characterized in that: In step 3(2), the 3D point P derived from the 3D point cloud is located in the UTM 49N coordinate system. Point P is converted from the UTM 49N coordinate system to the geocentric coordinate system. Based on the scaling factor, rotation matrix, and translation matrix, point P is further converted to the internal coordinate system. After 3D reconstruction, the camera intrinsic and extrinsic parameters and distortion coefficients of each image are obtained from Metashape. The camera extrinsic parameter matrix is ​​used to transform point P into the camera coordinate system of each image. The specific calculation formula is as follows: in, It is the x-coordinate in the camera coordinate system; It is the ordinate in the camera coordinate system; R is the vertical coordinate in the camera coordinate system; R is a 3×3 rotation matrix, and T is a 3×1 translation matrix. It is the x-coordinate in the internal coordinate system. It is the ordinate in the internal coordinate system. It is the vertical coordinate in the internal coordinate system; After z-coordinate normalization, the coordinates of point P are corrected according to the tangential and radial distortion coefficients, and the specific calculation formula is as follows: in, This is the distortion-corrected x-coordinate, where X is the normalized camera x-coordinate. It is the first-order radial distortion coefficient. It is the second-order radial distortion coefficient. It is the third-order radial distortion coefficient. , It is the first-order tangential distortion coefficient. These are the second-order tangential distortion coefficients, where Y is the normalized camera ordinate, and the distance to the camera imaging plane is 1. y is the y-coordinate after distortion correction, u is the pixel x-coordinate, v is the pixel y-coordinate, f is the distance from the camera focus to the imaging plane, dx represents the actual size of the pixel in the x-direction, dy represents the actual size of the pixel in the y-direction, cx is the x-position of the optical center, and cy is the y-position of the optical center. Finally, the pixel coordinates of point P are obtained using the camera's intrinsic parameters. If the pixel coordinates of point P are within the range of the original image, then the image is considered a backtracking image of point P. After traversing all images, the total number of images satisfying the backtracking condition is... .

3. The method for comparing the utilization rate of UAV flight path images according to claim 1, characterized in that: In step three (3), the D value is set to increase from 0.5 to 0.9, with an increment of 0.1.

Images