Automatic image labeling method based on unmanned aerial vehicle GPS and tower coordinates

Abstract

The invention relates to an automatic image labeling method based on an unmanned aerial vehicle GPS and tower coordinates. The method comprises the steps of S1, data preprocessing; s2, data matching; s3, a double-tower competition judgment mechanism; s4, image feature auxiliary verification; and S5, self-adaptive annotation output is carried out. According to the invention, through a mode of combining spatial distance calculation and image feature recognition, efficient and accurate labeling of the power tower in the unmanned aerial vehicle inspection image is realized, and scene adaptability and labeling reliability are improved.

Description

Technical Field

[0001] This invention belongs to the field of UAV inspection and image recognition technology, and in particular relates to an automatic image annotation method based on UAV GPS and tower coordinates. Background Technology

[0002] In the field of power facility inspection, traditional UAV image annotation methods have long relied on manual association of tower numbers, resulting in low efficiency and the risk of human error. While existing GPS coordinate-based automatic matching schemes can achieve basic positioning, their reliance on spatial coordinate matching makes them susceptible to UAV positioning drift, leading to a high error rate in tower identification. Furthermore, these methods generally ignore the impact of environmental parameters such as flight altitude and shooting angle on the tower's visible range, making them unsuitable for complex inspection scenarios. For scenarios with multiple towers side-by-side, existing algorithms lack effective competitive discrimination mechanisms, failing to accurately distinguish the spatial relationships between adjacent towers. It is worth noting that even when the target tower is not captured in the image, traditional automatic identification systems may still mislabel it, causing data recording distortion.

[0003] Although deep learning technology has shown significant advantages in the field of image feature extraction, there are still technical challenges in effectively integrating it with spatial positioning data, especially in dynamic environment adaptability and multimodal data collaborative discrimination. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of the prior art and provide an automatic image annotation method based on UAV GPS and tower coordinates, so as to achieve accurate, efficient and adaptive annotation of UAV inspection images.

[0005] The technical problem solved by this invention is achieved through the following technical solution: An automatic image annotation method based on UAV GPS and pole coordinates is proposed. This method aims to achieve efficient and accurate annotation of power poles in UAV inspection images by combining spatial distance calculation with image feature recognition. The steps of the method are as follows: S1. Data Preprocessing: Acquire and parse detailed parameters of the images taken by the UAV, extract the timestamp of the image capture time and GPS positioning information (including latitude, longitude, altitude and heading angle); at the same time, load the geographic information file KML, extract the number, type and spatial coordinate information of each power line tower, and construct a tower coordinate database. S2. Data Matching: The capture time of each image is matched with the corresponding drone flight trajectory point to obtain the precise drone position information at that moment. Then, the matched GPS coordinates in the WGS-84 coordinate system are converted into three-dimensional coordinates in the geocentric rectangular coordinate system (ECEF). Based on the converted ECEF coordinate system, the three-dimensional spatial distance between the current drone position and all poles in the database is calculated to obtain the spatial relative position relationship between each pole and the drone, which serves as a preliminary judgment basis. S3. Dual-tower competition discrimination mechanism: Select the two candidate towers closest to the drone's current position from all towers, and execute the following discrimination logic: a) If the distance between the nearest pole and the drone is less than or equal to the dynamically calculated distance threshold Dt, then the nearest pole is marked in the image. b) If the distance between both candidate towers and the drone is greater than Dt, it is determined that there are no valid towers visible in the current image and no annotation is made; c) If the distance ratio between two candidate towers is greater than the set ratio threshold (set according to the specific span between towers), then the tower with the closer distance is selected as the target for marking; d) If none of the above conditions are met, proceed to the next step; S4. Image Feature-Assisted Verification: When the target tower cannot be clearly determined based on spatial distance in S3, an image recognition algorithm is introduced to further analyze the candidate region: First, the approximate location of the tower in the image is estimated based on the spatial calculation results. Then, the YOLOv5 target detection model is used to identify the tower body and its nameplate information, and the recognition confidence score is output. If the recognition confidence score is greater than or equal to a preset threshold (this threshold is set according to the model training situation), the tower is confirmed as the target object and labeled; otherwise, it is determined that there is no valid tower in the image and no labeling is performed. S5. Adaptive Annotation Output: Generates an enhanced image with annotation information based on the discrimination results, specifically including: a) Embed information such as tower number, spatial coordinates, and discrimination confidence level into the image file in XMP metadata format to facilitate later retrieval and management; b) Add a semi-transparent watermark containing key information such as tower numbers to a preset area of ​​the image; c) Automatically select the appropriate annotation mode based on the image content, including single tower annotation, multi-tower annotation, or no annotation, to improve the applicability and reliability of the system in different inspection scenarios.

[0006] The positive effects that this invention can produce are: 1. This invention proposes a composite discrimination system of "dual threshold spatial correlation + image feature-assisted verification". By combining UAV PS positioning information and tower coordinate data in geographic information files to construct a three-dimensional spatial distance model, dynamic distance threshold and dual tower competition mechanism are introduced, which effectively solves the mislabeling problem caused by GPS drift in traditional methods and significantly improves the accuracy of image annotation.

[0007] 2. This invention introduces an image feature recognition algorithm as an auxiliary verification method to correct and confirm spatial calculation results in complex scenarios (such as multiple towers in parallel, long-distance shooting, non-frontal angles, etc.), thereby enhancing the system's adaptability to the actual inspection environment, avoiding the occurrence of missed or incorrect marking, and improving the level of intelligence in image processing. Attached Figure Description

[0008] Figure 1 This is a flowchart of the system architecture of the present invention; Figure 2 This is a diagram of the dual-tower competition discrimination system of the present invention; Figure 3 This is a flowchart illustrating the image feature-assisted verification process of the present invention. Detailed Implementation

[0009] The present invention will be further described in detail below through specific embodiments. The following embodiments are merely descriptive and not limiting, and should not be used to limit the scope of protection of the present invention.

[0010] like Figures 1-3 As shown, an automatic image annotation method based on UAV GPS and pole coordinates is innovative in that: the method aims to achieve efficient and accurate annotation of power poles in UAV inspection images by combining spatial distance calculation with image feature recognition. The steps of the method are as follows: S1. Data Preprocessing: Acquire and parse detailed parameters of the images taken by the UAV, extract the timestamp of the image capture time and GPS positioning information (including latitude, longitude, altitude and heading angle); at the same time, load the geographic information file KML, extract the number, type and spatial coordinate information of each power line tower, and construct a tower coordinate database. S2. Data Matching: The capture time of each image is matched with the corresponding drone flight trajectory point to obtain the precise drone position information at that moment. Then, the matched GPS coordinates in the WGS-84 coordinate system are converted into three-dimensional coordinates in the geocentric rectangular coordinate system (ECEF). Based on the converted ECEF coordinate system, the three-dimensional spatial distance between the current drone position and all poles in the database is calculated to obtain the spatial relative position relationship between each pole and the drone, which serves as a preliminary judgment basis. S3. Dual-tower competition discrimination mechanism: Select the two candidate towers closest to the drone's current position from all towers, and execute the following discrimination logic: a) If the distance between the nearest pole and the drone is less than or equal to the dynamically calculated distance threshold Dt, then the nearest pole is marked in the image. b) If the distance between both candidate towers and the drone is greater than Dt, it is determined that there are no valid towers visible in the current image and no annotation is made; c) If the distance ratio between two candidate towers is greater than the set ratio threshold (set according to the specific span between towers), then the tower with the closer distance is selected as the target for marking; d) If none of the above conditions are met, proceed to the next step; S4. Image Feature-Assisted Verification: When the target tower cannot be clearly determined based on spatial distance in S3, an image recognition algorithm is introduced to further analyze the candidate region: First, the approximate location of the tower in the image is estimated based on the spatial calculation results. Then, the YOLOv5 target detection model is used to identify the tower body and its nameplate information, and the recognition confidence score is output. If the recognition confidence score is greater than or equal to a preset threshold (this threshold is set according to the model training situation), the tower is confirmed as the target object and labeled; otherwise, it is determined that there is no valid tower in the image and no labeling is performed. S5. Adaptive Annotation Output: Generates an enhanced image with annotation information based on the discrimination results, specifically including: a) Embed information such as tower number, spatial coordinates, and discrimination confidence level into the image file in XMP metadata format to facilitate later retrieval and management; b) Add a semi-transparent watermark containing key information such as tower numbers to a preset area of ​​the image; c) Automatically select the appropriate annotation mode based on the image content, including single tower annotation, multi-tower annotation, or no annotation, to improve the applicability and reliability of the system in different inspection scenarios.

[0011] The specific steps of this invention are as follows: S1: After system startup, the input data is preprocessed to ensure the basic accuracy of subsequent calculations. By parsing the POS data from the UAV flight log, GPS positioning information (including latitude, longitude, altitude, and heading angle) is extracted from each inspection image, and the precise shooting timestamp is obtained from the image EXIF ​​metadata. Subsequently, a geographic information file (KML) is loaded, and the power pole coordinate data contained within is parsed to construct a database containing pole numbers, types, and spatial coordinates (as shown in Table 1), providing basic data support for subsequent spatial matching.

[0012] Table 1 Example of Tower Coordinate Database S2: After data preprocessing, the system establishes a ±3-second time window tolerance to match the image capture time with the timestamps of the UAV flight trajectory points, ensuring the real-time spatial location. Once the match is successful, the UAV's WGS-84 coordinate system data is converted to three-dimensional coordinates in the geocentric rectangular coordinate system (ECEF), laying the foundation for subsequent spatial distance calculations.

[0013] S3: Perform spatial distance calculation: Based on the ECEF coordinate system, the system calculates the three-dimensional Euclidean distance between the current position of the UAV and all poles, and generates a candidate pole list in ascending order of distance (as shown in Table 3). For example, assuming the current position coordinates of the UAV are (xu, yu, zu) and the coordinates of a certain pole are (xt, yt, zt), then the distance calculation formula is: d= ; The calculation results will serve as a key input for subsequent dual-tower competition determination.

[0014] S4: Dual-Tower Competition Detection The system selects the two nearest poles to the drone from the candidate pole list as candidate targets and executes dynamic threshold discrimination logic. The dynamic distance threshold (Dt) is calculated based on the drone's flight altitude, camera focal length, and safety factor (Dt = k·H·tan(θ / 2)). If the distance between the nearest poles is less than Dt, the pole is directly labeled; if the distance between both candidate poles exceeds the threshold, no poles are visible in the image; if the ratio of the distances between the two poles exceeds 3:1, the closer pole is labeled first; if none of the above conditions are met, the system proceeds to the image feature-assisted verification stage (as shown in Table 2 and...). Figure 2 (The discrimination rule shown).

[0015] Table 2. Rules for Judging Competition Between Two Towers S5: For cases where identification is not possible in S4, image feature-assisted verification is performed. For scenarios where spatial calculations cannot definitively determine the location, the system estimates the approximate area of ​​the candidate tower in the image based on its coordinates. Then, a Haar feature detector identifies typical structural features of the tower (such as beams and the tower body), and the confidence level of the tower body and nameplate is analyzed using the YOLOv5 target detection model. If the confidence level of the detected tower features is ≥0.7, the target tower is confirmed as valid; otherwise, it is determined to be a scenario with no visible tower (as shown in Table 3 for the comparison of detection results).

[0016] Table 3 Comparison of Image Feature Detection Results S6: Annotate and output the images after discrimination. Generate enhanced images based on the discrimination results: embed tower numbers, coordinates, and confidence information into the image file in XMP metadata format, and add semi-transparent watermarks to the four corners of the image. The system also supports adaptive annotation mode selection (as shown in Table 4), automatically switching between single-tower annotation, multi-tower annotation, or towerless annotation modes according to the scenario to ensure data integrity and annotation reliability.

[0017] Table 4 Adaptive Labeling Pattern Classification Although embodiments and drawings of the present invention have been disclosed for illustrative purposes, those skilled in the art will understand that various substitutions, variations and modifications are possible without departing from the spirit and scope of the present invention and the appended claims. Therefore, the scope of the present invention is not limited to the contents disclosed in the embodiments and drawings.

Claims

1. An automatic image annotation method based on UAV GPS and tower coordinates, characterized in that: The method aims to achieve efficient and accurate labeling of power poles in UAV inspection images by combining spatial distance calculation with image feature recognition. The steps of the method are as follows: S1. Data Preprocessing: Acquire and parse detailed parameters of the images taken by the UAV, extract the timestamp of the image capture time and GPS positioning information (including latitude, longitude, altitude and heading angle); at the same time, load the geographic information file KML, extract the number, type and spatial coordinate information of each power line tower, and construct a tower coordinate database. S2. Data Matching: The capture time of each image is matched with the corresponding drone flight trajectory point to obtain the precise drone position information at that moment. Then, the matched GPS coordinates in the WGS-84 coordinate system are converted into three-dimensional coordinates in the geocentric rectangular coordinate system (ECEF). Based on the converted ECEF coordinate system, the three-dimensional spatial distance between the current drone position and all poles in the database is calculated to obtain the spatial relative position relationship between each pole and the drone, which serves as a preliminary judgment basis. S3. Dual-tower competition discrimination mechanism: Select the two candidate towers closest to the drone's current position from all towers, and execute the following discrimination logic: a) If the distance between the nearest pole and the drone is less than or equal to the dynamically calculated distance threshold Dt, then the nearest pole is marked in the image. b) If the distance between both candidate towers and the drone is greater than Dt, it is determined that there are no valid towers visible in the current image and no annotation is made; c) If the distance ratio between two candidate towers is greater than the set ratio threshold (set according to the specific span between towers), then the tower with the closer distance is selected as the target for marking; d) If none of the above conditions are met, proceed to the next step; S4. Image Feature-Assisted Verification: When the target tower cannot be clearly determined based on spatial distance in S3, an image recognition algorithm is introduced to further analyze the candidate region: First, the approximate location of the tower in the image is estimated based on the spatial calculation results. Then, the YOLOv5 target detection model is used to identify the tower body and its nameplate information, and the recognition confidence score is output. If the recognition confidence score is greater than or equal to a preset threshold (this threshold is set according to the model training situation), the tower is confirmed as the target object and labeled; otherwise, it is determined that there is no valid tower in the image and no labeling is performed. S5. Adaptive Annotation Output: Generates an enhanced image with annotation information based on the discrimination results, specifically including: a) Embed information such as tower number, spatial coordinates, and discrimination confidence level into the image file in XMP metadata format to facilitate later retrieval and management; b) Add a semi-transparent watermark containing key information such as tower numbers to a preset area of ​​the image; c) Automatically select the appropriate annotation mode based on the image content, including single tower annotation, multi-tower annotation, or no annotation, to improve the applicability and reliability of the system in different inspection scenarios.

Images