A highway post number positioning method and system based on a UAV

Abstract

The application relates to an expressway pile number positioning method and system based on a UAV, and relates to the field of UAV expressway inspection. The method is applied to a UAV control system and comprises the following steps: acquiring dynamic coordinates of an inspection UAV and reference coordinates of a fixed nest; calculating the relative position relationship between the inspection UAV and the fixed nest according to the dynamic coordinates and the reference coordinates; judging whether the dynamic coordinates of the inspection UAV are valid coordinates according to the relative position relationship and a preset expressway direction map; if yes, according to a preset pile number-latitude and longitude table, the pile number range where the dynamic coordinates of the inspection UAV are located is inquired; acquiring a pile number sign image photographed by the inspection UAV in the pile number range, and extracting pile number coordinates in the pile number sign image; and according to the pile number coordinates, the photographing parameters of the inspection UAV and the expressway direction, the pile number coordinates of the inspection UAV are calculated. The method solves the problem of low pile number coordinate positioning precision of the current UAV.

Description

Technical Field

[0001] This application relates to the technical field of unmanned aerial vehicle (UAV) highway inspection, specifically to a UAV-based method and system for locating highway mileage markers. Background Technology

[0002] In the daily inspection of highways in my country, when road defects are found, their location is usually indicated by station coordinates in order to make it easier to know. For example, K120+220 indicates that it is 120 kilometers and 220 meters after the starting point of the highway.

[0003] In traditional technology, highway inspections are generally done manually. When inspectors discover road defects, they visually read the markers on the roadside to determine the specific coordinates of the defects. However, traditional inspection methods are not only inefficient, but also rely heavily on the inspectors' experience to pinpoint the coordinates, leading to inaccurate results. Therefore, with the widespread adoption of drone technology, automated highway inspection using drones has become an industry trend. Currently, drone inspections involve setting up drone nests along the roadside and releasing drones to fly along the road. During flight, the drones simultaneously capture images of the road to detect defects and use their onboard GPS system to determine their own latitude and longitude coordinates. Based on a pre-set marker-latitude / longitude data table, the drones convert their latitude and longitude coordinates into marker coordinates, thus completing highway inspections efficiently and intuitively.

[0004] However, in some complex highway scenarios (such as mountain roads and tunnels), GPS may experience coordinate drift due to multipath effects, resulting in a significant difference between the station coordinates of the UAV obtained by converting the UAV's latitude and longitude coordinates and the actual station coordinates. Summary of the Invention

[0005] To address the problem of low accuracy in mileage station positioning caused by GPS signal drift during the inspection of complex highways, this application provides a highway mileage station positioning method and system based on drones.

[0006] In a first aspect, this application provides a method for highway stationing based on unmanned aerial vehicles (UAVs), applied in a UAV control system, the method comprising: Obtain the dynamic coordinates of the inspection drone and the reference coordinates of the fixed drone nest; Based on the dynamic coordinates and the reference coordinates, the relative positional relationship between the inspection drone and the fixed drone nest is calculated, and the relative positional relationship includes the straight-line distance and the angle of orientation. Based on the relative positional relationship and the preset highway route map, determine whether the dynamic coordinates of the inspection drone are valid coordinates; If so, then according to the preset station number-latitude and longitude table, query the station number range where the dynamic coordinates of the inspection drone are located; The inspection drone captures images of the station markers within the specified station range, and extracts the station coordinates from the station marker images. The station coordinates of the inspection drone are calculated based on the station coordinates, the shooting parameters of the inspection drone, and the route of the highway.

[0007] Optionally, determining whether the dynamic coordinates of the inspection drone are valid coordinates based on the relative positional relationship and a preset highway route map specifically includes: Determine whether the orientation angle of the inspection drone is within a preset threshold value for the direction of the highway; Determine whether the straight-line distance of the inspection drone is within the preset straight-line distance range of the highway route; If the orientation angle of the inspection drone is within a preset angle threshold of the highway direction, and the straight-line distance of the inspection drone is within a preset straight-line distance of the highway direction, then the dynamic coordinates of the inspection drone are determined to be valid coordinates. If the orientation angle of the inspection drone is not within the preset angle threshold of the highway direction, and / or the straight distance of the inspection drone is not within the preset straight distance range of the highway direction, then the dynamic coordinates of the inspection drone are determined to be invalid coordinates. The corrected dynamic coordinates of the inspection drone were re-measured; Based on the corrected dynamic coordinates, the corrected orientation angle and corrected straight-line distance between the inspection drone and the fixed nest are recalculated to generate the corrected relative positional relationship. If the corrected orientation angle is within a preset angle threshold of the highway direction and the corrected straight-line distance is within a preset straight-line distance of the highway direction, then the corrected dynamic coordinates are used as the valid coordinates of the inspection drone.

[0008] Optionally, the step of calculating the relative positional relationship between the inspection drone and the fixed drone nest based on the dynamic coordinates and the reference coordinates further includes: If the corrected orientation angle is not within the preset angle threshold of the highway direction, and / or the corrected straight distance is not within the preset straight distance range of the highway direction, then the reference coordinates of the fixed nest will be replaced with the reference coordinates of the target communication beacon. The relative beacon dynamic coordinates of the inspection drone were re-determined; The relative positional relationship between the inspection drone and the target communication beacon is calculated based on the relative beacon dynamic coordinates and the reference coordinates of the target communication beacon. Replace the relative positional relationship between the inspection drone and the fixed nest with the relative positional relationship between the beacons.

[0009] Optionally, replacing the reference coordinates of the fixed nest with the reference coordinates of the target communication beacon specifically involves: Obtain a communication beacon deployment map along a predetermined highway, the communication beacon deployment map containing the reference coordinates of multiple temporary communication beacons deployed along the predetermined highway; Based on the corrected dynamic coordinates of the inspection drone, the nearest temporary communication beacon in the communication beacon deployment map is queried and used as the target communication beacon; Replace the reference coordinates of the fixed nest with the reference coordinates of the target communication beacon.

[0010] Optionally, the step of acquiring the station marker image taken by the inspection drone within the station range and extracting the station coordinates from the station marker image specifically involves: The RGB feature boxes of the station number sign in the station number sign image are identified; Obtain the infrared spectrum of the RGB feature box of the station number sign; Based on the infrared spectrum of the station sign, feature segmentation is performed on the noise features in the RGB feature box of the station sign and the station sign features to obtain the station sign feature image; The station number coordinates in the station number sign image are obtained by performing OCR recognition on the feature image of the station number sign.

[0011] Optionally, the step of performing OCR recognition on the feature image of the station sign to obtain the station coordinates in the station sign image further includes: The heading angle between the inspection drone and the preset route is detected, and the heading angle includes horizontal and vertical heading angles. When the heading angle is greater than the heading angle threshold, the path curvature and path slope of the road shoulder are calculated based on the station sign image. Based on the path curvature and the path slope, the feature image of the station sign is geometrically distorted to obtain the feature-corrected image of the station sign. The station number coordinates in the station number sign image are obtained by performing OCR recognition on the feature-corrected image of the station number sign.

[0012] Optionally, the step of performing geometric distortion correction on the station sign feature image based on the path curvature and the path slope to obtain a station sign feature-corrected image specifically involves: A spatial coordinate system is constructed with the inspection drone as the origin and the tangent and normal of the preset route as orthogonal axes; The feature image of the station number sign is mapped onto the spatial coordinate system to obtain the spatial image of the station number sign; Based on the path curvature and the path slope, a theoretical spatial image of the station sign is constructed. Calculate the inverse transformation matrix based on the theoretical spatial image of the station sign and the spatial image of the station sign; Using the inverse transformation matrix, the spatial image of the station sign is reprojected in the spatial coordinate system to obtain the feature-corrected image of the station sign.

[0013] Optionally, the step of calculating the station coordinates of the inspection drone based on the station coordinates, the shooting parameters of the inspection drone, and the highway route specifically involves: Use the station coordinates in the station marker image as the reference station coordinates; The shooting parameters of the inspection drone are converted into geometric parameters, including the straight-line distance and relative angle between the inspection drone and the station sign. Based on the geometric parameters, a geometric algorithm is used to calculate the relative station coordinates between the inspection drone and the station marker. Based on the highway route, the relative station coordinates are converted into the absolute station coordinates of the inspection drone, and the absolute station coordinates are used as the station coordinates of the inspection drone.

[0014] Secondly, this application provides a highway marker location system based on unmanned aerial vehicles (UAVs). The system is a UAV control system, which includes an acquisition module and a processing module, wherein: The acquisition module is used to acquire the dynamic coordinates of the inspection drone and the reference coordinates of the fixed drone nest; The processing module is used to calculate the relative positional relationship between the inspection drone and the fixed drone nest based on the dynamic coordinates and the reference coordinates. The relative positional relationship includes the straight-line distance and the angle of orientation. Based on the relative positional relationship and a preset highway route map, it determines whether the dynamic coordinates of the inspection drone are valid coordinates. If so, it queries the station range where the dynamic coordinates of the inspection drone are located based on a preset station-latitude and longitude table. The acquisition module is also used to acquire images of the station number signs taken by the inspection drone within the station number range, and to extract the station number coordinates from the station number signs images. The processing module is also used to calculate the station coordinates of the inspection drone based on the station coordinates, the shooting parameters of the inspection drone, and the direction of the highway.

[0015] Thirdly, this application provides an electronic device including a processor, a memory, a user interface, and a network interface. The memory is used to store instructions, the user interface and the network interface are used to communicate with other devices, and the processor is used to execute the instructions stored in the memory to cause the electronic device to perform the method as described in any one of the first aspects.

[0016] In summary, one or more technical solutions provided in the embodiments of this application have at least the following technical effects or advantages: This application uses the GPS coordinates of a fixed drone nest pre-aligned as the reference coordinates and stores these coordinates in the drone control system. During the inspection of highways by the drone, the drone uses its onboard GPS device to check its own dynamic coordinates at preset intervals. Since the dynamic coordinates may drift, this application calculates the relative positional relationship (straight-line distance and orientation angle) between the drone and the fixed drone nest. Based on this relationship, it determines whether the drone's dynamic coordinates are valid. If invalid, the dynamic coordinates are re-acquired and re-evaluated. If the drone's dynamic coordinates are valid, the approximate station range of the drone can be roughly determined. To obtain more accurate station coordinates, the drone's onboard camera photographs the station markers within the station range and extracts the station coordinates from them. These station coordinates are then used as the reference coordinates for drone positioning. Finally, based on the drone's shooting parameters and the highway's direction, a geometric algorithm is used to deduce the drone's accurate station coordinates. Therefore, when the dynamic coordinates of the UAV can only determine the approximate station range due to dynamic drift, this application further utilizes the accurate station coordinates on the station marker, combined with the shooting parameters and road direction, to effectively locate the accurate station coordinates of the UAV even when the dynamic coordinates of the UAV are inaccurate, thereby meeting the accuracy requirements for defect location in inspection work. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating a method for locating highway station numbers based on unmanned aerial vehicles (UAVs) provided in an embodiment of this application.

[0018] Figure 2This is a schematic diagram of a highway stationing system based on an unmanned aerial vehicle (UAV) provided in an embodiment of this application.

[0019] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0020] Explanation of reference numerals in the attached drawings: 1. Acquisition module; 2. Processing module; 300. Electronic device; 301. Processor; 302. Communication bus; 303. User interface; 304. Network interface; 305. Memory. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0022] This application provides a method for highway stationing based on unmanned aerial vehicles (UAVs). This method is applied in the UAV control system, such as... Figure 1 As shown, the method includes steps S101 to S106, which are as follows: S101. Obtain the dynamic coordinates of the inspection drone and the reference coordinates of the fixed drone nest.

[0023] In the above steps, firstly, the drone nest is fixedly installed at a designated location next to the highway, for example, in an open area 20-30m away from the shoulder, to avoid obstruction and signal blind spots. After the drone nest is installed, it is positioned and calibrated. Specifically, Beidou + GPS dual-mode positioning technology is used to collect the fixed latitude and longitude of the nest, and the positioning difference between the two is ensured to be within the range of (-0.5m, +0.5m). Then, the fixed latitude and longitude collected by the two are averaged to obtain the reference coordinates of the drone nest, and the reference coordinates of the drone nest are stored in the drone control system carried inside the inspection drone. Furthermore, based on the signal coverage range of the drone's nest, the route data of the highway to be inspected is retrieved. This route data consists of the latitude and longitude of the highway's shoulders. Since the marker signs are installed on the shoulders, a pre-built marker-latitude / longitude data table is also used to obtain the latitude, longitude, and marker numbers of multiple marker signs within the drone's nest's signal coverage range. These markers are then marked on the route data of the highway to be inspected, thus clarifying the spatial location of the highway corresponding to different marker numbers and providing a basis for subsequent positioning range calibration. Finally, the marked highway route data is stored in the drone control system of the inspection drone.

[0024] When the inspection drone performs highway inspection tasks, after taking off from the fixed nest, it uses its onboard Beidou + GPS dual positioning module to collect two dynamic latitude and longitude coordinates of its current position in real time at preset intervals. Then, it averages the two dynamic latitude and longitude coordinates to obtain the dynamic coordinates of the inspection drone's current position, and simultaneously reads the reference coordinates of the fixed nest.

[0025] S102. Based on the dynamic coordinates and the reference coordinates, calculate the relative positional relationship between the inspection drone and the fixed drone nest. The relative positional relationship includes the straight-line distance and the angle of orientation.

[0026] S103. Based on the relative positional relationship and the preset highway route map, determine whether the dynamic coordinates of the inspection drone are valid coordinates.

[0027] In steps S102 to S103 above, when the dynamic coordinates of the inspection drone drift due to environmental interference, the displayed drone position will deviate from the predetermined flight path. If the deviation is too large, it indicates that the dynamic coordinates are severely affected by interference, thus losing their positioning value. Conversely, if the deviation is small, it indicates that although the dynamic coordinates drift, the degree of drift is small, and the drone's position can still be roughly determined, thus possessing some positioning value. Therefore, this application calculates the straight-line distance between the inspection drone and the fixed drone nest, and the inspection... The orientation angle of the drone relative to the fixed drone nest is determined. Then, it is determined whether the orientation angle of the inspection drone is within the preset angle threshold of the highway route, and whether the straight distance of the inspection drone is within the preset straight distance range of the highway route. If the orientation angle of the inspection drone is within the preset angle threshold of the highway route, and the straight distance of the inspection drone is within the preset straight distance range of the highway route, then it is determined that the current position of the inspection drone deviates little from the predetermined route, and the dynamic coordinates of the current position are valid coordinates, which can be used as latitude and longitude coordinates to determine the approximate station range of the drone.

[0028] If the straight-line distance and the angle of orientation of the inspection drone meet only one of the above judgment conditions or neither of them, it is determined that the current position of the inspection drone deviates significantly from the predetermined route, and the dynamic coordinates of the current position are invalid coordinates. To avoid randomness, after determining that the dynamic coordinates of the inspection drone are invalid coordinates, the corrected dynamic coordinates of the inspection drone are re-measured. Based on the corrected dynamic coordinates, the corrected angle of orientation and the corrected straight-line distance between the inspection drone and the fixed nest are recalculated to generate a corrected relative position relationship. If the corrected relative position relationship meets the above two judgment conditions, the corrected dynamic coordinates can be used as the valid coordinates of the inspection drone for subsequent determination of the latitude and longitude coordinates of the approximate station range of the drone.

[0029] In one possible implementation, as the inspection work of the inspection drone continues, the distance between the inspection drone and the fixed nest gradually increases. If the dynamic coordinates of the inspection drone drift at this time, the relative positional relationship between the inspection drone and the fixed nest will be greatly affected by the dynamic drift due to the large distance. This will lead to a misjudgment of the validity of the dynamic coordinates of the inspection drone, that is, even after correcting the relative positional relationship, the two judgment conditions for valid coordinates are still not met. Therefore, this application first obtains a communication beacon deployment map along a pre-designed highway. The communication beacon deployment map contains the reference coordinates of multiple temporary communication beacons deployed along the pre-designed highway. Then, based on the corrected dynamic coordinates of the inspection drone, it queries the nearest temporary communication beacon in the communication beacon deployment map and uses it as the target communication beacon. At this time, the reference coordinates of the target communication beacon are read from the communication beacon deployment map as the new positioning reference for the inspection drone. Then, using the GPS positioning device on the inspection drone, the relative beacon dynamic coordinates of the inspection drone are re-determined. Then, based on the relative beacon dynamic coordinates and the reference coordinates of the target communication beacon, the relative positional relationship between the inspection drone and the target communication beacon is calculated. Finally, the relative positional relationship between the inspection drone and the fixed drone nest is replaced with the relative positional relationship of the beacon, thereby reducing the impact of the dynamic drift of the inspection drone caused by the excessive distance of the positioning reference on the calculation of the straight-line distance and the orientation angle.

[0030] In one possible implementation, when the relative distance between the inspection drone and the fixed drone nest is greater than or equal to a relative distance threshold, the reference coordinates of the fixed drone nest are replaced with the reference coordinates of the target communication beacon to further improve the positioning accuracy of the drone.

[0031] S104. If so, then according to the preset station number-latitude and longitude table, query the station number range where the dynamic coordinates of the inspection drone are located.

[0032] In the above steps, after determining that the dynamic coordinates of the inspection drone are valid coordinates, the latitude and longitude coordinates of multiple station signs near the dynamic coordinates can be queried according to the preset station number-latitude and longitude table and the preset query radius. Then, the station number coordinates of the latitude and longitude coordinates of the multiple station signs are extracted to confirm the approximate station number range of the dynamic coordinates of the inspection drone. For example, if there are 3 station number coordinates within the preset query radius, namely K120+000, K120+100, and K120+200, then the station number range can be K120+000 to K120+200, which means that the inspection drone is located between 120 kilometers and 120 kilometers and 200 meters after the starting point of the highway.

[0033] S105. Obtain images of the station markers taken by the inspection drone within the station range, and extract the station coordinates from the station marker images.

[0034] In the above steps, in order to determine a more accurate station location for the inspection drone, images of station signs taken by the inspection drone within the station range are acquired. Then, the station coordinates in the station sign images are extracted. Finally, the station coordinates are compared with the coordinates of multiple station signs included within a preset query radius to determine which station sign the inspection drone is located near.

[0035] In one possible implementation, over long-term use, the surface of the station marker may become covered with noise-generating materials (e.g., soil, paper, vegetation), leading to errors in the extracted station coordinate identification. To address this, this application employs the YOLOv8 image recognition algorithm. First, the RGB feature boxes of the station marker are extracted from the station marker image. Then, an infrared sensor mounted on an inspection drone is used to detect the infrared spectrum within the RGB feature boxes of the station marker. It should be noted that man-made station markers, due to their substrate material, reflective coating, and the paint used for printing characters, exhibit drastically different reflection and absorption characteristics in the infrared band compared to common noise-generating materials. Therefore, by identifying multiple different infrared bands in the infrared spectrum and extracting the corresponding infrared band for the station marker, the station marker within the RGB feature box can be segmented from the noise-generating materials, resulting in a station marker feature image. At this point, an OCR image recognition algorithm is used to identify the station numbers in the station marker feature image and convert them into a standard station coordinate format, thereby reducing the possibility of station coordinate identification errors.

[0036] In one possible implementation, in a continuous inspection drone station location scenario, the drone needs to constantly adjust its shooting angle to clearly capture images of the station markers. When planning the inspection route, a shooting route is pre-planned to ensure the shooting angle is optimal, and the inspection route is then formulated based on this route. When the drone follows this route, it is permissible to capture clear images of the station markers even with some deviation. However, when the drone deviates too much from the route, the shooting parameters become unsuitable for the current shooting scenario, resulting in geometric distortion in the captured station marker images (e.g., numbers are elongated or overlapped). This application addresses this issue by detecting… The heading angle between the inspection drone and the preset route includes horizontal and vertical heading angles. The horizontal heading angle represents the angle projected onto the horizontal plane between the drone's nose and the tangent of the preset route at that point. The vertical heading angle represents the angle projected onto the vertical plane between the drone's pitch angle and the longitudinal slope of the preset route at that point. If both the horizontal and vertical heading angles exceed preset thresholds, it indicates a significant heading yaw by the inspection drone, requiring correction of the captured station marker images. Specifically: Based on the images of the mileage markers, the path curvature and slope of the road shoulder (with the shoulder direction aligned with the flight path) can be identified. The path curvature reflects the correct horizontal heading of the flight path, and the path slope reflects the correct vertical heading. Therefore, a spatial coordinate system is first constructed using the inspection drone as the origin and the tangent and normal lines of the flight path as orthogonal axes to simulate the drone's shooting perspective on its normal flight path. Then, the characteristic image of the mileage marker is mapped onto this spatial coordinate system to obtain a spatial image of the mileage marker. Next, a theoretical spatial image of the mileage marker is constructed based on the path curvature and slope. This theoretical spatial image represents the orientation of the mileage marker as captured by the inspection drone on its normal flight path. It can be considered as a rectangular surface (containing four spatial corner points). In the spatial coordinate system, this rectangular surface is located at the projection point of the inspection drone's current position onto the road shoulder. The dimensions of the rectangular face of the standard station sign are the same as those of the standard station sign. The rectangular face is aligned with the tangent of the road shoulder. Due to the yaw of the inspection drone, the actual spatial image of the station sign has deviations at these four spatial corner points. To correct this deviation, an inverse transformation matrix is ​​calculated based on the theoretical spatial image and the actual spatial image of the station sign. The inverse transformation matrix represents the parameter matrix that transforms one spatial image point into another. Specifically, a 3x3 homography matrix is ​​first set. Then, the four source spatial corner points corresponding to the four spatial corner points in the theoretical spatial image of the station sign are extracted from the spatial image of the station sign. The homography matrix is ​​then used to construct the transformation relationship between the four spatial corner points and the four source spatial corner points. Based on this transformation, the homography matrix is ​​solved using a direct linear transformation algorithm, thus obtaining the inverse transformation matrix. At this point, the inverse transformation matrix is ​​used to perform an inverse affine transformation on other spatial image points of the station sign spatial image, and then the bilinear interpolation method is used to reproject them into the spatial coordinate system to obtain the feature-corrected image of the station sign.

[0037] After obtaining the feature-corrected image of the station marker, the station coordinates in the feature-corrected image are then identified using an OCR image recognition algorithm. This improves the image recognition accuracy of the inspection drone under yaw conditions and ensures the accuracy of subsequent station positioning by the inspection drone.

[0038] S106. Based on the station coordinates, the shooting parameters of the inspection drone, and the highway route, the station coordinates of the inspection drone are calculated.

[0039] In the above steps, after obtaining the station coordinates of the station marker near the inspection drone, these coordinates are used as the reference station coordinates. Then, the shooting parameters of the inspection drone are converted into geometric parameters, specifically including the straight-line distance and relative angle between the inspection drone and the station marker. Based on the straight-line distance and relative angle between the inspection drone and the station marker, a geometric algorithm (e.g., trigonometric functions, plane projection, etc.) can be used to calculate the relative station coordinates between the inspection drone and the station marker. For example, the relative station coordinates can be expressed as the inspection drone being located 20 meters southwest of the station marker at a 30-degree angle. Then, based on the highway direction, the relative station coordinates are converted into the absolute station coordinates of the inspection drone. For example, if the station marker coordinates are K120+100 and the highway direction is from south to north, then the absolute station coordinates of the inspection drone are: K120+100-20*sin30°=K120+90. At this point, the absolute station coordinates of the inspection drone are the same as the station coordinates of the inspection drone, thus achieving a precise conversion from the latitude and longitude coordinates of the inspection drone to the station coordinates.

[0040] Reference Figure 2 This application also provides a highway stationing system based on unmanned aerial vehicles (UAVs). The system is a UAV control system, which includes an acquisition module 1 and a processing module 2, wherein: Module 1 is used to acquire the dynamic coordinates of the inspection drone and the reference coordinates of the fixed drone nest; Processing module 2 is used to calculate the relative positional relationship between the inspection drone and the fixed drone nest based on the dynamic coordinates and the reference coordinates. The relative positional relationship includes the straight-line distance and the angle of orientation. Based on the relative positional relationship and the preset highway route map, it determines whether the dynamic coordinates of the inspection drone are valid coordinates. If so, it queries the station range where the dynamic coordinates of the inspection drone are located based on the preset station number-latitude and longitude table. Module 1 is also used to acquire images of the station markers taken by the inspection drone within the station range, and to extract the station coordinates from the station marker images; Processing module 2 is also used to calculate the station coordinates of the inspection drone based on the station coordinates, the shooting parameters of the inspection drone, and the direction of the highway.

[0041] It should be noted that the above embodiments of the apparatus are only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0042] This application also discloses an electronic device. (See reference...) Figure 3 , Figure 3 This is a schematic diagram of the structure of an electronic device disclosed in an embodiment of this application. The electronic device 300 may include: at least one processor 301, at least one network interface 304, a user interface 303, a memory 305, and at least one communication bus 302.

[0043] The communication bus 302 is used to enable communication between these components.

[0044] The user interface 303 may include a display screen and a camera. Optionally, the user interface 303 may also include a standard wired interface and a wireless interface.

[0045] The network interface 304 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface).

[0046] The processor 301 may include one or more processing cores. The processor 301 connects to various parts of the server using various interfaces and lines, and performs various server functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in memory 305, and by calling data stored in memory 305. Optionally, the processor 301 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 301 may integrate one or a combination of several of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content required for display; and the modem handles wireless communication. It is understood that the modem may also not be integrated into the processor 301 and may be implemented as a separate chip.

[0047] The memory 305 may include random access memory (RAM) or read-only memory. Optionally, the memory 305 may include a non-transitory computer-readable storage medium. The memory 305 may be used to store instructions, programs, code, code sets, or instruction sets. The memory 305 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the above-described method embodiments, etc.; the data storage area may store data involved in the above-described method embodiments, etc. Optionally, the memory 305 may also be at least one storage device located remotely from the aforementioned processor 301. (Refer to...) Figure 3 The memory 305, which serves as a computer storage medium, may include an operating system, a network communication module, a user interface module, and an application program for a highway stationing method based on unmanned aerial vehicles.

[0048] exist Figure 3 In the illustrated electronic device 300, the user interface 303 is mainly used to provide an input interface for the user and acquire user input data; while the processor 301 can be used to call an application program stored in the memory 305 for a highway marker location method based on a drone. When executed by one or more processors 301, the electronic device 300 performs one or more of the methods described in the above embodiments. It should be noted that, for the foregoing method embodiments, for the sake of simplicity, they are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, because according to this application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0049] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0050] In the various embodiments provided in this application, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some service interface; the indirect coupling or communication connection between apparatuses or units may be electrical or other forms.

[0051] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0052] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0053] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned memory includes various media capable of storing program code, such as USB flash drives, portable hard drives, magnetic disks, or optical disks.

[0054] The above description is merely an exemplary embodiment of this disclosure and should not be construed as limiting the scope of this disclosure. Any equivalent changes and modifications made in accordance with the teachings of this disclosure shall still fall within the scope of this disclosure. Other embodiments of this disclosure will be readily apparent to those skilled in the art upon consideration of the specification and the disclosure of practical truths.

[0055] This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not described in this disclosure. The specification and embodiments are to be considered exemplary only, and the scope and spirit of this disclosure are defined by the claims.

Claims

1. A method for highway stationing based on unmanned aerial vehicles (UAVs), characterized in that, The method, applied in unmanned aerial vehicle (UAV) control systems, includes: Obtain the dynamic coordinates of the inspection drone and the reference coordinates of the fixed drone nest; Based on the dynamic coordinates and the reference coordinates, the relative positional relationship between the inspection drone and the fixed drone nest is calculated, and the relative positional relationship includes the straight-line distance and the angle of orientation. Based on the relative positional relationship and the preset highway route map, determine whether the dynamic coordinates of the inspection drone are valid coordinates; If so, then according to the preset station number-latitude and longitude table, query the station number range where the dynamic coordinates of the inspection drone are located; The inspection drone captures images of the station markers within the specified station range, and extracts the station coordinates from the station marker images. Based on the station coordinates, the shooting parameters of the inspection drone, and the highway route, the station coordinates of the inspection drone are calculated, specifically as follows: Use the station coordinates in the station marker image as the reference station coordinates; The shooting parameters of the inspection drone are converted into geometric parameters, which include the straight-line distance and relative angle between the inspection drone and the station sign. Based on the geometric parameters, a geometric algorithm is used to calculate the relative station coordinates between the inspection drone and the station marker. Based on the highway route, the relative station coordinates are converted into the absolute station coordinates of the inspection drone, and the absolute station coordinates are used as the station coordinates of the inspection drone.

2. The method according to claim 1, characterized in that, The step of determining whether the dynamic coordinates of the inspection drone are valid coordinates based on the relative positional relationship and a preset highway route map specifically includes: Determine whether the orientation angle of the inspection drone is within a preset threshold value for the direction of the highway; Determine whether the straight-line distance of the inspection drone is within the preset straight-line distance range of the highway route; If the orientation angle of the inspection drone is within a preset angle threshold of the highway direction, and the straight-line distance of the inspection drone is within a preset straight-line distance of the highway direction, then the dynamic coordinates of the inspection drone are determined to be valid coordinates. If the orientation angle of the inspection drone is not within the preset angle threshold of the highway direction, and / or the straight distance of the inspection drone is not within the preset straight distance range of the highway direction, then the dynamic coordinates of the inspection drone are determined to be invalid coordinates. The corrected dynamic coordinates of the inspection drone were re-measured; Based on the corrected dynamic coordinates, the corrected orientation angle and corrected straight-line distance between the inspection drone and the fixed nest are recalculated to generate the corrected relative positional relationship. If the corrected orientation angle is within a preset angle threshold of the highway direction and the corrected straight-line distance is within a preset straight-line distance of the highway direction, then the corrected dynamic coordinates are used as the valid coordinates of the inspection drone.

3. The method according to claim 2, characterized in that, The step of calculating the relative positional relationship between the inspection drone and the fixed drone nest based on the dynamic coordinates and the reference coordinates specifically includes: If the corrected orientation angle is not within the preset angle threshold of the highway direction, and / or the corrected straight distance is not within the preset straight distance range of the highway direction, then the reference coordinates of the fixed nest will be replaced with the reference coordinates of the target communication beacon. The relative beacon dynamic coordinates of the inspection drone were re-determined; The relative positional relationship between the inspection drone and the target communication beacon is calculated based on the relative beacon dynamic coordinates and the reference coordinates of the target communication beacon. Replace the relative positional relationship between the inspection drone and the fixed nest with the relative positional relationship between the beacons.

4. The method according to claim 3, characterized in that, The step of replacing the reference coordinates of the fixed nest with the reference coordinates of the target communication beacon specifically involves: Obtain a communication beacon deployment map along a predetermined highway, the communication beacon deployment map containing the reference coordinates of multiple temporary communication beacons deployed along the predetermined highway; Based on the corrected dynamic coordinates of the inspection drone, the nearest temporary communication beacon in the communication beacon deployment map is queried and used as the target communication beacon; Replace the reference coordinates of the fixed nest with the reference coordinates of the target communication beacon.

5. The method according to claim 1, characterized in that, The specific steps for obtaining images of the station markers taken by the inspection drone within the station range and extracting the station coordinates from the station marker images are as follows: The RGB feature boxes of the station number sign in the station number sign image are identified; Obtain the infrared spectrum of the RGB feature box of the station number sign; Based on the infrared spectrum of the station sign, feature segmentation is performed on the noise features in the RGB feature box of the station sign and the station sign features to obtain the station sign feature image; The station number coordinates in the station number sign image are obtained by performing OCR recognition on the feature image of the station number sign.

6. The method according to claim 5, characterized in that, The step of performing OCR recognition on the feature image of the station sign to obtain the station coordinates in the station sign image specifically includes: The heading angle between the inspection drone and the preset route is detected, and the heading angle includes horizontal and vertical heading angles. When the heading angle is greater than the heading angle threshold, the path curvature and path slope of the road shoulder are calculated based on the station sign image. Based on the path curvature and the path slope, the feature image of the station sign is geometrically distorted to obtain the feature-corrected image of the station sign. The station number coordinates in the station number sign image are obtained by performing OCR recognition on the feature-corrected image of the station number sign.

7. The method according to claim 6, characterized in that, The geometric distortion correction of the station sign feature image is performed based on the path curvature and the path slope to obtain the station sign feature-corrected image, specifically as follows: A spatial coordinate system is constructed with the inspection drone as the origin and the tangent and normal of the preset route as orthogonal axes; The feature image of the station number sign is mapped onto the spatial coordinate system to obtain the spatial image of the station number sign; Based on the path curvature and the path slope, a theoretical spatial image of the station sign is constructed. Calculate the inverse transformation matrix based on the theoretical spatial image of the station sign and the spatial image of the station sign; Using the inverse transformation matrix, the spatial image of the station sign is reprojected in the spatial coordinate system to obtain the feature-corrected image of the station sign.

8. A highway stationing system based on unmanned aerial vehicles (UAVs), characterized in that, The system is used to execute the UAV-based highway stationing method as described in any one of claims 1 to 7. The UAV control system includes an acquisition module (1) and a processing module (2), wherein: The acquisition module (1) is used to acquire the dynamic coordinates of the inspection drone and the reference coordinates of the fixed nest; The processing module (2) is used to calculate the relative positional relationship between the inspection drone and the fixed drone nest based on the dynamic coordinates and the reference coordinates. The relative positional relationship includes the straight-line distance and the angle of orientation. Based on the relative positional relationship and the preset highway route map, it is determined whether the dynamic coordinates of the inspection drone are valid coordinates. If so, the station range of the dynamic coordinates of the inspection drone is queried according to the preset station number-latitude and longitude table. The acquisition module (1) is also used to acquire images of the station number signs taken by the inspection drone within the station number range, and extract the station number coordinates from the station number signs images; The processing module (2) is also used to calculate the station coordinates of the inspection drone based on the station coordinates, the shooting parameters of the inspection drone, and the direction of the highway.

9. An electronic device, characterized in that, The device includes a processor (301), a memory (305), a user interface (303), and a network interface (304). The memory (305) is used to store instructions. The user interface (303) and the network interface (304) are used to communicate with other devices. The processor (301) is used to execute the instructions stored in the memory (305) to cause the electronic device (300) to perform the method as described in any one of claims 1 to 7.

Images