A method for autonomous positioning of unmanned aerial vehicles in cities based on semantic maps

Abstract

This invention belongs to the field of autonomous drone localization, and relates to a method for autonomous drone localization within cities based on semantic maps. The method includes semantic map construction, extraction of semantic elements from drone images, semantic element matching, and drone localization under topological constraints. The semantic map construction process identifies elements that are prevalent in the city and can be extracted with high precision. For low-altitude drones, signs, road signs, and their text are identified from drone images. For mid- and high-altitude drones, areal semantic elements in the semantic map are identified from drone images. For low-altitude drones, the identified text is matched with the semantic map. For high-altitude drones, the extracted local spatial scene is retrieved from the semantic map; a successful match indicates the geographical location of the drone's imaging area. Finally, topological relationships are used to confirm the localization of multiple drone scenes. This invention has low computational complexity, high accuracy, and can meet the autonomous localization requirements of high, medium, and low-altitude drones.

Description

Technical Field

[0001] The present invention belongs to the field of autonomous positioning of unmanned aerial vehicles, and in particular relates to an autonomous positioning method of unmanned aerial vehicles in a city based on a semantic map, which is used to realize autonomous positioning of unmanned aerial vehicles under GNSS denial conditions. Background Art

[0002] An efficient and accurate positioning system is an important factor affecting whether a drone can successfully perform its mission. Currently, the commonly used drone positioning technology is a navigation method that combines satellite navigation positioning technology (GNSS) and inertial navigation technology (IMU). However, GNSS signals are usually susceptible to interference and malicious damage. Over-reliance on GNSS signals will reduce the reliability and safety of drone positioning and navigation. If the support of GNSS signals is lost, the single inertial navigation method will produce error accumulation in the positioning process due to its own device properties. Drones cannot achieve accurate positioning during long-duration flights relying solely on the inertial navigation system.

[0003] The most widely used outdoor UAV autonomous positioning method is visual positioning based on heterogeneous image matching, that is, using high-resolution remote sensing images as the base map, registering the images taken by the UAV to the base map, performing UAV image positioning, and finally achieving UAV positioning. This type of method has difficulty in meeting or even failing to meet matching accuracy due to differences in imaging time, illumination, and image resolution, especially when the UAV is flying at a very low altitude, it is impossible to perform UAV positioning; in addition, due to the large amount of high-resolution remote sensing image data, directly extracting and matching images requires large amounts of computation, making it difficult to perform large-scale autonomous positioning of UAVs.

[0004] In order to improve the autonomous positioning accuracy of drones and improve the positioning efficiency at the same time, the present invention proposes an autonomous positioning method for drones in cities based on semantic maps, starting from the way humans identify unfamiliar places. Summary of the invention

[0005] The purpose of the present invention is to solve the problem that outdoor UAVs cannot perform autonomous positioning when the GNSS signal is lost. A method for autonomous positioning of UAVs in cities based on semantic maps is proposed. A semantic map is constructed using multi-source geographic information data, and semantic elements are automatically identified using UAV images and matched with the semantic map to achieve autonomous positioning of the UAV.

[0006] In order to achieve the above object, the technical solution adopted by the present invention is as follows:

[0007] A method for autonomous positioning of a UAV in a city based on a semantic map comprises the following steps:

[0008] (1) Obtaining the autonomous positioning data required for high-altitude UAVs and the latitude and longitude information and attribute information of each feature, and obtaining the autonomous positioning data required for low-altitude UAVs and the latitude and longitude information and name of each point, and constructing a semantic map; the autonomous positioning data required for high-altitude UAVs include building outline data, basketball court outline data, football field outline data, water body outline data, road data, large-area grassland outline data, and large-area forest outline data, and the autonomous positioning data required for low-altitude UAVs include POI point data and road sign point data;

[0009] (2) For high-altitude drones, AI algorithms are used to extract information about buildings, basketball courts, football fields, water bodies, roads, grasslands, and woodlands from images taken by drones; for low-altitude drones, AI algorithms are used to distinguish individual signs and road signs from images taken by drones, and to recognize the text on signs and road signs;

[0010] (3) For medium and high altitude drones, the extracted outlines of buildings, basketball courts, football fields, water bodies, roads, grasslands and woodlands are matched with the semantic map using a spatial scene retrieval algorithm. The matching process takes into account the topological relationship and distance relationship between different semantic patches. The scene that is successfully matched is the drone image imaging scene, and the geographical location of the scene is the drone imaging scene location. For low altitude drones, the text of a single sign or road sign extracted is used to perform correlation retrieval with the POI point data and road sign point data in the semantic map. The retrieved POI or road sign location is the drone imaging scene location.

[0011] (4) Return to step (2), obtain the UAV imaging scene positions in other scenes, calculate the spatial distance between the scene positions, and estimate the UAV's movement distance range based on the UAV's movement time. If the spatial distance between the scene positions is within the UAV's movement distance range, then the UAV imaging scene position located using semantic information is correct.

[0012] Furthermore, in step (1), the autonomous positioning requirement data of the high-altitude UAV is stored in a surface shapefile, and the autonomous positioning requirement data of the low-altitude UAV is stored in a point shapefile.

[0013] Compared with the background technology, the present invention has the following advantages:

[0014] 1. The present invention proposes an autonomous positioning method for UAVs in cities based on semantic maps, which uses building outline data, basketball court outline data, football field outline data, water body outline data, road data, large-area grassland outline data, large-area woodland outline data, POI point data, and road sign point data as semantic maps. The method has the characteristics of being widely distributed in cities and easy to be extracted from images with high precision.

[0015] 2. The present invention proposes a method for autonomous positioning of UAVs in cities based on semantic maps, which stores semantic maps in point shapefiles and surface shapefiles, has a small amount of data and is easy to calculate.

[0016] 3. The present invention proposes a semantic map-based autonomous positioning framework for UAVs in cities, which can simultaneously meet the needs of autonomous positioning of low-altitude and medium- and high-altitude UAVs. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 This is a schematic diagram of a method for autonomous positioning of a UAV in a city based on a semantic map according to the present invention. DETAILED DESCRIPTION

[0018] The specific implementation of the present invention is described below in conjunction with the accompanying drawings so that those skilled in the art can better understand the present invention. It should be noted that in the following description, when the detailed description of known functions and designs may dilute the main content of the present invention, these descriptions will be omitted here.

[0019] Figure 1 It is a principle framework diagram of a specific implementation of a method for autonomous positioning of a UAV in a city based on a semantic map according to the present invention.

[0020] In this embodiment, if Figure 1 The method of autonomous positioning of UAV in the city based on semantic map includes four steps: semantic map construction, extraction of semantic elements from UAV images, semantic element matching and positioning of UAV under topological relationship constraints. The specific steps are as follows:

[0021] (1) Semantic map construction. To meet the autonomous positioning requirements of medium and high altitude UAVs, the semantic map needs to include building outline data, basketball court outline data, football field outline data, water body outline data, road data, large area grassland outline data, and large area forest outline data. The data should be stored in a surface shapefile to record the longitude and latitude information and attribute information of each feature. To meet the autonomous positioning requirements of low altitude UAVs, the semantic map needs to include POI point data and road sign point data. The data should be stored in a point shapefile to record the longitude and latitude information and name and other attribute information of each point.

[0022] (2) Extraction of semantic elements from drone images. For medium and high altitude drones, artificial intelligence algorithms are used to extract information about buildings, basketball courts, football fields, water bodies, roads, grasslands, and woodlands from drone images. For low altitude drones, artificial intelligence algorithms are used to first distinguish individual signs and road signs from drone images, and then identify the text on the signs and road signs.

[0023] (3) Semantic element matching: For medium and high altitude drones, the spatial scene retrieval algorithm is used to match the extracted outlines of buildings, basketball courts, football fields, water bodies, roads, grasslands, and woodlands with the semantic map. The matching process takes into account the topological relationship and distance relationship between different semantic patches. The scene that is successfully matched is the drone image imaging scene, and the geographical location of the scene is the drone imaging scene location. For low altitude drones, the text of the extracted single signboard or road sign is used to perform correlation retrieval with the POI point data and road sign point data in the semantic map. The retrieved POI or road sign location is the drone imaging scene location.

[0024] (4) UAV positioning under topological relationship constraints: return to step (2) to obtain the UAV imaging scene position in other scenes. That is, considering that there may be errors in the semantic element matching in a single scene, multiple scenes are used for matching, and the spatial distance between the scene positions is calculated. Finally, the UAV movement distance range is estimated based on the UAV movement time. If the spatial distance between the scene positions is within the UAV movement distance range, the UAV imaging scene position located using semantic information is considered to be correct.

[0025] The present invention realizes a framework for autonomous positioning of unmanned aerial vehicles in cities based on semantic maps, which includes four steps: semantic map construction, extraction of semantic elements from unmanned aerial vehicle images, semantic element matching, and positioning of unmanned aerial vehicles under topological relationship constraints. In the process of semantic map construction, the observation objects under different flight altitude conditions of unmanned aerial vehicles are considered, and elements that are commonly present in cities and can be extracted with high precision are found as semantic map elements; for low-altitude unmanned aerial vehicles, signboards and road signs, that is, the text on them, are identified from unmanned aerial vehicle images; for medium and high altitude unmanned aerial vehicles, surface semantic elements in other semantic maps are identified from unmanned aerial vehicle images; for low-altitude unmanned aerial vehicles, the identified text is matched with POI or road sign text, and for high-altitude unmanned aerial vehicles, the extracted local spatial scene is retrieved from the semantic map, and a successful match means that the geographical location of the unmanned aerial vehicle imaging area is retrieved; finally, considering that there may be errors in the matching of a single scene, the positioning of multiple unmanned aerial vehicle scenes is confirmed using topological relationships, and finally the precise position of the unmanned aerial vehicle is determined. Compared with the traditional unmanned aerial vehicle positioning based on image matching, the present invention proposes an autonomous positioning solution for unmanned aerial vehicles in cities based on semantic maps, which has small calculation amount, high accuracy, and can meet the autonomous positioning of high, medium and low altitude unmanned aerial vehicles at the same time.

[0026] Although the above describes the illustrative specific embodiments of the present invention to facilitate those skilled in the art to understand the present invention, it should be clear that the present invention is not limited to the scope of the specific embodiments. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations using the concept of the present invention are protected.

Claims

1. A method for autonomous positioning of unmanned aerial vehicles in cities based on semantic maps, characterized in that: The following steps are involved: (1) Obtaining the autonomous positioning data required for high-altitude UAVs and the latitude and longitude information and attribute information of each feature, and obtaining the autonomous positioning data required for low-altitude UAVs and the latitude and longitude information and name of each point, and constructing a semantic map; the autonomous positioning data required for high-altitude UAVs include building outline data, basketball court outline data, football field outline data, water body outline data, road data, large-area grassland outline data, and large-area forest outline data, and the autonomous positioning data required for low-altitude UAVs include POI point data and road sign point data; (2) For high-altitude drones, AI algorithms are used to extract information about buildings, basketball courts, football fields, water bodies, roads, grasslands, and woodlands from images taken by drones; for low-altitude drones, AI algorithms are used to distinguish individual signs and road signs from images taken by drones, and to recognize the text on signs and road signs; (3) For medium and high altitude drones, the extracted outlines of buildings, basketball courts, football fields, water bodies, roads, grasslands and woodlands are matched with the semantic map using a spatial scene retrieval algorithm. The matching process takes into account the topological relationship and distance relationship between different semantic patches. The scene that is successfully matched is the drone image imaging scene, and the geographical location of the scene is the drone imaging scene location. For low altitude drones, the text of a single sign or road sign extracted is used to perform correlation retrieval with the POI point data and road sign point data in the semantic map. The retrieved POI or road sign location is the drone imaging scene location. (4) Return to step (2), obtain the UAV imaging scene positions in other scenes, calculate the spatial distance between the scene positions, and estimate the UAV's movement distance range based on the UAV's movement time. If the spatial distance between the scene positions is within the UAV's movement distance range, then the UAV imaging scene position located using semantic information is correct.

2. The method for autonomous positioning of unmanned aerial vehicles in cities based on semantic maps according to claim 1 is characterized in that: In step (1), the autonomous positioning requirement data of high-altitude UAVs is stored in a surface shapefile, and the autonomous positioning requirement data of low-altitude UAVs is stored in a point shapefile.

Images