Unmanned aerial vehicle inspection route generation method and device, computer equipment and storage medium

Abstract

The invention relates to an unmanned aerial vehicle inspection route generation method and device, computer equipment and a storage medium. The method comprises the steps that under the condition that public interest position data matched with a target operation area does not exist in a pre-stored interest position database, a to-be-processed ground feature image of the target operation area is acquired; inputting the to-be-processed ground feature image into a pre-trained ground feature element extraction model to obtain a ground feature element set; acquiring different address description information input by a user for different ground feature elements in the ground feature element set; the address description information is semantic description of the user on the ground feature elements; obtaining user interest position data corresponding to the ground feature elements according to the address description information and the geographic coordinate information corresponding to the ground feature elements, and adding the user interest position data to an interest position database; and generating an inspection route of the unmanned aerial vehicle in the target operation area by using the interest position database added with the user interest position data. By adopting the method, the generation efficiency can be improved.

Description

Technical Field

[0001] The present application relates to the field of drone technology, and in particular to a method, device, computer equipment and storage medium for generating a drone inspection route. Background Art

[0002] With the development of drone technology, industrial drones are widely used in fields such as power inspection and forest fire patrol. However, drone missions in the above application fields often occur in remote areas without significant landmark features. In other words, these areas lack public geographic location of interest (LOI) data. Under such conditions, it is difficult for the system to issue precise control instructions to accurately control the drone to perform specific tasks. Summary of the invention

[0003] Based on this, it is necessary to provide a method, device, computer equipment and computer-readable storage medium for generating drone inspection routes that can accurately control drones in areas where public LOI data is lacking, in response to the above technical problems.

[0004] In a first aspect, the present application provides a method for generating a drone inspection route, comprising:

[0005] When there is no public interest location data matching the target operation area in the pre-stored interest location database, obtaining a to-be-processed ground feature image of the target operation area;

[0006] Inputting the to-be-processed ground object image into a pre-trained ground object element extraction model to obtain a ground object element set;

[0007] Acquire different address description information input by the user for different geographical features in the geographical feature set; the address description information is the semantic description of the geographical feature by the user;

[0008] According to the address description information and geographic coordinate information corresponding to each geographical feature, obtaining the user's interest location data corresponding to the geographical feature, and adding the user's interest location data to the interest location database;

[0009] The inspection route of the UAV in the target operation area is generated by using the interest location database after adding the user's interest location data.

[0010] In one embodiment, the step of obtaining geographic coordinate information corresponding to each feature element includes:

[0011] Based on the spatial information expression standard, obtaining the normalized feature contour corresponding to the feature element;

[0012] The normalized feature outline is projected into the geographic coordinate system of the geographic information system to obtain the polygonal geographic coordinates of the normalized feature outline, and the polygonal geographic coordinates are used as the geographic coordinate information corresponding to the feature element.

[0013] In one embodiment, the step of obtaining geographic coordinate information corresponding to each feature element includes:

[0014] The geographical features in the geographical feature set are image matched with the map tiles in the geographic information system, and the geographical coordinate information corresponding to the geographical features is determined according to the matched map tiles.

[0015] In one embodiment, the generating of the inspection route of the drone in the target operation area by using the interest location database after adding the user's interest location data includes:

[0016] Obtaining address description information for each target location in the target operation area;

[0017] The target user interest location data matching the address description information of the target location is obtained from the interest location database after the user interest location data is added, and an inspection route of the drone in the target operation area is generated according to the target user interest location data.

[0018] In one embodiment, the step of acquiring the target user's interest location data that matches the address description information of the target location from the interest location database after adding the user's interest location data includes:

[0019] Acquire candidate user interest location data matching the address description information of the target location from the interest location database after adding the user interest location data, and acquire geographic coordinate information in the candidate user interest location data;

[0020] The current location information and flight range of the drone are obtained, and based on the current location information and the geographic coordinate information in the candidate user interest location data, the candidate user interest location data that matches the flight range of the drone is determined and used as the target user interest location data.

[0021] In one embodiment, the method further comprises:

[0022] Obtaining an inspection result of the drone, the inspection result being obtained after the drone inspects the target operation area according to an inspection route;

[0023] Obtain the geographic coordinate information in the inspection result, determine the address description information that matches the geographic coordinate information in the inspection result, associate the matched address description information with the corresponding geographic coordinate information, update the inspection result according to the associated information, and display the updated inspection result.

[0024] In a second aspect, the present application also provides a drone inspection route generation device, comprising:

[0025] A ground object image acquisition module, used for acquiring a ground object image to be processed in the target operation area when there is no public interest location data matching the target operation area in the pre-stored interest location database;

[0026] An element extraction module is used to input the ground object image to be processed into a pre-trained ground object element extraction model to obtain a ground object element set;

[0027] An address description acquisition module is used to acquire different address description information input by a user for different geographical features in the geographical feature set; the address description information is a semantic description of the geographical feature by the user;

[0028] A location data generation module, used to obtain user interest location data corresponding to the geographical features according to the address description information and geographical coordinate information corresponding to the geographical features, and add the user interest location data to the interest location database;

[0029] The inspection route generation module is used to generate an inspection route for the UAV in the target operation area by using the interest location database after adding the user's interest location data.

[0030] In a third aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0031] When there is no public interest location data matching the target operation area in the pre-stored interest location database, obtaining a to-be-processed ground feature image of the target operation area;

[0032] Inputting the to-be-processed ground object image into a pre-trained ground object element extraction model to obtain a ground object element set;

[0033] Acquire different address description information input by the user for different geographical features in the geographical feature set; the address description information is the semantic description of the geographical feature by the user;

[0034] According to the address description information and geographic coordinate information corresponding to each geographical feature, obtaining the user's interest location data corresponding to the geographical feature, and adding the user's interest location data to the interest location database;

[0035] The inspection route of the UAV in the target operation area is generated by using the interest location database after adding the user's interest location data.

[0036] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the following steps are implemented:

[0037] When there is no public interest location data matching the target operation area in the pre-stored interest location database, obtaining a to-be-processed ground feature image of the target operation area;

[0038] Inputting the to-be-processed ground object image into a pre-trained ground object element extraction model to obtain a ground object element set;

[0039] Acquire different address description information input by the user for different geographical features in the geographical feature set; the address description information is the semantic description of the geographical feature by the user;

[0040] According to the address description information and geographic coordinate information corresponding to each geographical feature, obtaining the user's interest location data corresponding to the geographical feature, and adding the user's interest location data to the interest location database;

[0041] The inspection route of the UAV in the target operation area is generated by using the interest location database after adding the user's interest location data.

[0042] The above-mentioned method, device, computer equipment and computer-readable storage medium for generating a drone inspection route, when there is no public interest location data matching the target operation area in the pre-stored interest location database, first obtain the to-be-processed feature image of the target operation area; input the to-be-processed feature image into a pre-trained feature element extraction model to obtain a feature element set; obtain different address description information input by the user for different feature elements in the feature element set; the address description information is the user's semantic description of the feature element; obtain the user interest location data corresponding to the feature element according to the address description information and geographic coordinate information corresponding to each feature element, and add the user interest location data to the interest location database; generate the drone inspection route in the target operation area using the interest location database after adding the user interest location data. This application targets a target operation area that does not have matching public interest location data, obtains user interest location data through the address description information and geographic coordinate information corresponding to the feature element, and can perform related interactive operation area planning tasks for the target operation area through the user interest location data, which can improve the generation efficiency of drone inspection routes in operation areas without public interest location data. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0044] Figure 1 It is a flowchart of a method for generating a drone inspection route in one embodiment;

[0045] Figure 2 It is a flowchart of the steps of obtaining geographic coordinate information corresponding to each feature element in one embodiment;

[0046] Figure 3 A schematic diagram of a flow chart of steps for generating an inspection route in one embodiment;

[0047] Figure 4 A flowchart of a drone inspection method lacking geocoding of locations of interest in another embodiment;

[0048] Figure 5 It is a structural block diagram of a device for generating a drone inspection route in one embodiment;

[0049] Figure 6 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0050] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0051] In an exemplary embodiment, Figure 1 As shown, a method for generating a drone inspection route is provided. This embodiment uses the method applied to a server as an example. It can be understood that the method can also be applied to a terminal, and can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0052] Step S202: when there is no public interest location data matching the target operation area in the pre-stored interest location database, obtain a to-be-processed ground feature image of the target operation area.

[0053] Among them, the interest location database may refer to a database for storing interest location data. The interest location data may refer to data of key locations and their related geographic information, and the interest location data may be used to support the drone to perform precise positioning and route planning. The interest location database may be pre-stored in a server, or may be pre-stored in an onboard memory of a drone, so that the server can call the interest location database, and the onboard controller of the drone can call the interest location database. Public interest location data may refer to publicly available and accessible interest location data; the pre-stored interest location database pre-stores the public interest location data, and the public interest location data may also be obtained through online acquisition. The target operation area may refer to the specific geographical scope where the drone needs to perform inspection tasks, which can be obtained by pre-parsing the inspection tasks input by the user by the server. The image of the object to be processed may refer to the image data containing various types of objects in the target operation area. Objects may refer to surface elements such as buildings, roads, waters, and vegetation.

[0054] Exemplarily, the server searches for data matching the target operation area in a pre-stored database of locations of interest. If no matching public location of interest data exists, the server obtains the collected image of the target operation area to be processed. The image of the target operation area to be processed can be collected in advance or in real time by a drone.

[0055] Step S204: input the ground object image to be processed into a pre-trained ground object element extraction model to obtain a ground object element set.

[0056] The ground feature extraction model may refer to a trained artificial intelligence model for extracting all surface features in an input ground feature image. The ground feature extraction model may be deployed in a server.

[0057] Exemplarily, the server inputs the to-be-processed ground object image into a pre-trained ground object element extraction model to obtain a ground object element set consisting of a plurality of ground object elements.

[0058] Step S206, obtaining different address description information input by the user for different geographical features in the geographical feature set; wherein the address description information is the semantic description of the geographical feature by the user.

[0059] The address description information may be a user's semantic description of the feature, that is, it may refer to a text description provided by the user for the feature in the feature image to be processed based on his or her own experience and understanding. The text description may be the location or characteristics of the feature expressed by the user in natural language, which may be used to establish a more accurate definition of the point of interest. For example, the address description information for feature 1 may be Zhang's house, the address description information for feature 2 may be a pond next to a fork in the road, and the address description information for feature 3 may be a small bridge near a village, etc.

[0060] Exemplarily, the server may provide browsing prompts to the user for the geographical feature set and allow the user to provide a detailed semantic description. The server finally obtains different address description information input by the user for different geographical features in the geographical feature set.

[0061] Step S208 , obtaining user interest location data corresponding to the geographical feature elements according to the address description information and geographic coordinate information corresponding to the geographical feature elements, and adding the user interest location data to the interest location database.

[0062] Among them, the geographic coordinate information corresponding to the feature may refer to the coordinate information corresponding to the feature in the geographic information system. User interest location data may refer to new interest point record data formed by combining the user's semantic description and geographic coordinate information, which are added to the interest location database as user interest location data for subsequent use. In some examples, public interest location data may include address information of a location, such as a certain city, a certain street, a certain house number, and a general description of the location, such as a certain building, a certain park, etc.; and user interest location data is composed of address description information and geographic coordinate information corresponding to the feature. For example, user interest location data is composed of geographic coordinates (A, B) and address description information (Zhang's house). Based on this user interest location data, users can easily and quickly issue inspection instructions when inspecting the target operation area.

[0063] Exemplarily, the server combines the address description information corresponding to each geographical feature with the geographical coordinate information corresponding thereto to obtain the user interest location data corresponding to the geographical feature, and adds the user interest location data to the interest location database.

[0064] Step S210, generating an inspection route for the drone in the target operation area using the interest location database after adding the user's interest location data.

[0065] Among them, the inspection route can refer to a series of flight path points pre-set for the drone to ensure that the drone can cover the entire target operation area and focus on inspecting all designated locations of interest.

[0066] Exemplarily, the server plans the inspection route of the drone in the target operation area by using the interest location database after adding the user's interest location data.

[0067] In the above-mentioned UAV inspection route generation method, when there is no matching public interest location data in the target operation area, the to-be-processed object image of the target operation area is extracted through a pre-trained object feature extraction model to obtain a set of object features, and then the different address description information input by the user for different object features in the set of object features is obtained, and the user interest location data corresponding to the object features is obtained according to the address description information and geographic coordinate information corresponding to the object features, and the user interest location data is added to the interest location database; the UAV inspection route in the target operation area is generated through the user interest location data in the interest location database; for the target operation area without matching public interest location data, the user interest location data is obtained through the address description information and geographic coordinate information corresponding to the object features, and the user interest location data can be performed for the target operation area through the user interest location data. The related interactive operation area planning tasks can be performed for the target operation area, which can improve the generation efficiency of UAV inspection routes in the operation area without public interest location data. In some embodiments, when there is no public interest location data in some work areas, but there is matching user interest location data, when it is necessary to assign a drone to perform specific target inspections in the work area, the interest location database that stores matching user interest location data can be used through reverse geocoding technology to quickly locate and identify the specific area to be inspected, thereby achieving a high degree of automation and precision in the task allocation process.

[0068] In an exemplary embodiment, Figure 2 As shown, the steps for obtaining the geographic coordinate information corresponding to each feature element may specifically include:

[0069] Step S302: obtaining the normalized feature contours corresponding to the feature elements based on the spatial information expression standard.

[0070] The spatial information expression standard may refer to a series of rules and protocols for describing and encoding geospatial data, such as standards based on the OGC (Open Geospatial Consortium). The normalized feature outline may refer to a geometric representation obtained by standardizing the shape of the original feature element according to the spatial information expression standard.

[0071] Exemplarily, the server standardizes the expression of the outline of the land feature based on the spatial information expression standard to obtain the normalized land feature outline corresponding to the land feature.

[0072] Step S304, projecting the normalized feature outline into the geographic coordinate system of the geographic information system to obtain the polygonal geographic coordinates of the normalized feature outline, and using the polygonal geographic coordinates as the geographic coordinate information corresponding to the feature element.

[0073] Among them, geographic information system can refer to a specific spatial information system, which is a technical system that collects, stores, manages, calculates, analyzes, displays and describes the geographical distribution data in the entire or part of the earth's surface (including the atmosphere) space with the support of computer hardware and software systems. Geographic coordinate system can refer to a way of defining the position on the earth's surface, using longitude and latitude to represent the position of the point. Polygonal geographic coordinates can refer to the coordinate set of all vertices of the closed area of ​​the normalized feature outline. Each vertex has a pair of longitude and latitude values. The normalized feature outline formed by connecting these vertices can describe the range or boundary of the feature element.

[0074] Exemplarily, the server calls the geographic information system, projects the normalized feature outline into the geographic coordinate system in the geographic information system, finds the longitude and latitude coordinates corresponding to the polygon vertices of the normalized feature outline, forms a polygonal geographic coordinate set, and uses the polygonal geographic coordinates represented by the geographic coordinate set as the geographic coordinate information corresponding to the feature element.

[0075] In this embodiment, by following the spatial information expression standard, the standardization and accuracy of the feature outline can be ensured, thereby improving the accuracy of the geographic coordinate information corresponding to the feature elements obtained through the geographic information system.

[0076] In an exemplary embodiment, the step of acquiring geographic coordinate information corresponding to each feature may also include: performing image matching between features in the feature set and map tiles in the geographic information system, and determining the geographic coordinate information corresponding to the features based on the matched map tiles.

[0077] Among them, map tiles can refer to cutting a complete large map or image into several small tiles, each tile is a map tile, representing a part of the map; map tiles can be applied to geographic information systems. Among them, image matching can refer to finding the similarity and correspondence between two or more images. In this embodiment, it can refer to comparing the image corresponding to the feature element with the feature tile provided by the geographic information system, determining the geographical location of the feature element, and then obtaining the geographical coordinate information corresponding to the feature element.

[0078] Exemplarily, the server calls the geographic information system to obtain map tiles of the target operation area in the geographic information system, then performs image matching between the features in the feature set and the map tiles, and uses the geographic coordinate information of the matched map tiles as the geographic coordinate information corresponding to the corresponding features.

[0079] In this embodiment, by directly utilizing the map tile resources in the existing geographic information system, the geographic coordinate information of the ground feature elements can be quickly acquired without additional data collection work, thereby improving acquisition efficiency.

[0080] In an exemplary embodiment, Figure 3 As shown, the inspection route of the drone in the target operation area is generated by using the interest location database after adding the user's interest location data, which may specifically include:

[0081] Step S402: Acquire address description information for each target location in the target operation area.

[0082] The address description information for each target location in the target operation area may refer to the address description information for each target location in the target operation area contained in the inspection task generation request initiated by the user.

[0083] Exemplarily, a user may initiate an inspection task generation request to a server, in which the address description information of each target location in the target operation area may be provided. The server obtains the address description information for each target location in the target operation area through the inspection task generation request. For example, the inspection task generation request A may specifically include: first inspect the location corresponding to address description information a, then inspect the location corresponding to address description information b, then inspect the location corresponding to address description information c, and finally inspect the location corresponding to address description information d; the server may clean and standardize the above information to obtain address description information a, address description information b, address description information c, and address description information d. Such processing may obtain pure address description information for subsequent matching.

[0084] Step S404, obtaining the target user's interest location data that matches the address description information of the target location from the interest location database after adding the user's interest location data, and generating an inspection route for the drone in the target operation area according to the target user's interest location data.

[0085] Exemplarily, the server obtains target user interest location data that matches the address description information of the target location from the interest location database after the user interest location data is added. For example, the server searches for the address description information in each user interest location data in the interest location database after the user interest location data is added until the address description information in one user interest location data is found to match the address description information of the target location, and then uses this user interest location data as the target user interest location data; after the server obtains the target user interest location data corresponding to each target location, it plans the inspection route of the drone in the target operation area according to each target user interest location data.

[0086] In this embodiment, when users participate in the planning of drone inspection tasks, they do not need to use complex and semanticless geographic coordinates. Instead, they can use an intuitive and easy-to-use user interaction method to provide address description information for each target location in the target operation area. This can improve the convenience and simplicity of users' participation in the planning of drone inspection tasks and improve the efficiency of generating drone inspection tasks.

[0087] In an exemplary embodiment, acquiring the target user's interest location data matching the address description information of the target location from the interest location database after the user's interest location data is added may also include:

[0088] Step S51, acquiring candidate user interest location data matching the address description information of the target location from the interest location database after adding the user interest location data, and acquiring geographic coordinate information in the candidate user interest location data.

[0089] Among them, candidate user interest location data may refer to user interest location data that may match the address description information of the target location and is obtained through preliminary screening from the interest location database. For example, the address description information of the target location is Bridge A. The server matches multiple user interest location data with the address description information of Bridge A from the interest location database after adding the user interest location data, and uses these user interest location data with the address description information of Bridge A as candidate user interest location data. At the same time, the server obtains the geographic coordinate information in the candidate user interest location data.

[0090] Step S52, obtaining the current location information and flight range of the drone, and determining the candidate user interest location data that matches the flight range of the drone based on the current location information and the geographic coordinate information in the candidate user interest location data and using it as the target user interest location data.

[0091] The current location information of the drone may refer to the specific geographical location of the drone, and the current location information of the drone may be obtained through a positioning system. The flight range of the drone may refer to the maximum flight distance of the drone, that is, the farthest boundary that the drone can reach.

[0092] Exemplarily, the server obtains the current location information and flight range of the drone, and then can obtain the geographic distance based on the current location information of the drone and the geographic coordinate information in each candidate user interest location data, and then use the candidate user interest location data whose geographic distance is within the flight range of the drone as the target user interest location data; for example, the candidate user interest location data is user interest location data with multiple address description information of bridge A, and the specific locations corresponding to the user interest location data with different address description information of bridge A may be located in different operating areas; if the uniqueness of the address description information is controlled in the same area, the difficulty of generating user interest location data can be reduced, but it may cause the existence of specific locations with similar or identical address description information in multiple areas, such as area 1 contains a specific location with address description information of bridge A, area 2 also contains a specific location with address description information of bridge A, and area 3 also contains a specific location with address description information of bridge A. Based on the above situation, the server can determine the target bridge A among each bridge A whose distance from the drone meets the flight range of the drone based on the current location information of the drone and the specific locations of each bridge A, and then use the candidate user interest location data corresponding to the target bridge A as the target user interest location data.

[0093] In this embodiment, by intelligently filtering user interest location data, it is ensured that the interest location corresponding to the filtered target user interest location data not only semantically conforms to the user's semantic description of the target location, but also geographically meets the flight range of the drone, thereby improving the effectiveness of the drone inspection route.

[0094] In an exemplary embodiment, the method for generating a drone inspection route may further include the steps of:

[0095] Step S61, obtaining the inspection result of the drone, wherein the inspection result is obtained after the drone inspects the target operation area according to the inspection route.

[0096] Among them, the inspection result may refer to the data collected by the drone after completing the task according to the predetermined inspection route, and the predetermined inspection route refers to the inspection route of the drone in the target operation area generated by the interest location database after adding the user's interest location data.

[0097] Exemplarily, the drone performs inspections according to an inspection route in the target operation area, generates inspection results, and the server obtains the inspection results generated by the drone.

[0098] Step S62, obtain the geographic coordinate information in the inspection result, determine the address description information that matches the geographic coordinate information in the inspection result, associate the matched address description information with the corresponding geographic coordinate information, update the inspection result according to the associated information, and display the updated inspection result.

[0099] Among them, the geographic coordinate information in the inspection result may refer to the geographic coordinate information of the specific location involved in the inspection result. For example, the inspection result includes normal coordinate A, abnormal coordinate B, and abnormal coordinate C, that is, the geographic coordinate information in the inspection result is coordinate A, coordinate B, and coordinate C.

[0100] Exemplarily, the inspection results include "the building at coordinate A is damaged" and "the pond at coordinate B is dried up". The server obtains the geographic coordinate information from the inspection results to obtain coordinates A and coordinates B, and then determines that the address description information matching coordinate A is Zhang's house, and determines that the address description information matching coordinate B is Li's pond. The server then associates coordinate A with Zhang's house and coordinate B with Li's pond, and then updates the inspection results based on the above-mentioned association information to obtain updated inspection results, such as "the building at coordinate A is damaged (Zhang's house is damaged)" and "the pond at coordinate B is dried up (Li's pond is dried up)". The server can display the updated inspection results.

[0101] In this embodiment, by utilizing the exclusive information (address description information) in the user's interest location data, the report content can be displayed in a more intuitive and easy-to-understand form, allowing users to quickly understand the report content and enhance the effectiveness of delivering the report information of the inspection results.

[0102] In the context of today's widespread use of industrial drones in areas such as power inspections, forest fire patrols, and land surveys, these tasks often take place in remote areas that lack significant landmark features. Due to the scarcity of public geographic point of interest (LOI) data, in such an environment, it is difficult for operators to accurately control drones to perform specific tasks using only intuitive and easy-to-understand language instructions. In addition, when generating work reports, due to the limitations of existing technical conditions, it is usually only possible to rely on coordinates and image information to show the scope and results of the work to the decision-making level, and it is impossible to use public geocoding services to convert specific geographic locations into a more user-friendly and easy-to-understand form. Therefore, in an exemplary embodiment, if Figure 4 As shown, the present application also provides a drone inspection method lacking geocoding of interest locations, comprising:

[0103] Step S1, dynamic data addition.

[0104] Wherein, step S1 may specifically include:

[0105] Step S11, extracting information from the data collected by the drone. If the data is in the form of video, it needs to be converted into an image sequence; if the data is other payload data such as laser point cloud or side-looking radar, if it can be converted into an image, it is converted into an image and then processed.

[0106] Step S12, using a full-element segmentation model (pre-trained ground feature extraction model) to extract all elements from the image. Full-element extraction may refer to extracting all surface elements in the image, and the categories that may be extracted include cultivated land, forest land, grassland, buildings, roads, artificial bare land, natural bare land, water areas and other elements.

[0107] In step S13, the operator selects the object of interest (object element) and assigns a specific geographic location identifier (address description information) to it, thereby establishing a mapping relationship between the object and its actual location.

[0108] Step S14, expressing the outline of the feature based on the simple feature specification of OGC (Open Geospatial Consortium), and projecting the outline in the image coordinate system to the geographic coordinate system through a direct geolocation algorithm (or other algorithms that modify and optimize it), to ensure that each feature can find its precise corresponding point in the real world.

[0109] Step S15, generating AOI (Area Of Interest) data (user interest location data) based on the known polygonal geographic coordinates and their corresponding address information, and adding the data to the LOI (Location Of Interest) database.

[0110] like Figure 4 As shown, the overall process of step S1 is to first record a video of the feature through a drone, the server extracts the image of the feature from the video, and then segment the image to obtain the element target corresponding to each feature. Then the user selects the element target and enters the corresponding address (address description information, for example, Zhang San's house), sets this address as the attribute of the graphic corresponding to this element target, obtains AOI data, and then converts the AOI data into geographic coordinates through direct geolocation, and adds the AOI data and geographic coordinates to the LOI database.

[0111] Step S2, aircraft mission command.

[0112] Wherein, step S2 may specifically include:

[0113] Step S21: The operator inputs the address description information of the operation area.

[0114] In step S22, the system will structurally disassemble the address and search for the closest matching item (candidate user interest location data) in the LOI database through geocoding technology. Geocoding refers to forward geocoding and reverse geocoding. Forward geocoding can convert address description information into geographic coordinates (latitude and longitude), and reverse geocoding can convert geographic coordinates (latitude and longitude) into address description information.

[0115] Step S23, further filter the data according to the position (current position information) and flight radius (flight range) of the drone, and select the LOI data (target user interest location data) that meets the conditions.

[0116] Step S24, using this LOI data to create a mission objective, plan a mission route, and instruct the drone to perform a flight mission.

[0117] like Figure 4 As shown, the overall process of step S2 is that the user inputs the address description information, the server searches the LOI database according to the address description information, obtains candidate LOI data (LOIS), screens and scores the candidate LOI data according to the aircraft position and radius, obtains target LOI data (LOI), plans the route according to the target LOI data, and controls the drone according to the planned route.

[0118] In an exemplary embodiment, the above-mentioned drone inspection method lacking geocoding of the location of interest may specifically include:

[0119] Step S31, using a pre-set pod carried by the drone to collect data, and transmitting the data back to the server through the image transmission module.

[0120] Step S32, automatically screening out the most representative image samples according to preset standards such as overlap rate and image quality.

[0121] Step S33, using models including but not limited to the SAM (Segment Anything Model) image model (pre-trained ground feature extraction model) to extract all elements of the key frame, and also allows users to manually mark the target objects of interest.

[0122] In step S34, the operator confirms the selected element (geographical feature) and sets the address (address description information) in combination with the description (semantic description) of local personnel familiar with the area.

[0123] Step S35, the elements are directly geolocated into geographic elements that conform to OGC annotations, and are stored in a spatial database such as PostGIS in combination with address attributes (address description information).

[0124] Step S36, performing geocoding query from the spatial database using geocoders including but not limited to nominatim / pelias, and filtering the query results by distance.

[0125] Step S37, finally planning a route based on the query results and instructing the drone to perform operations.

[0126] With the help of geocoding technology, command operators can accurately control drones through natural language commands to perform tasks in key areas. However, in some scenarios of industrial drone applications, there are many remote areas or areas lacking detailed geographic data. The application of this method is limited due to the lack of public LOI data. Nevertheless, for those commanders who have lived in the local area for a long time or are very familiar with the place, each place has a unique and specific identifiable name (semantic description) - such as "Zhang San's fish pond". Therefore, we can collect these address description information rich in local characteristics provided by the command operators, and combine them with full-factor geographic feature extraction and direct geolocation technology to dynamically tailor an accurate spatial database for a specific area. In this way, even in the most remote or highly similar environments, an efficient and reliable drone geocoding application system can be built, which greatly broadens the application scope and flexibility of drone technology.

[0127] In the above-mentioned drone inspection method lacking geocoding of interest locations, there is a problem that the LOI data for the working area of ​​the drone is missing, and the relevant interactive operation area planning tasks cannot be carried out. This method can establish a unique geocoding database for the place of use, laying the foundation for solving the above problems. And when it is necessary to assign drones to perform specific target inspections, with the help of this database and reverse geocoding technology, the specific area to be inspected can be quickly located and identified, thereby achieving a high degree of automation and precision in the task allocation process. In addition, the system can also intelligently filter out those task points that are difficult for drones to reach due to geographical restrictions, ensuring that each instruction can be effectively executed. And in the stage of generating the final report, the exclusive information in the database can be used to make the report content more intuitive and easy to understand. For example, the expression "the building at coordinate A is damaged" is converted into "Zhang Moumou's house is damaged", so that all parties can easily understand the content of the report when using it, enhancing the effectiveness and affinity of information transmission.

[0128] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0129] Based on the same inventive concept, the embodiment of the present application also provides a drone inspection route generation device for implementing the drone inspection route generation method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more drone inspection route generation device embodiments provided below can refer to the limitations of the drone inspection route generation method above, and will not be repeated here.

[0130] In an exemplary embodiment, Figure 5 As shown, a device for generating a drone inspection route is provided, comprising: a ground feature image acquisition module 901, a feature extraction module 902, an address description acquisition module 903, a location data generation module 904 and an inspection route generation module 905, wherein:

[0131] A ground object image acquisition module, used for acquiring a ground object image to be processed in a target operation area when there is no public interest location data matching the target operation area in a pre-stored interest location database;

[0132] The feature extraction module is used to input the ground feature image to be processed into the pre-trained ground feature extraction model to obtain a ground feature feature set;

[0133] The address description acquisition module is used to obtain different address description information input by the user for different geographical features in the geographical feature set; the address description information is the semantic description of the geographical feature by the user;

[0134] A location data generation module, used to obtain user interest location data corresponding to the geographical features according to the address description information and geographic coordinate information corresponding to the geographical features, and add the user interest location data to the interest location database;

[0135] The inspection route generation module is used to generate the inspection route of the UAV in the target operation area by using the interest location database after adding the user's interest location data.

[0136] In an exemplary embodiment, the above-mentioned location data generation module is also used to obtain the normalized feature contour corresponding to the feature element based on the spatial information expression standard; project the normalized feature contour into the geographic coordinate system of the geographic information system to obtain the polygonal geographic coordinates of the normalized feature contour, and use the polygonal geographic coordinates as the geographic coordinate information corresponding to the feature element.

[0137] In an exemplary embodiment, the above-mentioned location data generation module is also used to perform image matching between the geographical features in the geographical feature set and the map tiles in the geographic information system, and determine the geographical coordinate information corresponding to the geographical features according to the matched map tiles.

[0138] In an exemplary embodiment, the above-mentioned inspection route generation module is also used to obtain address description information for each target location in the target operation area; obtain target user interest location data that matches the address description information of the target location from the interest location database after adding the user interest location data, and generate an inspection route for the drone in the target operation area based on the target user interest location data.

[0139] In an exemplary embodiment, the above-mentioned inspection route generation module is also used to obtain candidate user interest location data that matches the address description information of the target location from the interest location database after adding the user interest location data, and obtain the geographic coordinate information in the candidate user interest location data; obtain the current location information and flight range of the drone, and determine the candidate user interest location data that matches the flight range of the drone based on the current location information and the geographic coordinate information in the candidate user interest location data and use it as the target user interest location data.

[0140] In an exemplary embodiment, the above-mentioned drone inspection route generation device also includes an inspection result update module, which is used to obtain the inspection results of the drone, and the inspection results are obtained after the drone inspects the target operation area according to the inspection route; obtain the geographic coordinate information in the inspection results, determine the address description information that matches the geographic coordinate information in the inspection results, associate the matched address description information with the corresponding geographic coordinate information, update the inspection results according to the associated information, and display the updated inspection results.

[0141] Each module in the above-mentioned drone inspection route generation device can be implemented in whole or in part by software, hardware, or a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0142] In an exemplary embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. Figure 6 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for generating a drone inspection route is implemented.

[0143] Those skilled in the art will understand that Figure 6The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0144] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.

[0145] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0146] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0147] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0148] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0149] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0150] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for generating a drone inspection route, characterized in that: The method comprises: When there is no public interest location data matching the target operation area in the pre-stored interest location database, obtaining a to-be-processed ground feature image of the target operation area; Inputting the to-be-processed ground object image into a pre-trained ground object element extraction model to obtain a ground object element set; Acquire different address description information input by the user for different geographical features in the geographical feature set; the address description information is the semantic description of the geographical feature by the user; According to the address description information and geographic coordinate information corresponding to each geographical feature, obtaining the user's interest location data corresponding to the geographical feature, and adding the user's interest location data to the interest location database; The inspection route of the UAV in the target operation area is generated by using the interest location database after adding the user's interest location data.

2. The method according to claim 1, characterized in that The steps for obtaining the geographic coordinate information corresponding to each feature include: Based on the spatial information expression standard, obtaining the normalized feature contour corresponding to the feature element; The normalized feature outline is projected into the geographic coordinate system of the geographic information system to obtain the polygonal geographic coordinates of the normalized feature outline, and the polygonal geographic coordinates are used as the geographic coordinate information corresponding to the feature element.

3. The method according to claim 1, characterized in that The steps for obtaining the geographic coordinate information corresponding to each feature include: The geographical features in the geographical feature set are image matched with the map tiles in the geographic information system, and the geographical coordinate information corresponding to the geographical features is determined according to the matched map tiles.

4. The method according to claim 1, characterized in that: The method of generating a patrol route of a drone in a target operation area by using the interest location database after adding the user's interest location data includes: Obtaining address description information for each target location in the target operation area; The target user interest location data matching the address description information of the target location is obtained from the interest location database after the user interest location data is added, and an inspection route of the drone in the target operation area is generated according to the target user interest location data.

5. The method according to claim 4, characterized in that The step of acquiring the target user's interest location data that matches the address description information of the target location from the interest location database after adding the user's interest location data includes: Acquire candidate user interest location data matching the address description information of the target location from the interest location database after adding the user interest location data, and acquire geographic coordinate information in the candidate user interest location data; The current location information and flight range of the drone are obtained, and based on the current location information and the geographic coordinate information in the candidate user interest location data, the candidate user interest location data that matches the flight range of the drone is determined and used as the target user interest location data.

6. The method according to claim 1, characterized in that The method further comprises: Obtaining an inspection result of the drone, the inspection result being obtained after the drone inspects the target operation area according to an inspection route; Obtain the geographic coordinate information in the inspection result, determine the address description information that matches the geographic coordinate information in the inspection result, associate the matched address description information with the corresponding geographic coordinate information, update the inspection result according to the associated information, and display the updated inspection result.

7. A drone inspection route generation device, characterized in that: The device comprises: A ground object image acquisition module, used for acquiring a ground object image to be processed in the target operation area when there is no public interest location data matching the target operation area in the pre-stored interest location database; An element extraction module is used to input the ground object image to be processed into a pre-trained ground object element extraction model to obtain a ground object element set; An address description acquisition module is used to acquire different address description information input by a user for different geographical features in the geographical feature set; the address description information is a semantic description of the geographical feature by the user; A location data generation module, used to obtain user interest location data corresponding to the geographical features according to the address description information and geographical coordinate information corresponding to the geographical features, and add the user interest location data to the interest location database; The inspection route generation module is used to generate an inspection route for the UAV in the target operation area by using the interest location database after adding the user's interest location data.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images