Emergency material air transportation method based on road network characteristics for drones

Abstract

The embodiments of the present application relate to the field of drone technology, and in particular to a method for drone-based emergency material air transportation based on road network features, comprising: after the emergency drone takes off, instructing the emergency drone to fly toward a target service area according to a preset route, photographing the highway on the ground in real time through the optoelectronic pod system carried by the emergency drone itself, and obtaining surface image data; extracting features from the surface image data using a classification-based image segmentation method and a feature-based image segmentation method, respectively, to obtain multiple road features; optimizing and integrating the multiple road features to obtain comprehensive road features, forming an overall planned route guided along the highway, and instructing the emergency drone to fly at a constant altitude and speed along the highway toward the target service area according to the overall planned route; when the emergency drone arrives over the target service area, switching to the emergency drone autonomous landing mode, thereby realizing effective unmanned delivery of emergency materials.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of drone technology, and in particular to a method for air transporting emergency supplies based on road network characteristics for drones. Background Art

[0002] Drone logistics and material delivery are gradually becoming a new trend in the logistics industry, providing a new solution to the pain points of the traditional logistics industry. Drone logistics has the following advantages: First, drone logistics is efficient and fast, which greatly shortens the delivery time and improves the coverage of logistics; second, drone logistics can reduce labor costs and improve corporate competitiveness; third, drone logistics helps reduce traffic congestion, improve transportation efficiency, and reduce the cost of infrastructure construction and maintenance; finally, drone logistics is also green and environmentally friendly, which is more in line with the concept of sustainable and low-carbon development.

[0003] However, there are still many problems to be solved before drone logistics transportation can go from theory to popularization. For example, once a large-load drone crashes, it may cause a serious safety accident. Therefore, most drone logistics transportation currently chooses to conduct pilot applications in sparsely populated areas or airspace where there are no human movements on the ground. The flight navigation of drone logistics transportation mainly relies on satellite positioning. Some drones fly autonomously according to preset tracks, but they also rely on satellite positioning for navigation. In the event of emergencies, such as limited satellite positioning or failure of preset routes, drones have no corresponding emergency response strategies and can only rely on manual takeover of manual flight. However, for large-load drones, manual flight control is difficult and poses a great safety hazard. Summary of the invention

[0004] In view of this, the embodiments of the present application propose an emergency material air transportation method for drones based on road network characteristics, aiming to free drone logistics transportation from complete dependence on satellite positioning, and to still be able to safely fly to the target location and land safely in the event of an emergency, thereby realizing the safe unmanned delivery of emergency materials.

[0005] In order to achieve the above-mentioned purpose, an embodiment of the present application proposes a method for air transportation of emergency materials based on road network features for drones, the method comprising: when the emergency drone takes off from the current service area carrying emergency materials, instructing the flight control system of the emergency drone to control the emergency drone to fly toward the target service area according to a preset route, and at the same time, the optoelectronic pod system carried by the emergency drone itself is used to photograph the highway on the ground in real time to obtain surface image data; the surface image data obtained by the photography are respectively extracted using a classification-based image segmentation method and a feature-based image segmentation method to obtain multiple road features; the multiple road features are optimized and integrated to obtain comprehensive road features, and a general planned route guided along the highway is formed based on the comprehensive road features, and the flight control system of the emergency drone is instructed to control the emergency drone to fly at a constant altitude and speed along the highway toward the target service area according to the general planned route; when the emergency drone arrives over the target service area, it is switched to the autonomous landing mode of the emergency drone, and after the emergency drone lands on the ground and stabilizes, the ground personnel are notified to unload the emergency materials.

[0006] In order to achieve the above-mentioned purpose, the embodiment of the present application also proposes an emergency material air transport system for drones based on road network features, the system comprising: an initial guidance module, an acquisition module, a feature extraction module, a route drawing module, a real-time guidance module and a landing switching module; the initial guidance module is used to instruct the flight control system of the emergency drone to control the emergency drone to fly toward the target service area according to a preset route when the emergency drone takes off from the current service area carrying emergency materials; the acquisition module is used to obtain surface image data by photographing the highway on the ground in real time through the optoelectronic pod system carried by the emergency drone itself; the feature extraction module is used to extract the surface image data obtained by the photographing. Feature extraction is performed using a classification-based image segmentation method and a feature-based image segmentation method to obtain multiple road features; a route drawing module is used to optimize and integrate multiple road features to obtain comprehensive road features, and form a general planning route along the highway based on the comprehensive road features; a real-time guidance module is used to instruct the flight control system of the emergency UAV to control the emergency UAV to fly at a constant altitude and speed along the highway toward the target service area according to the overall planned route; a landing switching module is used to switch to the emergency UAV autonomous landing mode when the emergency UAV arrives over the target service area, wait for the emergency UAV to land on the ground and stabilize, and then notify ground personnel to unload emergency supplies.

[0007] In order to achieve the above-mentioned purpose, an embodiment of the present application also proposes an electronic device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute a method for air transporting emergency materials based on road network features for drones as described above.

[0008] To achieve the above-mentioned purpose, an embodiment of the present application also provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it can implement the above-mentioned emergency material air transportation method for drones based on road network characteristics.

[0009] The embodiment of the present application proposes a method for air transporting emergency materials based on road network features for drones. After the emergency drone takes off with emergency materials, the emergency drone is first instructed to fly toward the target service area according to a preset route. At the same time, the optoelectronic pod system carried by the emergency drone itself takes real-time photos of the highway on the ground to obtain surface image data. The surface image data obtained by shooting are respectively subjected to feature extraction in multiple ways to obtain multiple road features, and the multiple road features are optimized and integrated to obtain comprehensive road features. Finally, based on the comprehensive road features, a master planned route guided along the highway is formed, and the emergency drone is instructed to fly at a constant speed along the highway toward the target service area according to the master planned route. The formulation of the master planned route helps the emergency drone get rid of its complete dependence on satellite positioning, and can assist the emergency drone to continue flying along the highway when the satellite positioning fails or the preset route has problems, greatly reducing the risk of crashing caused by the emergency drone encountering emergencies and reducing economic losses. The emergency drone flies at a constant speed along the highway toward the target service area, so that the emergency drone maintains a relatively stable and safe distance from the highway, outdoor high-voltage circuits, mobile signal towers, etc., until it reaches the target service area and lands autonomously, which well realizes the safe unmanned delivery of emergency materials.

[0010] In some optional embodiments, the classification-based image segmentation method includes a watershed algorithm, a K-means clustering algorithm, and a support vector machine algorithm, and the surface image data obtained by shooting is subjected to feature extraction using the classification-based image segmentation method, including: using the watershed algorithm to perform image segmentation on the surface image data, taking the positions of strong edges and large gradients in the surface image data as the edges of the highway, and obtaining the first road feature; using the K-means clustering algorithm to perform image segmentation on the surface image data, clustering and grouping according to the RGB color space of each pixel point in the surface image data, and obtaining the second road feature based on the grouping corresponding to the highway; using the support vector machine algorithm to perform image segmentation on the surface image data, determining the discrimination surface between the highway pixel points and the non-highway pixel points, and obtaining the third road feature based on the discrimination surface. The watershed algorithm, the K-means clustering algorithm, and the support vector machine algorithm can perform classification-based image segmentation on the surface image data from three different aspects, providing rich basic information for the subsequent formulation of the overall planning route.

[0011] In some optional embodiments, the feature-based image segmentation method includes edge detection, morphological analysis, and texture analysis, and the feature-based image segmentation method is used to extract features from the captured surface image data, including: edge detection on the surface image data, based on the color difference, brightness difference and contrast difference in the surface image data, the edge of the highway is determined to obtain the fourth road feature; morphological analysis including expansion operation and erosion operation is performed on the surface image data to extract the long strip shape and straight line structure to obtain the fifth road feature; texture analysis is performed on the surface image data to detect the texture difference between the highway and the surrounding background, and the texture feature of the highway is extracted as the sixth road feature. Edge detection, morphological analysis and texture analysis can perform feature-based image segmentation on the surface image data from three different aspects, providing rich basic information for the subsequent formulation of the overall planning route.

[0012] In some optional embodiments, multiple road features are optimized and integrated to obtain a comprehensive road feature, including: assigning a first comprehensive weight and a second comprehensive weight to a classification-based image segmentation method and a feature-based image segmentation method, respectively, according to the current season, the current weather, the geographical location of the current service area, the geographical location of the target service area, and the real-time traffic information of the highway between the current service area and the target service area, and assigning a first confidence weight, a second confidence weight, a third confidence weight, a fourth confidence weight, a fifth confidence weight, and a sixth confidence weight to the first road feature, the second road feature, the third road feature, the fourth road feature, the fifth road feature, and the sixth road feature, respectively; optimizing and weightedly splicing and integrating the first road feature, the second road feature, the third road feature, the fourth road feature, the fifth road feature, and the sixth road feature based on the first comprehensive weight, the second comprehensive weight, the first confidence weight, the second confidence weight, the third confidence weight, the fourth confidence weight, the fifth confidence weight, and the sixth confidence weight to obtain a comprehensive road feature. For different seasons, weather, and geographical locations, the quality of road features extracted by different feature extraction methods varies. Therefore, this application assigns a first comprehensive weight and a second comprehensive weight to the classification-based image segmentation method and the feature-based image segmentation method, respectively, according to the current season, current weather, the geographical location of the current service area, the geographical location of the target service area, and the real-time traffic information of the highway between the current service area and the target service area, and assigns confidence weights to the six road features, to ensure that high-quality road features are given priority consideration, improve the scientific nature of the comprehensive road features, and thereby improve the quality of the overall planned route.

[0013] In some optional embodiments, based on the first comprehensive weight, the second comprehensive weight, the first confidence weight, the second confidence weight, the third confidence weight, the fourth confidence weight, the fifth confidence weight and the sixth confidence weight, the first road feature, the second road feature, the third road feature, the fourth road feature, the fifth road feature and the sixth road feature are optimized and weightedly spliced ​​to obtain a comprehensive road feature, which is implemented by the following formula:

[0014] ;

[0015] ;

[0016] ;

[0017] in, represents the first comprehensive weight, represents the second comprehensive weight, represents the first confidence weight, represents the second confidence weight, represents the third confidence weight, represents the fourth confidence weight, represents the fifth confidence weight, represents the sixth confidence weight, represents the first road feature, represents the second road feature, Indicates the third road feature, represents the fourth road feature, Indicates the fifth road feature, represents the sixth road feature, Indicates a splicing operation. Represents the comprehensive characteristics of the road.

[0018] In some optional embodiments, a master planned route guided along the highway is formed based on the comprehensive road features, and the flight control system of the emergency drone is instructed to control the emergency drone to fly at a constant altitude and speed along the highway toward the target service area according to the master planned route, including: inputting the comprehensive road features into the pre-trained recognition model to obtain the highway data output by the recognition model; adding m meters of width data, setting n meters of flight altitude data, and setting s kilometers per hour of flight speed data on the basis of the highway data output by the recognition model to form a master planned route guided along the highway; sending the master planned route to the flight control system of the emergency drone, instructing the flight control system of the emergency drone to control the emergency drone according to the master planned route, along the outer side of the highway m meters, according to the flight altitude of n meters, and the flight speed of s kilometers per hour, and to fly at a constant altitude and speed to the target service area. The use of the recognition model can greatly improve the recognition speed of the highway, and well ensure the real-time nature of the master planned route formulation.

[0019] In some optional embodiments, after obtaining the highway data output by the recognition model, the method further includes: comparing the highway data output by the recognition model with the standard highway data; if the deviation between the highway data output by the recognition model and the standard highway data does not meet the preset acceptable range, or the recognition model cannot recognize the highway data, then instructing the flight control system of the emergency drone to control the emergency drone to hover and wait for receiving the next instruction; judging whether the conditions for continuing to fly are met based on the surface image data; if it is judged that the conditions for continuing to fly are not met, sending a return instruction to the flight control system of the emergency drone, instructing the flight control system of the emergency drone to control the emergency drone to return; if it is judged that the conditions for continuing to fly are met, then formulating and sending a temporary route to the flight control system of the emergency drone, instructing the flight control system of the emergency drone to control the emergency drone to continue flying according to the temporary route, until the deviation between the highway data output by the recognition model and the standard highway data meets the preset acceptable range. When the highway is damaged by force majeure, the comprehensive characteristics of the road will no longer be accurate, the recognition model will not be able to accurately identify the highway, and the formulated overall planning route will no longer be reliable. If the emergency drone continues to fly, it will face great risks. Therefore, before formulating the overall planned route, this application needs to compare the highway data output by the identification model with the standard highway data and determine whether the conditions for continued flight are met. If the conditions for continued flight are met, a temporary route is formulated for risk avoidance. If the conditions for continued flight are not met, the flight returns directly, thereby further ensuring the safe transportation of emergency drones and emergency supplies. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the related technologies, the drawings required for use in the embodiments of the present application or the related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0021] Figure 1 It is a flow chart of a method for air transporting emergency materials based on road network characteristics for drones provided in one embodiment of the present application;

[0022] Figure 2 It is a schematic diagram of an emergency drone provided in an embodiment of the present application flying at a constant altitude and speed along a highway toward a target service area according to a generally planned route;

[0023] Figure 3 is a schematic diagram of the track of an emergency drone provided in one embodiment of the present application;

[0024] Figure 4is a structural schematic diagram of an emergency material air transport system based on road network characteristics for drones provided by another embodiment of the present application;

[0025] Figure 5 It is a structural schematic diagram of an electronic device provided in another embodiment of the present application. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the embodiments of the present application will be described in detail below in conjunction with the accompanying drawings. It will be appreciated by those skilled in the art that in the embodiments of the present application, many technical details are proposed in order to enable the reader to better understand the present application. However, even without these technical details and various changes and modifications based on the following embodiments, the technical scheme claimed in the present application can also be implemented. The division of the following embodiments is for the convenience of description, and the specific implementation of the present application should not be construed as any limitation, and the various embodiments can be combined and referenced with each other under the premise of no contradiction.

[0027] An embodiment of the present application proposes an emergency material air transportation method based on road network characteristics for drones, which is applied to electronic devices, wherein the electronic device can be a terminal or a server. This embodiment and the following embodiments are all described using the server as an example. The implementation details of the emergency material air transportation method based on road network characteristics for drones proposed in this embodiment are specifically described below. The following content is only the implementation details provided for ease of understanding and is not necessary for the implementation of this solution.

[0028] The specific process of the emergency material air transportation method based on road network characteristics for drones proposed in this embodiment can be as follows: Figure 1 As shown, including:

[0029] Step 101, when the emergency drone takes off from the current service area with emergency supplies, the flight control system of the emergency drone is instructed to control the emergency drone to fly toward the target service area according to a preset route. At the same time, the optoelectronic pod system carried by the emergency drone itself takes real-time photos of the highway on the ground to obtain surface image data.

[0030] In the specific implementation, based on the power and safety considerations of the emergency drone, the emergency drone can use the service area as a transfer point in the air transportation of emergency supplies, and carry out emergency material loading and unloading, battery replacement and whole machine maintenance in the service area. When the emergency drone takes off from the current service area with emergency supplies, the server needs to instruct the flight control system (also called the flight control system) of the emergency drone to control the emergency drone to fly toward the target service area according to the preset route. At the same time, the optoelectronic pod system carried by the emergency drone itself takes real-time photos of the highway on the ground to obtain surface image data. Among them, the preset route can be formulated by technicians in this field based on the highway between the current service area and the target service area. If the emergency drone always flies according to the preset route, real-time satellite positioning is required to prevent deviation.

[0031] It is understandable that satellite positioning is likely to fail in the disaster-stricken areas. Once satellite positioning fails, the emergency drone will not be able to detect whether it has deviated. Therefore, the server needs to use the optoelectronic pod system carried by the emergency drone itself to take real-time photos of the highway on the ground to obtain surface image data, thereby exploring a flight method that does not rely entirely on satellite positioning.

[0032] It is worth noting that the visual range of the emergency drone is limited, so the server needs to continuously shoot the ground highway through the optoelectronic pod system carried by the emergency drone itself during the entire flight of the emergency drone.

[0033] Step 102 : extracting features from the captured surface image data using a classification-based image segmentation method and a feature-based image segmentation method to obtain a plurality of road features.

[0034] In a specific implementation, after obtaining the surface image data, the server can perform feature extraction on the captured surface image data using a classification-based image segmentation method and a feature-based image segmentation method, respectively, and describe the features of the surface image data from different aspects and methods, thereby obtaining multiple road features.

[0035] In one example, the classification-based image segmentation method includes a watershed algorithm, a K-means clustering algorithm, and a support vector machine algorithm. The server will use the watershed algorithm, the K-means clustering algorithm, and the support vector machine algorithm in sequence to perform image segmentation on the surface image data.

[0036] The watershed algorithm is an image segmentation algorithm based on the principle of mathematical morphology. Its basic idea is to regard the image as a landform, where the peaks represent the target and the valleys represent the background. The image is divided into multiple parts by calculating the water flow path and flow rate. When the server uses the watershed algorithm to segment the surface image data, it needs to take the locations with strong edges and large gradients (the judgment condition can be set as the gradient is greater than the preset gradient threshold) in the surface image data as the edges of the highway, thereby obtaining the first road feature.

[0037] The K-means clustering algorithm is a clustering-based image segmentation algorithm that uses the similarity between pixels to perform clustering and assigns a centroid to each cluster. When the server uses the K-means clustering algorithm to segment surface image data, it needs to perform two clustering groups (K=2) based on the RGB color space of each pixel in the surface image data, and obtain the second road feature based on the grouping corresponding to the highway.

[0038] The support vector machine algorithm is a commonly used classification algorithm. Its core idea is to find a maximum decision surface to classify pixels into different categories. When the server uses the support vector machine algorithm to segment surface image data, it needs to determine the decision surface between highway pixels and non-highway pixels, and obtain the third road feature based on the decision surface between highway pixels and non-highway pixels.

[0039] The watershed algorithm, K-means clustering algorithm and support vector machine algorithm can perform classification-based image segmentation on surface image data from three different aspects, providing rich basic information for the subsequent formulation of the overall planning route.

[0040] In one example, the feature-based image segmentation method includes edge detection, morphological analysis, and texture analysis, and the server will perform edge detection, morphological analysis, and texture analysis on the surface image data in sequence.

[0041] Edge detection is a feature extraction method based on image gradients. Its basic idea is to find locations in an image where pixel values ​​vary greatly. When the server performs edge detection on surface image data, it needs to determine the edge of the highway based on the color difference, brightness difference, and contrast difference in the surface image data to obtain the fourth road feature.

[0042] Morphological analysis is a structure-based feature extraction method. Its basic idea is to extract areas with specific shapes and structures by dilating and corroding the image. The server needs to perform morphological analysis on the surface image data, including dilation and corrosion operations, to extract long strip shapes (i.e. rectangular areas with an aspect ratio greater than a preset aspect ratio threshold) and straight line structures to obtain the fifth road feature.

[0043] Texture analysis is a texture-based feature extraction method. Its basic idea is to distinguish different areas by analyzing the texture features of the image. When the server performs texture analysis on the surface image data, it needs to detect the texture difference between the highway and the surrounding background and extract the texture features of the highway as the sixth road feature.

[0044] Edge detection, morphological analysis and texture analysis can perform feature-based image segmentation on surface image data from three different aspects, providing rich basic information for the subsequent formulation of overall planning routes.

[0045] Step 103, optimize and integrate multiple road features to obtain comprehensive road features, form a master planned route along the highway based on the comprehensive road features, and instruct the flight control system of the emergency UAV to control the emergency UAV to fly at a constant altitude and speed along the highway toward the target service area according to the master planned route.

[0046] In a specific implementation, after extracting multiple road features, the server can optimize and integrate the multiple road features to obtain comprehensive road features, form a master planned route along the highway based on the comprehensive road features, and instruct the flight control system of the emergency UAV to control the emergency UAV to fly at a constant altitude and speed along the highway toward the target service area according to the master planned route.

[0047] In one example, the server needs to allocate a first comprehensive weight and a second comprehensive weight to the classification-based image segmentation method and the feature-based image segmentation method respectively according to the current season, the current weather, the geographical location of the current service area, the geographical location of the target service area, and the real-time traffic information of the highway between the current service area and the target service area, and allocate a first confidence weight, a second confidence weight, a third confidence weight, a fourth confidence weight, a fifth confidence weight, and a sixth confidence weight to the first road feature, the second road feature, the third road feature, the fourth road feature, the fifth road feature, and the sixth road feature respectively. Subsequently, based on the first comprehensive weight, the second comprehensive weight, the first confidence weight, the second confidence weight, the third confidence weight, the fourth confidence weight, the fifth confidence weight, and the sixth confidence weight, the first road feature, the second road feature, the third road feature, the fourth road feature, the fifth road feature, and the sixth road feature are optimized and weightedly spliced ​​to obtain a comprehensive road feature.

[0048] It is understandable that for different seasons, weather, and geographical locations, the quality of road features extracted by different feature extraction methods varies. For example, in the northern region in winter, when it snows, the sixth road feature (texture feature) is very unclear, and the first road feature (watershed feature) is less affected. A higher confidence weight should be assigned to the first road feature, and a lower confidence weight should be assigned to the sixth road feature. For another example, when there is severe congestion on the highway, the second road feature (K-means clustering feature) is no longer reliable, and the fourth road feature (edge ​​feature) is less affected. A higher confidence weight should be assigned to the fourth road feature, and a lower confidence weight should be assigned to the second road feature. Therefore, the server needs to assign the first comprehensive weight and the second comprehensive weight to the classification-based image segmentation method and the feature-based image segmentation method respectively according to the current season, current weather, geographical location of the current service area, geographical location of the target service area, and real-time traffic information of the highway between the current service area and the target service area, and assign confidence weights to the six road features respectively, to ensure that high-quality road features are given priority consideration, improve the scientific nature of the comprehensive road features, and thus improve the quality of the overall planned route.

[0049] In one example, the server optimizes and weightedly splices and synthesizes the first road feature, the second road feature, the third road feature, the fourth road feature, the fifth road feature, and the sixth road feature based on the first comprehensive weight, the second comprehensive weight, the first confidence weight, the second confidence weight, the third confidence weight, the fourth confidence weight, the fifth confidence weight, and the sixth confidence weight to obtain a comprehensive road feature, which can be achieved by the following formula:

[0050] ;

[0051] ;

[0052] ;

[0053] in, represents the first comprehensive weight, represents the second comprehensive weight, represents the first confidence weight, represents the second confidence weight, represents the third confidence weight, represents the fourth confidence weight, represents the fifth confidence weight, represents the sixth confidence weight, represents the first road feature, represents the second road feature, Indicates the third road feature, represents the fourth road feature, Indicates the fifth road feature, represents the sixth road feature, Indicates a splicing operation. Represents the comprehensive characteristics of the road.

[0054] In one example, when the server forms a master planned route guided along the highway based on the comprehensive road features and instructs the flight control system of the emergency drone to control the emergency drone to fly at a constant speed along the highway toward the target service area according to the master planned route, it is necessary to input the comprehensive road features into the pre-trained recognition model to obtain the highway data output by the recognition model. Subsequently, based on the highway data output by the recognition model, the width data of m meters is added, the flight altitude data of n meters is set, and the flight speed data of s kilometers per hour is set to form a master planned route guided along the highway. Finally, the master planned route is sent to the flight control system of the emergency drone, instructing the flight control system of the emergency drone to control the emergency drone according to the master planned route, along the outer side of the highway m meters, according to the flight altitude of n meters, and the flight speed of s kilometers per hour, and to fly at a constant speed to the target service area. Among them, m is an integer greater than 5, which can generally be set to 10, n is an integer greater than 100, which can generally be set to 200, and s is an integer greater than 50, which can generally be set to 60. That is to say, the server can instruct the flight control system of the emergency drone to control the emergency drone to fly at a constant altitude and speed to the target service area along the overall planned route, 10 meters outside the highway, at a flight altitude of 200 meters and a flight speed of 60 kilometers per hour.

[0055] It is worth noting that the use of recognition models can greatly improve the recognition speed of highways, and well ensure the real-time nature of the overall planning route. The recognition model can directly use an open source model, or it can be selected from a recognition model trained by technicians based on a large number of highway sample images.

[0056] In one example, the process of an emergency drone flying at a constant altitude and speed along a highway toward a target service area according to a master planned route can be as follows: Figure 2 As shown in Figure 2, the trajectory of the emergency drone can be Figure 3 shown.

[0057] In one example, if the emergency drone has sufficient computing power and sufficient power reserves, the overall planned route can be directly formulated by the emergency drone's flight control system.

[0058] In one example, after obtaining the highway data output by the recognition model, the server also needs to compare the highway data output by the recognition model with the standard highway data. If the deviation between the highway data output by the recognition model and the standard highway data meets the preset acceptable range, the overall planning route can be formulated according to the normal process. If the deviation between the highway data output by the recognition model and the standard highway data does not meet the preset acceptable range, or the recognition model cannot recognize the highway data, it is necessary to instruct the flight control system of the emergency drone to control the emergency drone to hover and wait for the next instruction. Subsequently, the server determines whether the conditions for continuing flight are met based on the surface image data. If it is determined that the conditions for continuing flight are not met, a return instruction is sent to the flight control system of the emergency drone, instructing the flight control system of the emergency drone to control the emergency drone to return. If it is determined that the conditions for continuing flight are met, a temporary route is formulated and sent to the flight control system of the emergency drone, instructing the flight control system of the emergency drone to control the emergency drone to continue flying according to the temporary route until the deviation between the highway data output by the recognition model and the standard highway data meets the preset acceptable range.

[0059] When a highway is damaged by force majeure, the comprehensive characteristics of the road will no longer be accurate, the recognition model will not be able to accurately identify the highway, and the master planned route will no longer be reliable. If the emergency drone continues to fly, it will face great risks. Therefore, before formulating the master planned route, this application needs to compare the highway data output by the recognition model with the standard highway data and determine whether the conditions for continued flight are met. If the conditions for continued flight are met, a temporary route will be formulated for risk avoidance. If the conditions for continued flight are not met, it will return directly, thereby further ensuring the safe transportation of emergency drones and emergency supplies.

[0060] Step 104, when the emergency drone arrives above the target service area, it switches to the emergency drone autonomous landing mode, waits for the emergency drone to land on the ground and stabilize, and then notifies ground personnel to unload emergency supplies.

[0061] In the specific implementation, when the emergency drone arrives above the target service area, it can switch to the emergency drone autonomous landing mode, and the flight control system of the emergency drone will control the emergency drone to land autonomously. After the emergency drone lands on the ground and stabilizes, the ground personnel will be notified to unload the emergency supplies, replace the battery for the emergency drone, and perform a complete machine inspection. If there is a follow-up task, the emergency drone will be arranged to return or fly to the next target service area.

[0062] This embodiment proposes a method for air transporting emergency materials based on road network features for drones. After the emergency drone takes off with emergency materials, the emergency drone is first instructed to fly toward the target service area according to a preset route. At the same time, the optoelectronic pod system carried by the emergency drone itself takes real-time photos of the highway on the ground to obtain surface image data. The surface image data obtained by shooting are respectively subjected to feature extraction in multiple ways to obtain multiple road features, and the multiple road features are optimized and integrated to obtain comprehensive road features. Finally, based on the comprehensive road features, a master planned route guided along the highway is formed, and the emergency drone is instructed to fly at a constant speed along the highway toward the target service area according to the master planned route. The formulation of the master planned route helps the emergency drone get rid of its complete dependence on satellite positioning, and can assist the emergency drone to continue flying along the highway when satellite positioning fails or the preset route has problems, greatly reducing the risk of crashing caused by the emergency drone encountering emergencies and reducing economic losses. The emergency drone flies at a constant speed along the highway toward the target service area, so that the emergency drone maintains a relatively stable and safe distance from the highway, outdoor high-voltage circuits, mobile signal towers, etc., until it reaches the target service area and lands autonomously, thus realizing the safe unmanned delivery of emergency materials.

[0063] The above division of steps of various methods is only for clear description. When implemented, they can be combined into one step or some steps can be split and decomposed into multiple steps. As long as they include the same logical relationship, they are all within the scope of protection of this application; adding insignificant modifications to the algorithm or process or introducing insignificant designs without changing the core design of the algorithm and process are all within the scope of protection of this application.

[0064] Correspondingly, another embodiment of the present application proposes an emergency material air transport system based on road network characteristics for drones. The details of the emergency material air transport system based on road network characteristics for drones proposed in this embodiment are specifically described below. The following content is only the implementation details provided for the convenience of understanding and is not necessary for the implementation of this example. Figure 4 It is a structural diagram of an emergency material air transport system for drones based on road network features proposed in this embodiment. The emergency material air transport system 200 includes: an initial guidance module 201, an acquisition module 202, a feature extraction module 203, a route drawing module 204, a real-time guidance module 205 and a landing switching module 206.

[0065] The initial guidance module 201 is used to instruct the flight control system 301 of the emergency drone 300 to control the emergency drone 300 to fly toward the target service area according to a preset route after the emergency drone 300 takes off from the current service area carrying emergency supplies.

[0066] The acquisition module 202 is used to photograph the highway on the ground in real time through the optoelectronic pod system 302 carried by the emergency drone 300 itself to obtain surface image data.

[0067] The feature extraction module 203 is used to extract features from the captured surface image data using a classification-based image segmentation method and a feature-based image segmentation method to obtain a plurality of road features.

[0068] The route drawing module 204 is used to optimize and integrate multiple road features to obtain comprehensive road features, and form a general planning route guided along the highway based on the comprehensive road features.

[0069] The real-time guidance module 205 is used to instruct the flight control system 301 of the emergency UAV 300 to control the emergency UAV 300 to fly at a constant altitude and speed along the highway toward the target service area according to the overall planned route.

[0070] The landing switching module 206 is used to switch to the emergency drone autonomous landing mode when the emergency drone 300 arrives above the target service area, wait for the emergency drone 300 to land on the ground and stabilize, and then notify the ground personnel to unload the emergency supplies.

[0071] It is not difficult to find that this embodiment is a system embodiment corresponding to the above method embodiment, and this embodiment can be implemented in conjunction with the above method embodiment. The relevant technical details and technical effects mentioned in the above method embodiment are still valid in this embodiment, and in order to reduce repetition, they are not repeated here. Accordingly, the relevant technical details mentioned in this embodiment can also be applied in the above method embodiment.

[0072] It is worth mentioning that all modules involved in this embodiment are logic modules. In practical applications, a logic unit can be a physical unit, a part of a physical unit, or a combination of multiple physical units. In addition, in order to highlight the innovative part of this application, this embodiment does not introduce units that are not closely related to solving the technical problems proposed by this application, but this does not mean that there are no other units in this embodiment.

[0073] Another embodiment of the present application provides an electronic device, the structure of which can be as follows: Figure 5 As shown, it includes: at least one processor 401; and a memory 402 that is communicatively connected to the at least one processor 401; wherein the memory 402 stores instructions that can be executed by the at least one processor 401, and the instructions are executed by the at least one processor 401 so that the at least one processor 401 can execute a method for air transportation of emergency materials based on road network characteristics for drones as described in the above-mentioned method embodiments.

[0074] Among them, the memory and the processor can be connected in a bus manner, and the bus can include any number of interconnected buses and bridges, and the bus connects various circuits of one or more processors and memories together. The bus can also connect various other circuits such as peripheral devices, voltage regulators and power management circuits together, which are all well known in the art, so this article will not describe them further. The bus interface provides an interface between the bus and the transceiver. The transceiver can be one element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices on a transmission medium. The data processed by the processor is transmitted on the wireless medium through the antenna, and further, the antenna also receives data and transmits the data to the processor.

[0075] The processor is responsible for managing the bus and general processing, and can also provide various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. Memory can be used to store data used by the processor when performing operations.

[0076] Another embodiment of the present application proposes a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it can implement a method for air transportation of emergency materials based on road network characteristics for drones as described in the above method embodiment.

[0077] That is, those skilled in the art can understand that all or part of the steps in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a program, and the program is stored in a storage medium, including a number of instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, ROM (Read-Only Memory), RAM (Random Access Memory), disk or optical disk and other media that can store program codes.

[0078] Those skilled in the art will appreciate that the above embodiments are specific embodiments for implementing the present application, and in actual applications, various changes may be made thereto in form and detail without departing from the spirit and scope of the present application.

Claims

1. A method for air transporting emergency materials based on road network characteristics for drones, characterized in that: include: When the emergency drone takes off from the current service area with emergency supplies, the flight control system of the emergency drone is instructed to control the emergency drone to fly toward the target service area according to the preset route. At the same time, the optoelectronic pod system carried by the emergency drone takes real-time photos of the highway on the ground to obtain surface image data; The surface image data obtained by shooting are subjected to feature extraction using a classification-based image segmentation method and a feature-based image segmentation method to obtain a plurality of road features; Optimize and integrate multiple road features to obtain comprehensive road features, form a master planned route along the highway based on the comprehensive road features, and instruct the flight control system of the emergency drone to control the emergency drone to fly at a constant altitude and speed along the highway toward the target service area according to the master planned route; When the emergency drone arrives over the target service area, it switches to the emergency drone autonomous landing mode, waits for the emergency drone to land on the ground and stabilize, and then notifies the ground personnel to unload the emergency supplies; Optimize and integrate multiple road features to obtain comprehensive road features, including: According to the current season, the current weather, the geographical location of the current service area, the geographical location of the target service area, and the real-time traffic information of the highway between the current service area and the target service area, a first comprehensive weight and a second comprehensive weight are respectively assigned to the classification-based image segmentation method and the feature-based image segmentation method, and a confidence weight is respectively assigned to each road feature; Based on the first comprehensive weight, the second comprehensive weight, and the confidence weight of each road feature, multiple road features are optimized and weightedly spliced ​​and integrated to obtain a comprehensive road feature.

2. According to claim 1, a method for air transporting emergency materials for drones based on road network characteristics is characterized in that: The classification-based image segmentation methods include the watershed algorithm, the K-means clustering algorithm, and the support vector machine algorithm. The image segmentation methods based on classification are used to extract features from the surface image data obtained by shooting, including: The surface image data is segmented using a watershed algorithm, and the positions of strong edges and large gradients in the surface image data are taken as the edges of the highway to obtain the first road feature; The surface image data is segmented using the K-means clustering algorithm, and clustering is performed according to the RGB color space of each pixel in the surface image data, and the second road feature is obtained based on the grouping corresponding to the highway; The support vector machine algorithm is used to segment the surface image data, determine the discrimination surface between the highway pixel points and the non-highway pixel points, and obtain the third road feature based on the discrimination surface.

3. The method for air transporting emergency materials based on road network characteristics for drones according to claim 2 is characterized in that: Feature-based image segmentation methods include edge detection, morphological analysis, and texture analysis. Feature extraction is performed on the surface image data obtained by shooting using feature-based image segmentation methods, including: Performing edge detection on the surface image data, determining the edge of the highway based on color difference, brightness difference and contrast difference in the surface image data, and obtaining a fourth road feature; Performing morphological analysis including dilation and erosion operations on the surface image data, extracting the strip shape and straight line structure, and obtaining the fifth road feature; Texture analysis is performed on the surface image data to detect the texture difference between the expressway and the surrounding background, and the texture feature of the expressway is extracted as the sixth road feature.

4. The method for air transporting emergency materials based on road network characteristics for drones according to claim 3 is characterized in that: Optimize and integrate multiple road features to obtain comprehensive road features, including: According to the current season, the current weather, the geographical location of the current service area, the geographical location of the target service area, and the real-time traffic information of the highway between the current service area and the target service area, a first comprehensive weight and a second comprehensive weight are respectively assigned to the classification-based image segmentation method and the feature-based image segmentation method, and a first confidence weight, a second confidence weight, a third confidence weight, a fourth confidence weight, a fifth confidence weight, and a sixth confidence weight are respectively assigned to the first road feature, the second road feature, the third road feature, the fourth road feature, the fifth road feature, and the sixth road feature; Based on the first comprehensive weight, the second comprehensive weight, the first confidence weight, the second confidence weight, the third confidence weight, the fourth confidence weight, the fifth confidence weight and the sixth confidence weight, the first road feature, the second road feature, the third road feature, the fourth road feature, the fifth road feature and the sixth road feature are optimized and weightedly spliced ​​to obtain a comprehensive road feature.

5. The method for air transporting emergency materials based on road network characteristics for drones according to claim 4 is characterized in that: Based on the first comprehensive weight, the second comprehensive weight, the first confidence weight, the second confidence weight, the third confidence weight, the fourth confidence weight, the fifth confidence weight and the sixth confidence weight, the first road feature, the second road feature, the third road feature, the fourth road feature, the fifth road feature and the sixth road feature are optimized and weightedly spliced ​​to obtain a comprehensive road feature, which is achieved by the following formula: ； ； ； in, represents the first comprehensive weight, represents the second comprehensive weight, represents the first confidence weight, represents the second confidence weight, represents the third confidence weight, represents the fourth confidence weight, represents the fifth confidence weight, represents the sixth confidence weight, represents the first road feature, represents the second road feature, Indicates the third road feature, represents the fourth road feature, Indicates the fifth road feature, represents the sixth road feature, Indicates a splicing operation. Represents the comprehensive characteristics of the road.

6. A method for air transporting emergency supplies based on road network characteristics for drones according to any one of claims 1 to 5, characterized in that: Based on the comprehensive characteristics of the road, a master planned route is formed along the highway, and the flight control system of the emergency drone is instructed to control the emergency drone to fly at a constant altitude and speed along the highway toward the target service area according to the master planned route, including: Input the comprehensive road features into the pre-trained recognition model to obtain the highway data output by the recognition model; On the basis of the highway data output by the recognition model, add m meters of width data, set n meters of flight altitude data, and set s kilometers per hour of flight speed data to form a general planning route guided along the highway; The overall planned route is sent to the flight control system of the emergency UAV, instructing the flight control system of the emergency UAV to control the emergency UAV to fly to the target service area at a constant speed and at a fixed altitude along the outer side of the highway m meters away, at a flight altitude of n meters and a flight speed of s kilometers per hour according to the overall planned route.

7. The method for air transporting emergency materials based on road network characteristics for drones according to claim 6 is characterized in that: After obtaining the highway data output by the recognition model, the method further includes: Compare the highway data output by the recognition model with the standard highway data; If the deviation between the highway data output by the recognition model and the standard highway data does not meet the preset acceptable range, or the recognition model cannot recognize the highway data, the flight control system of the emergency drone is instructed to control the emergency drone to hover and wait for receiving the next instruction; Determine whether the conditions for continued flight are met based on surface image data; If it is determined that the conditions for continuing to fly are not met, a return command is sent to the flight control system of the emergency UAV, instructing the flight control system of the emergency UAV to control the emergency UAV to return; If it is determined that the conditions for continued flight are met, a temporary route will be formulated and sent to the flight control system of the emergency UAV, instructing the flight control system of the emergency UAV to control the emergency UAV to continue flying along the temporary route until the deviation between the highway data output by the recognition model and the standard highway data is within the preset acceptable range.

8. An emergency material air transport system for drones based on road network characteristics, characterized in that: include: Initial guidance module, acquisition module, feature extraction module, route drawing module, real-time guidance module and landing switching module; The initial guidance module is used to instruct the flight control system of the emergency drone to control the emergency drone to fly toward the target service area according to the preset route after the emergency drone takes off from the current service area with emergency supplies; The acquisition module is used to take real-time photos of the highway on the ground through the optoelectronic pod system carried by the emergency drone itself to obtain surface image data; A feature extraction module is used to extract features from the surface image data obtained by shooting using a classification-based image segmentation method and a feature-based image segmentation method to obtain multiple road features; A route drawing module is used to optimize and integrate multiple road features to obtain comprehensive road features, and form a general planning route along the highway based on the comprehensive road features; The real-time guidance module is used to instruct the flight control system of the emergency UAV to control the emergency UAV to fly at a constant altitude and speed along the highway toward the target service area according to the overall planned route; The landing switching module is used to switch to the autonomous landing mode of the emergency drone when the emergency drone arrives above the target service area, wait for the emergency drone to land on the ground and stabilize, and then notify the ground personnel to unload the emergency supplies; Optimize and integrate multiple road features to obtain comprehensive road features, including: According to the current season, the current weather, the geographical location of the current service area, the geographical location of the target service area, and the real-time traffic information of the highway between the current service area and the target service area, a first comprehensive weight and a second comprehensive weight are respectively assigned to the classification-based image segmentation method and the feature-based image segmentation method, and a confidence weight is respectively assigned to each road feature; Based on the first comprehensive weight, the second comprehensive weight, and the confidence weight of each road feature, multiple road features are optimized and weightedly spliced ​​and integrated to obtain a comprehensive road feature.

9. An electronic device, characterized in that: include: at least one processor; and, a memory communicatively coupled to the at least one processor; In which, the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute an emergency material air transportation method based on road network characteristics for drones as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, it can implement a method for air transportation of emergency supplies based on road network characteristics for drones as described in any one of claims 1 to 7.

Images