An unmanned aerial vehicle intelligent flight control method and system based on an emergency rescue scene

Abstract

The present application relates to the field of unmanned aerial vehicle, specifically to an unmanned aerial vehicle intelligent flight control method and system based on emergency rescue scene, which utilizes the initial air-to-ground image queue of the rescue area to construct an initial three-dimensional model, and corrects the reconstructed three-dimensional model according to the shooting field of view features during the image queue collection; extracts the spatial form features of the ground objects from the reconstructed three-dimensional model and combines the atmospheric features to determine the three-dimensional flight path of the unmanned aerial vehicle cluster; adjusts the flight state of part of the unmanned aerial vehicles according to the wake distribution inside the unmanned aerial vehicle cluster; dynamically tracks the relative spatial relationship between the unmanned aerial vehicle cluster and the drop range boundary to control the drop action strategy. The present application takes the terrain and atmospheric conditions of the rescue area as the consideration factors of the flight path planning, which can ensure the safe and continuous flight of the unmanned aerial vehicle; and formulates the flight adjustment and material drop strategy according to the mutual airflow influence inside the unmanned aerial vehicle cluster to ensure the efficient and accurate execution of the emergency rescue task.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of unmanned aerial vehicles, in particular to an intelligent flight control method and system for unmanned aerial vehicles in emergency rescue scenarios. BACKGROUND

[0002] As a kind of unmanned aerial vehicle, unmanned aerial vehicle has the characteristics of light, flexible, low cost and carrying heavy objects. Unmanned aerial vehicle can effectively and quickly perform emergency rescue tasks in disaster scenarios such as earthquake and landslide, which helps to improve the efficiency and reliability of emergency rescue.

[0003] In order to make the unmanned aerial vehicle perform the task stably in the emergency rescue scene, the existing technology will give priority to setting the appropriate flight path for the unmanned aerial vehicle. Currently, the Grey Wolf Optimization Algorithm has been widely used in flight path planning and design. However, the path planning method based on a specific algorithm mostly only considers the topographic condition restriction of the on-site environment, and does not take other factors such as atmospheric conditions into account in the path planning and design process, which cannot guarantee that the finally generated flight path is fully adapted to the emergency rescue scene, which may cause the unmanned aerial vehicle to be out of balance and other accidents. In addition, the existing flight control strategy for unmanned aerial vehicles is limited to the control level of a single unmanned aerial vehicle, and does not fully consider the air flow traction effect between the unmanned aerial vehicle clusters, which cannot maintain overall stable control of the unmanned aerial vehicle cluster and reduces the effectiveness of unmanned aerial vehicle flight control. Therefore, how to plan the flight path under the combined action of topography and atmospheric factors and formulate the flight and material dropping strategy according to the air flow influence between the unmanned aerial vehicle clusters is of great significance for the efficient and safe execution of emergency rescue tasks by unmanned aerial vehicles. SUMMARY

[0004] Considering that the topography and atmospheric conditions in the emergency rescue scene jointly affect the flight feasibility of the unmanned aerial vehicle in the three-dimensional space and the air flow traction effect between different unmanned aerial vehicles in the unmanned aerial vehicle cluster, the existing unmanned aerial vehicle flight control method cannot ensure the safe and continuous flight of the unmanned aerial vehicle cluster in the real environment, nor can it ensure the accurate and stable dropping of materials by the unmanned aerial vehicle cluster, which affects the efficiency and reliability of the execution of emergency rescue tasks. The present application provides an intelligent flight control method for unmanned aerial vehicles in emergency rescue scenarios, which comprises the following steps:

[0005] S100: Collecting an initial image queue of the rescue area, and constructing an initial three-dimensional model of the rescue area based on the initial image queue; modifying the initial three-dimensional model to a reconstructed three-dimensional model according to the shooting field characteristics during the collection of the initial image queue;

[0006] S200: Extracting the spatial form features of the ground objects from the reconstructed three-dimensional model, and planning and screening the three-dimensional flight path of the unmanned aerial vehicle cluster according to the spatial form features and the atmospheric characteristics of the rescue area;

[0007] S300: Obtain a wake flow distribution inside the UAV cluster, estimate a flight deviation interval inside the UAV cluster according to the wake flow distribution, and adjust a flight state of at least part of the UAVs;

[0008] S400: Dynamically track a relative spatial relationship between the UAV cluster and a drop range boundary, and control a drop action strategy of the UAV cluster based on the relative spatial relationship.

[0009] Preferably, in S100, a queue of initial air-to-ground images of the rescue area is collected to construct an initial three-dimensional model of the rescue area; and the initial three-dimensional model is corrected to a reconstructed three-dimensional model according to a shooting field feature during collection of the queue of initial air-to-ground images, specifically as follows:

[0010] An occlusion spatial relationship between ground objects is obtained from remote sensing image analysis of the rescue area, so as to determine an oblique shooting parameter for the rescue area; a three-dimensional partition shooting is performed on the rescue area according to the oblique shooting parameter, and a queue of initial air-to-ground images corresponding to all partition ranges is collected; object contour extraction, contour smoothing connection and partition range splicing are performed on the queue of initial air-to-ground images, so as to construct an initial three-dimensional model of the rescue area.

[0011] A shooting operation point spatial distribution during collection of the queue of initial air-to-ground images is obtained, so as to estimate a shooting field overlap feature during collection of the queue of initial air-to-ground images; a shooting operation point spatial layout is adjusted according to the shooting field overlap feature, and a queue of local air-to-ground images of the rescue area is obtained; and the initial three-dimensional model is corrected to a reconstructed three-dimensional model according to the queue of local air-to-ground images.

[0012] Preferably, in S200, a spatial form feature of a ground object is extracted from the reconstructed three-dimensional model, and a three-dimensional flight path of a UAV cluster is planned and screened according to the spatial form feature and an atmospheric feature of the rescue area, specifically as follows:

[0013] A global spatial granularity feature of each ground object is extracted from the reconstructed three-dimensional model, and a plurality of flight paths allowed for the UAV cluster to pass between ground objects in the rescue area are planned according to the global spatial granularity feature and relative positions of all ground objects.

[0014] A flow disturbance spatial distribution along each flight path is estimated according to the global spatial granularity feature of the ground object and flow pressure and speed features of the rescue area; and the three-dimensional flight path of the UAV cluster is screened according to the flow disturbance spatial distribution.

[0015] Preferably, in S300, the wake flow distribution inside the UAV cluster is acquired, the flight deviation range inside the UAV cluster is estimated according to the wake flow distribution, and the flight state of at least part of the UAVs is adjusted, in particular:

[0016] The formation shape of the UAV cluster and the atmospheric characteristics of the environment where the UAV cluster is located are modeled and analyzed, the vortex distribution of the wake flow formed by each UAV inside the UAV cluster is acquired;

[0017] According to the vortex distribution and the wing attitude of the UAV behind the wake flow, it is judged whether the UAV behind the wake flow has a flight imbalance event; according to the spatial distribution of the occurrence of the flight imbalance event, the flight deviation range inside the UAV cluster is estimated;

[0018] According to the actual air flow effect on the UAV in the flight deviation range, the flight attitude of the UAV in the flight deviation range is adjusted.

[0019] Preferably, in S400, the relative spatial relationship between the UAV cluster and the drop range boundary is dynamically tracked to control the drop action strategy of the UAV cluster, in particular:

[0020] The drop range boundary of the ground expected drop material in the rescue area is calibrated, and the change relationship between the relative distance and the relative orientation between each UAV in the UAV cluster and the drop range boundary is dynamically tracked according to the screened three-dimensional flight path and the formation shape of the UAV cluster;

[0021] According to the change relationship between the relative distance and the relative orientation, the UAV performing the drop action is determined; according to the expected material drop trajectory of the UAV performing the drop action, the initial drop speed and / or the initial drop angle of the UAV performing the drop action are controlled.

[0022] On the other hand, the present application provides a UAV intelligent flight control system based on an emergency rescue scene, the system comprising the following modules:

[0023] A three-dimensional model processing module is used to collect an initial air-to-ground image queue of a rescue area to construct an initial three-dimensional model of the rescue area; the initial three-dimensional model is corrected to a reconstructed three-dimensional model according to the shooting field characteristics during the collection of the initial air-to-ground image queue;

[0024] A flight path determination module is used to extract the spatial form characteristics of ground objects from the reconstructed three-dimensional model, and plan and screen the three-dimensional flight path of the UAV cluster according to the spatial form characteristics and the atmospheric characteristics of the rescue area;

[0025] A wake flow state acquisition module is used to acquire the wake flow distribution inside the UAV cluster;

[0026] a flight state adjustment module configured to estimate a flight deviation interval within the UAV cluster according to the wake distribution, and to adjust a flight state of at least part of the UAVs;

[0027] a drop strategy control module configured to dynamically track a relative spatial relationship between the UAV cluster and a drop range boundary, and to control a drop action strategy of the UAV cluster.

[0028] Preferably, the three-dimensional model processing module is configured to collect an initial aerial-to-ground image queue of the rescue area, and to construct an initial three-dimensional model of the rescue area; to correct the initial three-dimensional model into a reconstructed three-dimensional model according to a shooting field of view feature during collection of the initial aerial-to-ground image queue, specifically as follows:

[0029] to obtain a shielding spatial relationship between ground objects from remote sensing image analysis of the rescue area, and to determine an oblique shooting parameter for the rescue area; to perform three-dimensional partition shooting on the rescue area according to the oblique shooting parameter, and to collect an initial aerial-to-ground image queue corresponding to all partition ranges; to perform object contour extraction, contour smoothing connection, and partition range splicing on the initial aerial-to-ground image queue, and to construct an initial three-dimensional model of the rescue area;

[0030] to obtain a shooting operation point spatial distribution during collection of the initial aerial-to-ground image queue, and to estimate a shooting field of view overlap feature during collection of the initial aerial-to-ground image queue; to adjust a shooting operation point spatial layout according to the shooting field of view overlap feature, and to obtain a local aerial-to-ground image queue of the rescue area; and to correct the initial three-dimensional model into a reconstructed three-dimensional model according to the local aerial-to-ground image queue.

[0031] Preferably, the flight path determination module is configured to extract a spatial form feature of a ground object from the reconstructed three-dimensional model, and to plan and screen a three-dimensional flight path of a UAV cluster according to the spatial form feature and an atmospheric feature of the rescue area, specifically as follows:

[0032] to extract a global spatial granularity feature of each ground object from the reconstructed three-dimensional model, and to plan a plurality of flight paths in the rescue area that allow the UAV cluster to pass between ground objects according to the global spatial granularity feature and relative positions of all ground objects;

[0033] to estimate a spatial distribution of air flow disturbance along each flight path according to the global spatial granularity feature of the ground object and an air flow pressure and speed feature of the rescue area; and to screen a three-dimensional flight path of the UAV cluster according to the spatial distribution of air flow disturbance.

[0034] Preferably, the wake state acquisition module is configured to acquire the wake distribution inside the UAV cluster, in particular:

[0035] modeling the formation shape of the UAV cluster and the atmospheric characteristics of the environment where the UAV cluster is located, and acquiring the vortex distribution of the wake formed by each UAV inside the UAV cluster;

[0036] The flight state adjustment module is configured to estimate the flight deviation range inside the UAV cluster according to the wake distribution, and adjust the flight state of at least part of the UAVs, in particular:

[0037] According to the vortex distribution and the wing attitude of the UAV behind the wake, it is determined whether the UAV behind the wake has a flight imbalance event; and according to the spatial distribution of the flight imbalance event, the flight deviation range inside the UAV cluster is estimated;

[0038] According to the actual airflow acting on the UAV in the flight deviation range, the flight attitude of the UAV in the flight deviation range is adjusted.

[0039] Preferably, the drop strategy control module is configured to dynamically track the relative spatial relationship between the UAV cluster and the drop range boundary, so as to control the drop action strategy of the UAV cluster, in particular:

[0040] The drop range boundary of the ground expected drop material in the rescue area is calibrated, and the change relationship between the relative distance and the relative orientation between each UAV in the UAV cluster and the drop range boundary is dynamically tracked according to the screened three-dimensional flight path and the formation shape of the UAV cluster;

[0041] According to the change relationship between the relative distance and the relative orientation, the UAV performing the drop action is determined; and according to the expected material drop trajectory of the UAV performing the drop action, the initial drop speed and / or the initial drop angle of the UAV performing the drop action are controlled.

[0042] Compared with the prior art, the present application has the following beneficial effects:

[0043] The unmanned aerial vehicle intelligent flight control method and system based on an emergency rescue scene provided by the application utilizes an air-to-ground initial image queue of a rescue area to construct an initial three-dimensional model, and further corrects a reconstructed three-dimensional model according to a shooting field of view feature during image queue collection; spatial form features of ground objects are extracted from the reconstructed three-dimensional model, and combined with atmospheric features to determine a three-dimensional flight path of a cluster of unmanned aerial vehicles; flight states of part of the unmanned aerial vehicles are adjusted according to the wake distribution inside the cluster of unmanned aerial vehicles; and a relative spatial relationship between the cluster of unmanned aerial vehicles and a drop range boundary is dynamically tracked to control a drop action strategy. By taking the terrain of the rescue area and atmospheric conditions as factors for flight path planning, the safety and continuity of flight of the unmanned aerial vehicles can be ensured; and flight adjustment and material drop strategies are formulated according to the mutual airflow influence inside the cluster of unmanned aerial vehicles to ensure efficient and accurate execution of an emergency rescue task. BRIEF DESCRIPTION OF DRAWINGS

[0044] In order to more clearly illustrate the technical solutions in the embodiments of the application, the following will briefly introduce the drawings needed to be used in the embodiment description. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can be obtained by those skilled in the art without any creative effort based on these drawings. Among them:

[0045] Figure 1 is a flowchart of an unmanned aerial vehicle intelligent flight control method based on an emergency rescue scene provided by the application.

[0046] Figure 2 is an initial three-dimensional model.

[0047] Figure 3 is a shooting operation point space layout.

[0048] Figure 4 is a global spatial granularity condition of ground objects.

[0049] Figure 5 is flight path planning and screening.

[0050] Figure 6 is a wake distribution inside a cluster of unmanned aerial vehicles.

[0051] Figure 7 is a material drop trajectory of an unmanned aerial vehicle.

[0052] Figure 8 is a structure diagram of an unmanned aerial vehicle intelligent flight control system based on an emergency rescue scene provided by the application. DETAILED DESCRIPTION

[0053] In order to make the above objectives, characteristics and advantages of the present application more obvious and easy to understand, the specific embodiments of the present application are described in detail below with reference to the drawings. It should be understood that the specific embodiments described herein are only used to explain the present application, but not to limit the present application. In addition, it should be noted that only the parts related to the present application are shown in the drawings, not all the structures. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of the present application.

[0054] The terms "comprising" and "having" and any variations thereof in the present application are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device including a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices.

[0055] In this paper, "embodiment" means that the specific features, structures or characteristics described in conjunction with the embodiment can be included in at least one embodiment of the present application. The phrase appears at various places in the specification does not necessarily refer to the same embodiment, nor is it independent or alternative to other embodiments. Those skilled in the art explicitly and implicitly understand that the embodiments described herein can be combined with other embodiments.

[0056] Please refer to Figure 1 As shown in the drawings, the present application provides an unmanned aerial vehicle intelligent flight control method based on emergency rescue scene, which comprises the following steps:

[0057] S100: Collecting the initial image queue of air-to-ground in the rescue area, and constructing the initial three-dimensional model of the rescue area; according to the shooting field of view characteristics during the collection of the initial image queue of air-to-ground, the initial three-dimensional model is corrected to the reconstructed three-dimensional model.

[0058] Further, in S100, the initial image queue of air-to-ground in the rescue area is collected, and the initial three-dimensional model of the rescue area is constructed; according to the shooting field of view characteristics during the collection of the initial image queue of air-to-ground, the initial three-dimensional model is corrected to the reconstructed three-dimensional model, specifically:

[0059] The occlusion space relationship between the ground objects is obtained by analyzing the remote sensing image of the rescue area, so as to determine the oblique shooting parameters of the rescue area; according to the oblique shooting parameters, the rescue area is three-dimensionally partitioned and shot, and the initial image queue of air-to-ground corresponding to all partition ranges is collected; the object contour extraction, contour smoothing connection and partition range splicing are performed on the initial image queue of air-to-ground, so as to construct the initial three-dimensional model of the rescue area;

[0060] The space distribution of the shooting operation points during the acquisition of the initial air-to-ground image queue is acquired to estimate the shooting field of view overlap characteristics during the acquisition of the initial air-to-ground image queue. According to the shooting field of view overlap characteristics, the space layout of the shooting operation points is adjusted, and the air-to-ground local image queue of the rescue area is acquired. According to the air-to-ground local image queue, the initial three-dimensional model is corrected into a reconstructed three-dimensional model.

[0061] In an emergency rescue scene, the unmanned aerial vehicle needs to perform rescue material delivery and near-ground search rescue tasks, at which time the unmanned aerial vehicle needs to cruise at a low flight height in the rescue area. At the same time, the rescue area is distributed with ground objects such as buildings and mountains, which occupy a certain range in three-dimensional space. When the unmanned aerial vehicle is at a low flight height, at least part of the ground objects will inevitably become flight obstacles. The primary consideration in planning the flight path of the unmanned aerial vehicle is the obstacle. In order to ensure that the unmanned aerial vehicle does not collide with the obstacle during cruising and performing tasks in the rescue area, the first step is to globally locate all ground objects in the rescue area. Specifically, the rescue area is first scanned and photographed by an unmanned aerial vehicle with a large shooting field of view range. In addition, different ground objects in the rescue area are affected by factors such as their own height, size and position. Ground objects that are close to each other will form an occluded blind area, which will prevent the unmanned aerial vehicle from accurately collecting the ground object information in the rescue area during air-to-ground scanning and photography. It is inevitable that there will be a partial missing problem of the ground object. In order to obtain the spatial shape and size of all ground objects in the rescue area as much as possible, the global remote sensing image of the rescue area is analyzed to obtain the occluded spatial relationship between different ground objects. The occluded spatial relationship refers to the size and position of the visual occlusion range of one ground object to another ground object at the corresponding viewing angle. The occluded spatial relationship between any two ground objects in the rescue area is collected and analyzed to determine the shooting parameters such as the tilt angle and field of view range for air-to-ground oblique photography at a high altitude level of the rescue area. It can be understood that when the rescue area is air-to-ground oblique photographed with the above shooting parameters, the missing range of the part of the ground object in the image is the smallest.

[0062] According to the determined oblique shooting parameters, the UAV is controlled to perform three-dimensional partition shooting on the rescue area. The three-dimensional partition shooting refers to that when the UAV moves to a shooting operation point, the UAV performs air-to-ground shooting on a local range of the rescue area according to the oblique shooting parameters. In this way, the UAV collects an air-to-ground initial image at each shooting operation point, and the air-to-ground initial image corresponds to a local range of the rescue area. All the air-to-ground initial images collected by the UAV at all the shooting operation points are arranged in time sequence to form an air-to-ground initial image queue. Specifically, the ground object surface contour of each air-to-ground initial image is extracted, and the ground object surface contours in each air-to-ground initial image are connected smoothly to generate a ground object surface contour distribution feature corresponding to a local range of each air-to-ground initial image. Then, the ground object surface contour distribution features corresponding to all the air-to-ground initial images are spliced according to the relative spatial positions of the air-to-ground initial images to construct an initial three-dimensional model of the rescue area. Please refer to Figure 2 The initial three-dimensional model of the rescue area is obtained through the above process, wherein the X axis and the Y axis respectively correspond to two directions perpendicular to each other on the ground where the rescue area is located, and the Z axis corresponds to the height direction. Figure 2 It can be seen that the shape and size of the ground objects such as buildings and mountains in the rescue area in the three-dimensional space can be fully calibrated.

[0063] Please refer to Figure 3 During the collection of the air-to-ground initial image queue, the UAV collects an air-to-ground initial image at each initial shooting operation point on a local range of the rescue area. In this way, the UAV traverses a plurality of initial shooting operation points during the collection of the entire air-to-ground initial image queue, and these initial shooting operation points jointly form a continuously and smoothly distributed operation point distribution line. Figure 3It can be seen that the UAVs form a shooting overlap range when shooting the space of the rescue area at two adjacent initial shooting operation points. The shooting overlap range corresponds to the shooting field of view overlap feature during the collection of the air-to-ground image queue. The pictures of the two air-to-ground initial images collected by the UAVs at the two adjacent initial shooting operation points have problems such as defocus in the shooting overlap range, which causes both of the two air-to-ground initial images to fail to clearly and accurately record the ground objects corresponding to the shooting overlap range, so that the initial three-dimensional model has problems such as fuzzy ground object contour in the shooting overlap range. In order to correct the problems of the initial three-dimensional model, an additional shooting operation point is arranged between the two adjacent initial shooting operation points along the operation point distribution line. The position of the additional shooting operation point is preferably the midpoint of the operation point distribution line segment between the two adjacent initial shooting operation points. In this way, the spatial layout of the shooting operation points is adjusted, and the UAV is controlled to shoot the local range of the rescue area at all additional shooting operation points to obtain an air-to-ground local image queue. Then, according to the position of each additional shooting operation point on the operation point distribution line, the object contour features of the corresponding air-to-ground local images in the air-to-ground local image queue are fused into the corresponding regions of the initial three-dimensional model, so as to obtain a reconstructed three-dimensional model by modification. The reconstructed three-dimensional model has more object contour details and higher clarity and resolution in the global range than the initial three-dimensional model, and provides accurate three-dimensional spatial information basis for subsequent planning of the flight path.

[0064] S200: Extracting the spatial form features of the ground objects from the reconstructed three-dimensional model, and planning and screening the three-dimensional flight paths of the UAV cluster according to the spatial form features and the atmospheric features of the rescue area.

[0065] Further, in S200, the spatial form features of the ground objects are extracted from the reconstructed three-dimensional model, and the three-dimensional flight paths of the UAV cluster are planned and screened according to the spatial form features and the atmospheric features of the rescue area. Specifically,

[0066] Extracting the global spatial granularity features of each ground object from the reconstructed three-dimensional model, and planning a plurality of flight paths that allow the UAV cluster to pass between the ground objects in the rescue area according to the global spatial granularity features and the relative positions of all ground objects;

[0067] According to the global spatial granularity features of the ground objects and the airflow pressure and speed features of the rescue area, estimating the airflow disturbance spatial distribution along each planned flight path; and screening the three-dimensional flight paths of the UAV cluster according to the airflow disturbance spatial distribution.

[0068] The rescue area contains different types of ground objects such as buildings and mountains, and the surface profiles of these ground objects are different, resulting in different global spatial granularity sizes of different ground objects in the reconstructed three-dimensional model, where the global spatial granularity size refers to the size range of the entity of the ground object. Generally speaking, the global spatial granularity size of the ground object with a relatively flat surface profile change in the reconstructed three-dimensional model is larger, and the global spatial granularity size of the ground object with a relatively steep surface profile change in the reconstructed three-dimensional model is smaller. Please refer to Figure 4 , respectively, the global spatial granularity of the ground object with different surface profile change conditions in the reconstructed three-dimensional model, where (a) and (b) correspond to the ground object with a larger global spatial granularity size and a smaller global spatial granularity size, respectively. The global spatial granularity size of each ground object in the reconstructed three-dimensional model reflects the size range of the entity of each ground object, that is, it reflects the size range of the obstacle formed by each ground object to the flight of the UAV. In order to enable the UAV to accurately avoid the ground object and not to form any physical contact and interference with the surface of the ground object, according to the global spatial granularity size of all ground objects and their positional relationship with each other, a number of flight paths that allow the UAV swarm to continuously fly through in the corresponding formation formation are planned in the space between different ground objects, and these flight paths can exist in the form of three-dimensional paths in the three-dimensional space corresponding to the rescue area. Compared with the prior art which can only plan flight paths in a two-dimensional plane, the global spatial granularity size distribution of each ground object in the three-dimensional space is combined on the basis of the reconstructed three-dimensional model to set three-dimensional layout flight paths in the three-dimensional space where the rescue area is located, and the continuous gap interval of the three-dimensional space where the rescue area is located is fully utilized.

[0069] The UAV flying in the three-dimensional space where the rescue area is located not only needs to consider the continuous gap interval that the above three-dimensional space can provide, but also needs to consider the atmospheric flow characteristics inside the above three-dimensional space. Atmospheric flow will cause disturbance to the flight of the UAV, and the UAV is more likely to experience abnormal conditions such as imbalance and stall in the interval where the atmospheric flow is intense or unstable, affecting the normal flight of the UAV. Please refer to Figure 5, two flight paths, flight path 1 and 2, are obtained by the above planning process. Considering that the different flight paths pass through different ground objects in the rescue area in the three-dimensional space, the global spatial granularity of the different ground objects, and the atmospheric flow pressure and speed are all different, the larger the global spatial granularity of the ground object, the smaller the dimension of the flight path passing through the corresponding interval of the ground object, the easier the narrow tube effect occurs in the above interval, and the greater the atmospheric flow pressure difference and the higher the atmospheric flow speed, the greater the degree of narrow tube effect. The greater the degree of narrow tube effect, the greater the flow disturbance the UAV receives, and once the flow disturbance intensity exceeds the disturbance limit that the UAV can withstand, the UAV will not be able to fly safely through the corresponding interval. In order to ensure that the UAV can maintain safe and stable flight throughout the flight path, according to the global spatial granularity characteristics of the ground object and the flow pressure and speed characteristics of the rescue area, the flow disturbance spatial distribution along each flight path is estimated to screen the three-dimensional flight path that can maintain safe and stable flight throughout the flight path. Figure 5 The flow disturbance received by the UAV cluster throughout the flight path 1 is significantly smaller than the flow disturbance received throughout the flight path 2, so flight path 1 is selected as the final three-dimensional flight path of the UAV cluster.

[0070] S300: Obtain the wake distribution inside the UAV cluster, estimate the flight deviation interval inside the UAV cluster according to the wake distribution, and adjust the flight state of at least part of the UAVs.

[0071] Further, in S300, the wake distribution inside the UAV cluster is obtained, the flight deviation interval inside the UAV cluster is estimated according to the wake distribution, and the flight state of at least part of the UAVs is adjusted, specifically:

[0072] Modeling and analyzing the formation shape of the UAV cluster and the atmospheric characteristics of the environment, obtaining the vortex distribution of the wake formed by each UAV inside the UAV cluster;

[0073] According to the vortex distribution and the wing attitude of the UAV behind the wake, it is judged whether the UAV behind the wake has a flight imbalance event; according to the spatial distribution of the flight imbalance event, the flight deviation interval inside the UAV cluster is estimated;

[0074] According to the actual flow acting on the UAV in the flight deviation interval, the flight attitude of the UAV in the flight deviation interval is adjusted.

[0075] In order to keep all the UAVs in the UAV cluster flying in synchronization, the UAV cluster usually flies in a corresponding formation shape, and common formation shapes include a "one" formation and a "V" formation. Regardless of the formation shape in which the UAV cluster flies, the UAV cluster will exhibit a front-to-back arrangement of UAVs, and the UAVs in the front will generate atmospheric wake during flight, which will have an atmospheric effect on the UAVs in the back, thereby affecting the flight state of the UAVs in the back, such as generating adverse flow resistance or air flow lift. If the UAVs in the back maintain the original flight state, the current formation shape of the UAV cluster will not be able to be maintained, and therefore the flight state of the UAVs in the back needs to be adaptively adjusted in response to the wake vortex effect generated by the UAVs in the front.

[0076] Referring to Figure 6 The UAV cluster has a wake distribution between two UAVs with a front-to-back positional relationship, where UAV A is in front of UAV B, and UAV B is affected by the wake vortex effect from UAV A. In order to enable all the UAVs in the UAV cluster to fly stably to maintain the current formation shape, the formation shape of the UAV cluster and the atmospheric characteristics of the environment in which the UAV cluster is located are first modeled and analyzed to obtain the vortex distribution of the wake formed by each UAV in the UAV cluster; the formation shape can include but is not limited to the relative distance and relative orientation between any two adjacent UAVs in the UAV cluster formation, and the atmospheric characteristics of the environment can include but are not limited to the flow velocity and pressure distribution of the atmospheric environment in which each UAV in the UAV cluster is located, and the like. Then, according to the vortex distribution and the wing attitude of the UAV behind the wake, it is determined whether the wake vortex effect on the UAV causes adverse flow resistance or uneven air flow lift on the left and right wings; if so, it is determined that the UAV behind the wake has a flight imbalance event; if not, it is determined that the UAV behind the wake has no flight imbalance event. And according to the spatial distribution of the UAVs in the flight imbalance event within the entire formation shape, the flight deviation interval (i.e., the interval in which the formation shape deviates within the UAV cluster) is estimated; and according to the actual air flow effect on the UAVs in the flight deviation interval, the flight attitude of the UAVs in the flight deviation interval is adjusted, such as adjusting the wing orientation of the UAVs in the flight deviation interval, so as to convert the actual air flow effect into an auxiliary force for stabilizing flight.

[0077] S400: dynamically track the relative spatial relationship between the UAV cluster and the drop range boundary, to control the drop action strategy of the UAV cluster.

[0078] Further, in S400, the relative spatial relationship between the UAV cluster and the drop range boundary is dynamically tracked to control the drop action strategy of the UAV cluster, specifically:

[0079] The drop range boundary of the ground of the rescue area where the material is expected to be dropped is calibrated, and the relative distance and relative orientation between each UAV in the UAV cluster and the drop range boundary are dynamically tracked according to the screened three-dimensional flight path and the formation shape of the UAV cluster;

[0080] According to the change relationship of the relative distance and the relative orientation, a UAV performing the drop action is determined; and the initial drop speed and / or initial drop angle of the UAV performing the drop action are controlled according to the expected material drop trajectory of the UAV performing the drop action.

[0081] The UAV drops the material to the rescue area, which is an important part of the rescue task. The UAV cluster flies along the three-dimensional flight path with a corresponding formation shape, and the drop range relative to the ground of the rescue area is in a dynamic change relationship. Considering that there are differences in the relative spatial relationship between each UAV in the UAV cluster and the drop range, for each specific drop range, a UAV that can accurately drop the material into the drop range can always be found in the UAV cluster. Specifically, the drop range boundary of the ground of the rescue area where the material is expected to be dropped is calibrated, and the relative distance and relative orientation between each UAV in the UAV cluster and the drop range boundary are dynamically tracked according to the screened three-dimensional flight path and the formation shape of the UAV cluster; the change relationship between the UAV and the drop range boundary is tracked, which represents the change of the relative distance and the relative orientation between the UAV and the drop range boundary over time during the flight; then, according to the change relationship, a UAV that can accurately perform the drop action to the drop range is determined. Finally, the initial drop speed and / or initial drop angle of the UAV performing the drop action are controlled according to the expected material drop trajectory of the UAV performing the drop action, to ensure accurate material drop strategy. Please refer to Figure 7 , the UAVs at the same flight height drop the material to the rescue area at different initial drop angles, and the corresponding drop horizontal displacement of the material is obtained. It can be seen that by changing the initial drop angle of the material, different drop horizontal displacements can be obtained to meet the needs of accurate dropping of the material to different drop ranges on the ground of the rescue area.

[0082] Please refer to Figure 8 , the present application provides a UAV intelligent flight control system based on emergency rescue scene, which comprises the following modules:

[0083] A three-dimensional model processing module is used to collect an initial image queue of the rescue area, and to construct an initial three-dimensional model of the rescue area; the initial three-dimensional model is corrected to a reconstructed three-dimensional model according to the shooting field characteristics during the collection of the initial image queue;

[0084] a flight path determination module configured to extract spatial form features of the ground objects from the reconstructed three-dimensional model, and plan and screen three-dimensional flight paths of the UAV cluster according to the spatial form features and atmospheric features of the rescue area;

[0085] a wake state acquisition module configured to acquire a wake distribution inside the UAV cluster;

[0086] a flight state adjustment module configured to estimate a flight deviation range inside the UAV cluster according to the wake distribution, and adjust a flight state of at least part of the UAVs;

[0087] a dropping strategy control module configured to dynamically track a relative spatial relationship between the UAV cluster and a dropping range boundary, and control a dropping action strategy of the UAV cluster.

[0088] Further, the three-dimensional model processing module is configured to collect an initial air-to-ground image queue of the rescue area, and construct an initial three-dimensional model of the rescue area; and correct the initial three-dimensional model into the reconstructed three-dimensional model according to shooting field of view features during collection of the initial air-to-ground image queue, specifically as follows:

[0089] obtain occlusion spatial relationships between the ground objects from remote sensing images of the rescue area, and determine oblique shooting parameters of the rescue area according to the occlusion spatial relationships; perform three-dimensional partition shooting on the rescue area according to the oblique shooting parameters, and collect an initial air-to-ground image queue corresponding to all partition ranges; perform object contour extraction, contour smoothing connection and partition range splicing on the initial air-to-ground image queue, and construct the initial three-dimensional model of the rescue area;

[0090] acquire a shooting operation point spatial distribution during collection of the initial air-to-ground image queue, and estimate shooting field of view overlap features during collection of the initial air-to-ground image queue according to the shooting operation point spatial distribution; adjust a shooting operation point spatial layout according to the shooting field of view overlap features, and acquire a local air-to-ground image queue of the rescue area; and correct the initial three-dimensional model into the reconstructed three-dimensional model according to the local air-to-ground image queue.

[0091] Further, the flight path determination module is configured to extract spatial form features of the ground objects from the reconstructed three-dimensional model, and plan and screen three-dimensional flight paths of the UAV cluster according to the spatial form features and atmospheric features of the rescue area, specifically as follows:

[0092] extract global spatial granularity features of each ground object from the reconstructed three-dimensional model, and plan a plurality of flight paths allowed for the UAV cluster to pass through between the ground objects in the rescue area according to the global spatial granularity features and relative positions of all the ground objects;

[0093] According to the global spatial granularity characteristics of the ground object and the airflow pressure and speed characteristics of the rescue area, an airflow disturbance spatial distribution along each flight path of the planning is estimated; and according to the airflow disturbance spatial distribution, the three-dimensional flight path of the UAV cluster is screened.

[0094] Further, the wake state acquisition module is configured to acquire the wake distribution inside the UAV cluster, specifically:

[0095] The modeling analyzes the formation shape of the UAV cluster and the atmospheric characteristics of the environment where the UAV cluster is located, and acquires the vortex distribution of the wake formed by each UAV inside the UAV cluster;

[0096] The flight state adjustment module is configured to estimate the flight deviation interval inside the UAV cluster according to the wake distribution, and adjust the flight state of at least part of the UAVs, specifically:

[0097] According to the vortex distribution and the wing attitude of the UAV behind the wake, it is judged whether the UAV behind the wake has a flight imbalance event; and according to the spatial distribution of the flight imbalance event, the flight deviation interval inside the UAV cluster is estimated;

[0098] According to the actual airflow acting on the UAV in the flight deviation interval, the flight attitude of the UAV in the flight deviation interval is adjusted.

[0099] Further, the drop strategy control module is configured to dynamically track the relative spatial relationship between the UAV cluster and the drop range boundary, so as to control the drop action strategy of the UAV cluster, specifically:

[0100] The drop range boundary of the ground expected drop material in the rescue area is calibrated, and the change relationship between the relative distance and the relative orientation between each UAV in the UAV cluster and the drop range boundary is dynamically tracked according to the screened three-dimensional flight path and the formation shape of the UAV cluster;

[0101] According to the change relationship between the relative distance and the relative orientation, the UAV performing the drop action is determined; and according to the expected material drop trajectory of the UAV performing the drop action, the initial drop speed and / or the initial drop angle of the UAV performing the drop action are controlled.

[0102] The UAV intelligent flight control system based on the emergency rescue scene of the application and the operation and effect of the UAV intelligent flight control method based on the emergency rescue scene described above are relatively consistent, and the UAV intelligent flight control system based on the emergency rescue scene will not be repeated here.

[0103] Through the above description of the embodiments, those skilled in the art can clearly understand that the embodiments can be implemented by means of the necessary universal hardware platform, and of course can also be implemented by means of the combination of hardware and software. Based on such understanding, the above technical solutions can be embodied in the form of a computer product in essence or in the form of a contribution to the prior art. The present application can be in the form of a computer program product implemented on one or more computer usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer usable program code.

[0104] Finally, it should be noted that: the above examples are only used to illustrate the technical solutions of the present application, and are not limited thereto, and other embodiments can also be used; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for intelligent flight control of unmanned aerial vehicles (UAVs) in emergency rescue scenarios, characterized in that, The method includes the following steps: S100: Acquire an initial air-to-ground image queue of the rescue area to construct an initial 3D model of the rescue area; based on the field-of-view characteristics captured during the acquisition of the initial air-to-ground image queue, correct the initial 3D model into a reconstructed 3D model, specifically as follows: The spatial relationships of occlusion between ground objects are obtained by analyzing remote sensing images of the rescue area, thereby determining the tilt shooting parameters for the rescue area; based on the tilt shooting parameters, the rescue area is divided into three-dimensional zones for shooting, and an initial air-to-ground image queue corresponding to all zone ranges is collected; object contour extraction, contour smoothing connection, and zone range stitching are performed on the initial air-to-ground image queue to construct an initial three-dimensional model of the rescue area. The spatial distribution of shooting operation points during the initial air-to-ground image queue acquisition is obtained to estimate the field-of-view overlap characteristics during the initial air-to-ground image queue acquisition; the spatial layout of shooting operation points is adjusted according to the field-of-view overlap characteristics, and the air-to-ground local image queue of the rescue area is obtained; the initial 3D model is corrected into a reconstructed 3D model according to the air-to-ground local image queue. S200: Extract spatial morphological features of ground objects from the reconstructed 3D model; based on the spatial morphological features and the atmospheric characteristics of the rescue area, plan and select the 3D flight path of the UAV swarm, specifically: The global spatial granularity features of each ground object are extracted from the reconstructed 3D model. Based on the global spatial granularity features and relative positions of all ground objects, several flight paths are planned within the rescue area that allow the drone swarm to traverse between ground objects. Based on the global spatial granularity characteristics of the ground object and the airflow pressure and velocity characteristics of the rescue area, the spatial distribution of airflow disturbance along each planned flight path is estimated; based on the spatial distribution of airflow disturbance, a three-dimensional flight path is selected in which the UAV cluster can maintain a safe and stable flight state throughout the entire process, and the airflow disturbance intensity experienced by each UAV in the UAV cluster in the three-dimensional flight path does not exceed its own permissible disturbance limit. S300: Obtain the wake distribution within the drone swarm; based on the wake distribution, estimate the flight offset range within the drone swarm, thereby adjusting the flight state of at least some drones, specifically: Modeling and analyzing the formation shape of the UAV swarm and the atmospheric characteristics of its environment, the vortex distribution of the wake formed by each UAV within the UAV swarm is obtained; Based on the vortex distribution and the wing attitude of the UAV behind the wake, determine whether the wake vortex effect on the UAV causes backflow obstruction or whether the airflow lifting effect on the left and right wings is uneven; if so, it is determined that the UAV behind the wake has experienced a flight imbalance event; if not, it is determined that the UAV behind the wake has not experienced a flight imbalance event. Based on the spatial distribution of the flight imbalance events, the flight offset interval within the UAV swarm is estimated; wherein, the flight offset interval refers to the interval within the formation of the UAV swarm where shape shifts occur. Based on the actual airflow action experienced by the UAV within the flight offset range, the flight attitude of the UAV within the flight offset range is adjusted, thereby converting the actual airflow action into a force that assists the UAV in stabilizing its flight; wherein, the flight attitude includes the wing orientation of the UAV within the flight offset range; S400: Dynamically track the relative spatial relationship between the drone cluster and the boundary of the deployment range, thereby controlling the deployment action strategy of the drone cluster.

2. The method according to claim 1, characterized in that, In S400, the relative spatial relationship between the drone swarm and the deployment range boundary is dynamically tracked to control the deployment strategy of the drone swarm, specifically: The boundary of the expected delivery range of materials in the rescue area is marked, and the relative distance and relative orientation between each drone in the drone cluster and the delivery range boundary are dynamically tracked based on the selected three-dimensional flight path and the formation shape of the drone cluster. Based on the relationship between the relative distance and relative orientation, the drone to perform the delivery action is determined; based on the expected material delivery trajectory of the drone to perform the delivery action, the initial delivery speed and / or initial delivery angle of the drone to perform the delivery action are controlled.

3. A drone intelligent flight control system for emergency rescue scenarios, characterized in that, The system includes the following modules: The 3D model processing module is used to acquire an initial air-to-ground image queue of the rescue area to construct an initial 3D model of the rescue area; based on the field-of-view characteristics captured during the acquisition of the initial air-to-ground image queue, the initial 3D model is corrected into a reconstructed 3D model, specifically as follows: The spatial relationships of occlusion between ground objects are obtained by analyzing remote sensing images of the rescue area, thereby determining the tilt shooting parameters for the rescue area; based on the tilt shooting parameters, the rescue area is divided into three-dimensional zones for shooting, and an initial air-to-ground image queue corresponding to all zone ranges is collected; object contour extraction, contour smoothing connection, and zone range stitching are performed on the initial air-to-ground image queue to construct an initial three-dimensional model of the rescue area. The spatial distribution of shooting operation points during the initial air-to-ground image queue acquisition is obtained to estimate the field-of-view overlap characteristics during the initial air-to-ground image queue acquisition; the spatial layout of shooting operation points is adjusted according to the field-of-view overlap characteristics, and the air-to-ground local image queue of the rescue area is obtained; the initial 3D model is corrected into a reconstructed 3D model according to the air-to-ground local image queue. The flight path determination module is used to extract the spatial morphological features of ground objects from the reconstructed 3D model, and plan and select the 3D flight paths of the UAV swarm based on the spatial morphological features and the atmospheric characteristics of the rescue area. Specifically: The global spatial granularity features of each ground object are extracted from the reconstructed 3D model. Based on the global spatial granularity features and relative positions of all ground objects, several flight paths are planned within the rescue area that allow the drone swarm to traverse between ground objects. Based on the global spatial granularity characteristics of the ground object and the airflow pressure and velocity characteristics of the rescue area, the spatial distribution of airflow disturbance along each planned flight path is estimated; based on the spatial distribution of airflow disturbance, a three-dimensional flight path is selected in which the UAV cluster can maintain a safe and stable flight state throughout the entire process, and the airflow disturbance intensity experienced by each UAV in the UAV cluster in the three-dimensional flight path does not exceed its own permissible disturbance limit. The wake state acquisition module is used to acquire the wake distribution within the UAV cluster, specifically: Modeling and analyzing the formation shape of the UAV swarm and the atmospheric characteristics of its environment, the vortex distribution of the wake formed by each UAV within the UAV swarm is obtained; The flight status adjustment module is used to estimate the flight offset range within the UAV cluster based on the wake distribution, thereby adjusting the flight status of at least some of the UAVs, specifically: Based on the vortex distribution and the wing attitude of the UAV behind the wake, determine whether the wake vortex effect on the UAV causes backflow obstruction or whether the airflow lifting effect on the left and right wings is uneven; if so, it is determined that the UAV behind the wake has experienced a flight imbalance event; if not, it is determined that the UAV behind the wake has not experienced a flight imbalance event. Based on the spatial distribution of the flight imbalance events, the flight offset interval within the UAV swarm is estimated; wherein, the flight offset interval refers to the interval within the formation of the UAV swarm where shape shifts occur. Based on the actual airflow action experienced by the UAV within the flight offset range, the flight attitude of the UAV within the flight offset range is adjusted, thereby converting the actual airflow action into a force that assists the UAV in stabilizing its flight; wherein, the flight attitude includes the wing orientation of the UAV within the flight offset range; The deployment strategy control module is used to dynamically track the relative spatial relationship between the drone cluster and the deployment range boundary, thereby controlling the deployment action strategy of the drone cluster.

4. The system according to claim 3, characterized in that, The deployment strategy control module is used to dynamically track the relative spatial relationship between the drone cluster and the deployment range boundary, thereby controlling the deployment action strategy of the drone cluster, specifically: The boundary of the expected delivery range of materials in the rescue area is marked, and the relative distance and relative orientation between each drone in the drone cluster and the delivery range boundary are dynamically tracked based on the selected three-dimensional flight path and the formation shape of the drone cluster. Based on the relationship between the relative distance and relative orientation, the drone to perform the delivery action is determined; based on the expected material delivery trajectory of the drone to perform the delivery action, the initial delivery speed and / or initial delivery angle of the drone to perform the delivery action are controlled.

Images