Unmanned aerial vehicle ground simulation flight operation method and device based on 3D terrain perception and interaction method

Abstract

The invention provides an unmanned aerial vehicle ground-imitating flight operation method and device based on 3D terrain perception and an interaction method, and the operation method comprises the steps: planning a flight route task of ground-imitating flight on ground station software, and obtaining the longitudes and latitudes and heights of all waypoints; determining a flight area according to the waypoint, and downloading a corresponding elevation data tif file; the packaging function is used for acquiring the corresponding terrain height according to the latitude and longitude of the flight area; segmenting the route to obtain a plurality of waypoints and longitudes and latitudes thereof, calling a packaging function to obtain a corresponding terrain height, and drawing a 3D terrain effect on the route of the route; uploading a route to a flight control, requesting elevation data of a specified area where a sampling point is located by the flight control, and responding a corresponding terrain height to the flight control by ground station software by calling a packaging function; and the flight controller executes a route task according to the returned terrain height and the preset height of the simulated flight. According to the method, the three-dimensional terrain perception capability and the interaction capability can be dynamically created on the map software.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of unmanned aerial vehicles, in particular to a method, device and interaction method for ground effect flight of unmanned aerial vehicles based on 3D terrain perception. BACKGROUND

[0002] Currently, in the field of unmanned aerial vehicle ground effect flight monitoring, there is a general technical contradiction: the decision of the aircraft depends on the perception of the terrain, but the mainstream Web electronic map service (such as Gaode Map and Google Map) can only provide three-dimensional building models and cannot display the three-dimensional undulations of natural terrain. This leads to the fact that the operator is faced with a "flat" world in terms of vision when monitoring, and cannot intuitively perceive the spatial relationship between the aircraft and the real terrain.

[0003] Existing solutions usually need to pre-acquire and process professional digital elevation model (DEM) data, which is complex and costly, and difficult to achieve real-time rendering of large-scale terrain on the Web. More importantly, these solutions are passive "visualization" and lack direct and intelligent interaction capabilities with three-dimensional scenes. For example, when commanding the unmanned aerial vehicle to fly to a new target point, the latitude, longitude and height values still need to be manually input, and the three-dimensional click interaction of "what you see is what you get" cannot be achieved.

[0004] Therefore, there is an urgent need for a lightweight, widely applicable and intelligent interactive solution that can dynamically create three-dimensional terrain perception capabilities on standard electronic maps. SUMMARY

[0005] To solve the above technical problems, the present application proposes a method, device and interaction method for ground effect flight of unmanned aerial vehicles based on 3D terrain perception.

[0006] According to a first aspect of the present application, a method for ground effect flight of unmanned aerial vehicles based on 3D terrain perception is proposed, comprising: planning a ground effect flight route task on a ground station software and obtaining the latitude, longitude and height of all waypoints; determining the flight area of the unmanned aerial vehicle according to the waypoints, and downloading the corresponding elevation data tif file from a cloud server based on the flight area; packaging a function to analyze the tif file by means of a third-party library, map the latitude and longitude of the flight area to pixel coordinates in the tif file, and obtain the corresponding terrain height according to the pixel coordinates; segmenting the route to obtain a plurality of path points and their latitude and longitude on the route path, calling the packaged function to obtain the terrain height of each path point, and drawing the 3D terrain effect on the route path on the ground station software; uploading the flight path to a flight control, the flight control requesting elevation data of a designated area where a sampling point is located from the ground station software, the ground station software responding to the flight control with a terrain height of the designated area where the sampling point is located by calling the encapsulated function; the flight control executing the flight path task according to the returned terrain height of the designated area where each sampling point is located and a preset height of the ground effect flight.

[0007] Preferably, the flight area of the unmanned aerial vehicle is determined according to the waypoints, and a corresponding elevation data tif file of the flight area is downloaded from a cloud server, comprising: determining a first flight area according to the maximum and minimum latitude and longitude values in all the waypoints; extending the first flight area by a first preset distance to obtain a second flight area; downloading an elevation data tif file of the second flight area from the cloud server.

[0008] Preferably, the second flight area is obtained by extending the first flight area by a first preset distance, comprising: converting the latitude and longitude coordinates of each vertex of the first flight area into UTM coordinates; extending each vertex by a first preset distance in the UTM coordinate system; converting the UTM coordinates of each extended vertex back to latitude and longitude coordinates to obtain the second flight area.

[0009] Preferably, the 3D terrain effect on the flight path is drawn on the ground station software, comprising: drawing the 3D terrain effect on the flight path through ThreeJS according to the latitude, longitude and terrain height of each path point of the flight path; loading the 3D terrain effect onto the map software through the 3D container of the map software of the ground station software.

[0010] Preferably, the flight control requests elevation data of a designated area where a sampling point is located from the ground station software, and the ground station software responds to the flight control with a terrain height of the designated area where the sampling point is located by calling the encapsulated function, comprising: converting the latitude and longitude coordinates of the sampling point requested by the flight control into UTM coordinates; extending the sampling point in four directions by a second preset distance to obtain a plurality of sub-sampling points in the UTM coordinate system, and constructing the designated area; convert the UTM coordinate set of the sampling point and the plurality of sub-sampling points into a latitude and longitude coordinate set, and call the encapsulation function to respond the terrain height of the set to the flight control.

[0011] According to a second aspect of the present application, a 3D terrain perception based unmanned aerial vehicle (UAV) ground effect flight operation device is provided, comprising: a flight path planning module configured to plan a ground effect flight path task on a ground station software, and obtain latitude, longitude and height of all waypoints; a data downloading module configured to determine a flight area of the UAV according to the waypoints, and download a corresponding height data tif file from a cloud server based on the flight area; a function encapsulation module configured to encapsulate a function, parse the tif file by means of a third-party library, map latitude and longitude of the flight area to pixel coordinates in the tif file, and obtain corresponding terrain height according to the pixel coordinates; a 3D terrain drawing module configured to segment the flight path to obtain a plurality of path points and latitude and longitude thereof on the flight path, call the encapsulation function to obtain terrain height of each path point, and draw a 3D terrain effect on the flight path on the ground station software; a data response module configured to upload the flight path to a flight control, and request the flight control to request height data of a specified area where the sampling point is located from the ground station software, and respond terrain height of the specified area where the sampling point is located to the flight control by calling the encapsulation function; a flight operation module configured to enable the flight control to execute the flight path task according to returned terrain height of each specified area where the sampling point is located and a preset height of the ground effect flight.

[0012] According to a third aspect of the present application, an electronic device is provided, comprising one or more processors, a memory for storing one or more programs, when the one or more programs are executed by the one or more processors, the electronic device implements the 3D terrain perception based unmanned aerial vehicle (UAV) ground effect flight operation method provided in any of the embodiments of the first aspect.

[0013] According to a fourth aspect of the present application, a computer readable storage medium is provided, and a computer program is stored on the computer readable storage medium, when the program is executed by a processor, the 3D terrain perception based unmanned aerial vehicle (UAV) ground effect flight operation method provided in any of the embodiments of the first aspect is implemented.

[0014] According to a fifth aspect of the present application, a method for interacting with a UAV flying along a terrain is provided, which is performed on the UAV flying along a terrain device according to any of the embodiments of the second aspect. The method comprises: According to the latitude, longitude and height of the waypoints of the flight path, a 3D flight path and 3D waypoint effect is drawn on the ground station software; Obtaining the screen pixel coordinates clicked by the user, and converting the screen pixel coordinates into NDC coordinates; Creating a ray parameter equation in the camera view space of the map software according to the NDC coordinates; Detecting whether there is an intersection between the ray parameter equation and the waypoints in the flight path, and if so, controlling the UAV to fly to the waypoint with which the ray parameter equation intersects.

[0015] Preferably, the drawing of the 3D flight path and 3D waypoint effect on the ground station software according to the latitude, longitude and height of the waypoints of the flight path comprises: Drawing the 3D flight path and 3D waypoint on the ground station software through ThreeJS according to the latitude, longitude and height of the waypoints of the flight path; Loading the 3D flight path and 3D waypoint on the map software through the 3D container of the map software of the ground station software; Setting the pitch angle and rotation angle parameters of the map software, and enabling the 3D flight path and 3D waypoint effect.

[0016] Preferably, the obtaining of the screen pixel coordinates clicked by the user and the conversion of the screen pixel coordinates into NDC coordinates comprise: Converting the screen pixel coordinates into NDC coordinates through the following conversion formula: NDC_x=(pixel_x*scale / width)*2-1 NDC_y=-(pixel_y*scale / height)*2+1 wherein NDC_x and NDC_y are NDC coordinates, pixel_x and pixel_y are screen pixel coordinates, width and height are the width and height of the screen canvas respectively, and scale is the screen scaling ratio.

[0017] Preferably, the creating of the ray parameter equation in the camera view space of the map software according to the NDC coordinates comprises: Setting the clipping space coordinates as (NDC_x, NDC_y, -1, 1); The projection matrix and the camera view matrix are used to map the clipping space coordinates to the camera space and the world space in sequence to obtain a world space point P, and then a ray direction vector is obtained as follows: wherein the ray origin is O=(O x , O y , O z ) and the direction vector is D=(D x , D y , D z ); Any point R(t) on the ray is expressed by a ray parameter equation as follows: R(t)=O+t*D wherein the parameter t≥0.

[0018] Preferably, the method further comprises: Supposing that the center of the 3D waypoint is C=(C x , C y , C z ) and the radius is r; a vector is defined; a projection is calculated as follows: the shortest distance square of the ray to the center of the 3D waypoint is calculated as follows: If d 2 ≤r 2 , the ray intersects with the 3D waypoint.

[0019] Preferably, the method further comprises: If it is detected that the ray parameter equation intersects with multiple waypoints in the flight path, the UAV is controlled to fly from the nearest intersection point to the farthest intersection point in sequence.

[0020] The present application provides a UAV ground effect flight operation method and device based on 3D terrain perception and an interactive method. In a map software without relying on a preset three-dimensional terrain database, terrain three-dimensional undulating effects are generated and rendered in real time by using ground effect flight task data, and three-dimensional flight path waypoints are rendered to provide users with intuitive three-dimensional space interactive functions for controlling functions such as homing and fixed waypoint flight. BRIEF DESCRIPTION OF DRAWINGS

[0021] The accompanying drawings are included to provide a further understanding of embodiments and are incorporated in and constitute a part of this specification. The drawings illustrate embodiments and, together with the description, serve to explain principles of the application. Other embodiments and many of the intended advantages of the present application will be readily appreciated as the same becomes better understood by reference to the following detailed description. The elements of the drawings are not necessarily to scale as use of scale would tend to obscure certain example features of the application. Like reference numerals designate corresponding similar parts throughout the drawings.

[0022] Figure 1 is a flow chart of a method for UAV hill flying operation based on 3D terrain perception according to an embodiment of the present application; Figure 2 is a schematic diagram of a device for UAV hill flying operation based on 3D terrain perception according to an embodiment of the present application; Figure 3 is a schematic diagram of an electronic device according to an embodiment of the present application; Figure 4 is a flow chart of a method for UAV hill flying interaction based on 3D terrain perception according to an embodiment of the present application. DETAILED DESCRIPTION

[0023] The features and exemplary embodiments of the various aspects of the present application will be described in detail below with reference to the drawings. To make the purpose, technical solutions and advantages of the present application more clear, the present application will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are configured only to explain the present application and are not configured to limit the present application. The present application can be implemented without some of the specific details by those skilled in the art. The following description of the embodiments is merely provided to provide a better understanding of the present application by showing examples of the present application.

[0024] It should be noted that, in this document, the terms such as first and second are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. Moreover, the terms "include", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such a process, method, article or device. Without more limitations, the elements defined by the statement "include" do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0025] The present application proposes a method for UAV hill flying operation based on 3D terrain perception. Figure 1A flow chart of a method for UAV hill-hugging flight operation based on 3D terrain perception is shown according to an embodiment of the present application, as shown in Figure 1 The operation method includes the following steps: Step S101: Plan a route task for hill-hugging flight on the ground station software, and obtain the longitude, latitude and height of all waypoints.

[0026] In one embodiment, several waypoints are planned on the map software of the ground station software (such as Gaode Map, Google Map, etc.), and the longitude, latitude and height information of all waypoints are obtained, and the waypoints are automatically connected to generate a route task. Then the preset height of the hill-hugging flight is planned on the ground station software.

[0027] Step S102: Determine the flight area of the UAV according to the waypoints, and download the corresponding elevation data tif file from the cloud server based on the flight area.

[0028] In one embodiment, all waypoints are traversed to find the maximum and minimum latitude and longitude values, and a first flight area containing all waypoints is determined, which is a rectangular area. Then the first flight area is expanded by a first preset distance in the east, west, south and north directions to obtain a second flight area which is a rectangular area.

[0029] In one embodiment, the generation process of the second flight area specifically includes: converting the latitude and longitude coordinates of each vertex of the first flight area into UTM coordinates, expanding each vertex by a first preset distance in the UTM coordinate system, and then converting the UTM coordinates of each expanded vertex back to latitude and longitude coordinates to obtain the second flight area. Finally, the corresponding elevation data tif file is downloaded from the cloud server according to the second flight area.

[0030] Step S103: Package a function to parse the tif file by means of a third-party library, map the latitude and longitude of the flight area to the pixel coordinates in the tif file, and obtain the corresponding terrain height according to the pixel coordinates.

[0031] In one embodiment, the function is packaged for subsequent direct calling of the packaged function, which can output the corresponding terrain height by inputting the latitude and longitude coordinates. The process of the packaged function includes: parsing the elevation data tif file by means of the third-party library geotiff, mapping the latitude and longitude coordinates of the second flight area to the pixel coordinates in the tif file, and directly reading the corresponding terrain height according to the pixel coordinates.

[0032] Step S104: The route is segmented to obtain a plurality of path points and their latitude and longitude on the route path, an encapsulation function is called to obtain the terrain height of each path point, and a 3D terrain effect on the route path is drawn on the ground station software.

[0033] In one specific embodiment, according to the number of path points that want to be segmented on the route path, a linear interpolation method is used to generate a plurality of path points and obtain corresponding latitude and longitude coordinates, and then an encapsulation function is called to obtain the terrain height of each path point. According to the latitude and longitude coordinates and the terrain height of each path point, a 3D terrain effect on the route path is drawn through the ThreeJS software, and then the 3D terrain effect is loaded onto the map software through the 3D container of the map software of the ground station software.

[0034] For example, the following gives an embodiment of how to generate a 3D terrain effect on the map software of the ground station software: If the map software used by the ground station software is Gaode Map, the 3D terrain effect is loaded onto Gaode Map through the 3D container GLCustomLayer of Gaode Map; if the map software used by the ground station software is Google Map, the 3D terrain effect is loaded onto Google Map through the 3D container WebGLOverlayView of Google Map.

[0035] Step S105: The route is uploaded to the flight control, the flight control requests the elevation data of the specified area where the sampling point is located to the ground station software, and the ground station software responds to the flight control by calling an encapsulation function to convert the data format to the terrain height required by the flight control.

[0036] In one specific embodiment, since the elevation data (DEM slice) downloaded from the cloud server is in tif file format, it is necessary to convert the data format to the terrain height required by the flight control by calling an encapsulation function.

[0037] Specifically, the flight control first determines a sampling point, and then requests the elevation data of the specified area where the sampling point is located to the ground station software. The ground station software converts the latitude and longitude coordinates of the sampling point into UTM coordinates, and then performs multiple expansions in the east, west, south and north directions with the sampling point as the center and a second preset distance as the expansion distance, to obtain a plurality of sub-sampling points, thereby constructing the specified area where the sampling point is located. The ground station software converts the UTM coordinate set of the sampling point and the plurality of sub-sampling points into a latitude and longitude coordinate set, and then calls an encapsulation function to obtain the terrain height corresponding to the latitude and longitude coordinate set and returns it to the flight control.

[0038] For example, the following gives an embodiment of how to construct a specified area and generate sub-sampling points: The flight control determines a sampling point, the ground station software converts the longitude and latitude coordinates of the sampling point into UTM coordinates, and constructs 8*7=56 grid blocks (expanded) near the sampling point, each grid block is 100m*100m (second preset distance), generates a designated area where the sampling point is located, and each grid block is further divided into 4*4=16 sub-sampling points, so as to obtain a UTM coordinate set of 56*16=896 points.

[0039] Step S106: The flight control executes the flight route task according to the terrain height of the designated area where each sampling point is located and the preset height of the ground effect flight.

[0040] Therefore, the unmanned aerial vehicle ground effect flight operation method based on 3D terrain perception provided in the application can generate and render the terrain three-dimensional undulating effect in real time by using the ground effect flight task data on the map software without relying on the pre-set three-dimensional terrain database, so that the user can intuitively perceive the spatial relationship between the unmanned aerial vehicle and the real terrain.

[0041] Based on the same inventive concept as the above unmanned aerial vehicle ground effect flight operation method based on 3D terrain perception, the application further provides an unmanned aerial vehicle ground effect flight operation device based on 3D terrain perception. Figure 2 A schematic diagram of an unmanned aerial vehicle ground effect flight operation device based on 3D terrain perception according to an embodiment of the application is shown, as shown in the figure, the device comprises: Figure 2 As shown in the figure, the device comprises: The flight route planning module 201 is configured to plan a flight route task of the ground effect flight on the ground station software, and obtain the longitude, latitude and height of all waypoints.

[0042] The data download module 202 is configured to determine the flight area of the unmanned aerial vehicle according to the waypoints, and download the corresponding elevation data tif file from the cloud server based on the flight area.

[0043] The function encapsulation module 203 is configured to encapsulate functions, analyze the tif file by means of a third-party library, map the longitude and latitude of the flight area to pixel coordinates in the tif file, and obtain the corresponding terrain height according to the pixel coordinates.

[0044] The 3D terrain drawing module 204 is configured to segment the flight route to obtain a plurality of path points and their longitude and latitude on the flight route path, call the encapsulated function to obtain the terrain height of each path point, and draw the 3D terrain effect on the flight route path on the ground station software.

[0045] The data response module 205 is configured to upload the flight route to the flight control, the flight control requests the elevation data of the designated area where the sampling point is located to the ground station software, and the ground station software responds the terrain height of the designated area where the sampling point is located to the flight control by calling the encapsulated function.

[0046] The flight operation module 206 is configured to cause the flight controller to perform the flight task according to the terrain height of the designated area where each sampling point is located and the preset height of the ground effect flight.

[0047] According to the unmanned aerial vehicle ground effect flight operation method based on 3D terrain sensing, based on the same inventive concept, the present application also provides an electronic device.

[0048] Figure 3 A schematic diagram of an electronic device according to an embodiment of the present application is shown, as shown in Figure 3 The electronic device includes one or more processors 301, a memory 302, a bus 303, and a communication interface 304. Among them, one or more processors 301, memory 302 and communication interface 304 are connected through bus 303. The memory 302 is used to store one or more programs, when the one or more programs are executed by the one or more processors 301, so that the electronic device implements the unmanned aerial vehicle ground effect flight operation method based on 3D terrain sensing provided by any of the above embodiments.

[0049] According to the unmanned aerial vehicle ground effect flight operation method based on 3D terrain sensing, based on the same inventive concept, the present application also provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the unmanned aerial vehicle ground effect flight operation method based on 3D terrain sensing provided by any of the above embodiments.

[0050] According to the unmanned aerial vehicle ground effect flight operation method based on 3D terrain sensing, based on the same inventive concept, the present application also provides an unmanned aerial vehicle ground effect flight interaction method based on 3D terrain sensing, which is executed on the unmanned aerial vehicle ground effect flight operation device based on 3D terrain sensing provided by any of the above embodiments. Figure 4 A flowchart of an unmanned aerial vehicle ground effect flight interaction method based on 3D terrain sensing according to an embodiment of the present application is shown, as shown in Figure 4 The interaction method includes the following steps: Step S401: Draw 3D route and 3D waypoint effects on the ground station software according to the latitude, longitude and height of the route waypoint.

[0051] In one specific embodiment, similar to the method of generating 3D terrain effects, first draw 3D route and 3D waypoint on the ground station software according to the latitude and longitude coordinates and terrain height of the route waypoint through ThreeJS software, then load 3D route and 3D waypoint effects on the map software through the 3D container of the map software of the ground station software, and then adjust the pitch angle and rotation angle parameters of the map software to open 3D route and 3D waypoint effects.

[0052] For example, the following gives an embodiment of how to generate 3D route and 3D waypoint effects on the map software of the ground station software: If the map software used by the ground station software is Gaode Map, the 3D route and 3D waypoint effects are loaded onto Gaode Map through the 3D container GLCustomLayer of Gaode Map, and then the pitch angle and rotation angle parameters of Gaode Map are adjusted and set to turn on the 3D route and 3D waypoint effects.

[0053] If the map software used by the ground station software is Google Map, the 3D route and 3D waypoint effects are loaded onto Google Map through the 3D container WebGLOverlayView of Google Map, and then the pitch angle and rotation angle parameters of Google Map are adjusted and set to turn on the 3D route and 3D waypoint effects.

[0054] Step S402: Obtain the screen pixel coordinates clicked by the user, and convert the screen pixel coordinates into NDC coordinates.

[0055] One of the purposes of 3D interaction is to accurately identify which waypoint in the 3D scene is clicked through the two-dimensional screen pixel coordinates clicked by the user, so as to make the UAV perform the return or flight action to the waypoint.

[0056] Therefore, in a specific embodiment, after the user clicks the screen, the screen pixel coordinates clicked by the user are obtained, and the screen pixel coordinates are converted into NDC coordinates (normalized device coordinates) through the following conversion formula: NDC_x=(pixel_x*scale / width)*2-1 NDC_y=-(pixel_y*scale / height)*2+1 Where NDC_x, NDC_y are NDC coordinates, ranging from -1 to 1, pixel_x, pixel_y are screen pixel coordinates, width, height are the width and height of the screen canvas respectively, and scale is the screen scaling ratio.

[0057] Step S403: Create a ray parameter equation in the camera view space of the map software according to the NDC coordinates.

[0058] First of all, the process of rendering 3D waypoints by the camera of ThreeJS is as follows: World standard coordinates of waypoints → conversion using camera view matrix → camera space coordinates → projection matrix conversion → clipping space coordinates → screen pixel coordinates.

[0059] Therefore, in a specific embodiment, according to the above process, the process of creating a ray parameter equation can be obtained as follows: Let the clipping space coordinates be (NDC_x, NDC_y, -1, 1); Using the projection matrix and the camera view matrix, the clipping space coordinates are mapped to the camera space and the world space in sequence to obtain a world space point P, and then a ray direction vector is obtained as: Where the ray starting point is O = (O x , O y , O z ) and the direction vector is D = (D x , D y , D z ); Then any point R(t) on the ray is expressed by a ray parameter equation as: R(t) = O + t * D Where the parameter t ≥ 0.

[0060] Step S404: detecting whether there is an intersection point between the ray parameter equation and the waypoint in the flight route, and if so, controlling the UAV to fly to the waypoint with the intersection point.

[0061] In one specific embodiment, detecting whether there is an intersection point between the ray parameter equation and the waypoint in the flight route includes the following steps: Let the center of the 3D waypoint be C = (C x , C y , C z ) and the radius be r; Define a vector ; Calculate the projection: Calculate the shortest distance square of the ray to the center of the 3D waypoint: If d 2 ≤ r 2 , the ray intersects with the 3D waypoint.

[0062] Step S405: if it is detected that there are intersection points between the ray parameter equation and multiple waypoints in the flight route, then controlling the UAV to fly from the nearest intersection point to the farthest intersection point in sequence.

[0063] In one specific embodiment, the user may drag the screen while clicking multiple 3D waypoints, and the detection result is that there are intersection points between the ray parameter equation and multiple waypoints in the flight route, and at this time, the UAV is controlled to fly to the waypoint of the nearest intersection point first, and then to the waypoint of the farthest intersection point in sequence.

[0064] Therefore, the unmanned aerial vehicle ground-hugging flight interaction method based on 3D terrain perception provides intuitive three-dimensional space interaction function for users by rendering three-dimensional flight path points, and is used for function control of return flight, fixed flight points and the like.

[0065] In the embodiments of the present application, it should be understood that the disclosed technical contents can be implemented in other ways. Among them, the above-mentioned device / system / method embodiment is only illustrative, for example, the division of the units can be a logical function division, and actual implementation can have another division mode, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection between units or modules through some interfaces, and can be electrical or other forms.

[0066] The units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, that is, they can be located in one place, or they can be distributed to multiple units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0067] In addition, the functional units in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0068] The integrated unit, if realized in the form of a software functional unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a plurality of instructions for causing a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The foregoing storage medium includes: a U disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store program codes.

[0069] It is clear that a person skilled in the art can make various modifications and alterations to the embodiments of the application without departing from the spirit and scope of the application. In this manner, the application is also intended to cover these modifications and alterations if they come within the scope of the claims of the application and their equivalents. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The mere fact that different claims enumerate mutually different measures does not indicate that a combination of these measures cannot be used to advantage. Any reference signs in the claims should not be construed as limiting the scope.

Claims

1. A method for UAV terrain-following flight operations based on 3D terrain perception, characterized in that, include: Plan the flight path for terrain-following flight on the ground station software and obtain the latitude, longitude and altitude of all waypoints; The flight area of ​​the UAV is determined based on the waypoints, and the corresponding elevation data tif file is downloaded from the cloud server based on the flight area; The encapsulated function parses the tif file with the help of a third-party library, maps the latitude and longitude of the flight area to pixel coordinates in the tif file, and obtains the corresponding terrain height based on the pixel coordinates; The flight path is segmented to obtain multiple path points and their latitude and longitude along the flight path. The encapsulated function is called to obtain the terrain height of each path point, and the 3D terrain effect along the flight path is drawn on the ground station software. The flight path is uploaded to the flight controller, which requests elevation data of the designated area where the sampling point is located from the ground station software. The ground station software responds to the flight controller with the terrain elevation of the designated area where the sampling point is located by calling the encapsulated function. The flight controller executes the flight path task based on the terrain altitude of the designated area where each of the returned sampling points is located and the preset altitude for terrain-following flight.

2. The method according to claim 1, characterized in that, The step of determining the flight area of ​​the UAV based on the waypoints and downloading the corresponding elevation data tif file from the cloud server based on the flight area includes: The first flight area is determined based on the maximum and minimum latitude and longitude values ​​among all the waypoints. A second flight area is obtained by extending the first flight area outward by a first preset distance; Based on the second flight area, download the elevation data tif file of that area from the cloud server.

3. The method according to claim 2, characterized in that, The step of expanding the first flight area by a first preset distance to obtain a second flight area includes: Convert the latitude and longitude coordinates of each vertex in the first flight area to UTM coordinates; In the UTM coordinate system, extend each vertex outward by a first preset distance; The UTM coordinates of each vertex after expansion are converted back to latitude and longitude coordinates to obtain the second flight area.

4. The method according to claim 1, characterized in that, The process of drawing 3D terrain effects along the flight path on the ground station software includes: Based on the latitude, longitude and terrain elevation of each path point of the route, use ThreeJS to draw the 3D terrain effect on the route path; The 3D terrain effect is loaded into the map software through the 3D container of the ground station software.

5. The method according to claim 1, characterized in that, The flight controller requests elevation data of the designated area where the sampling point is located from the ground station software. The ground station software responds to the flight controller with the terrain elevation of the designated area where the sampling point is located by calling the encapsulated function, including: Convert the latitude and longitude coordinates of the flight control request sampling points to UTM coordinates; In the UTM coordinate system, with the sampling point as the center and the second preset distance as the outward expansion distance, the sampling point is expanded multiple times in four directions to obtain multiple sub-sampling points, which are then constructed into the specified region. The UTM coordinate set of the sampling point and the multiple sub-sampling points is converted into a latitude and longitude coordinate set, and the encapsulation function is called to transmit the terrain altitude of the set to the flight controller.

6. A drone terrain-following flight operation device based on 3D terrain perception, characterized in that, include: The route planning module is configured to plan flight routes for terrain-following flight on the ground station software and obtain the latitude, longitude and altitude of all waypoints. The data download module is configured to determine the flight area of ​​the UAV based on the waypoints, and download the corresponding elevation data tif file from the cloud server based on the flight area; The function encapsulation module is configured to encapsulate functions. By using a third-party library to parse the tif file, it maps the latitude and longitude of the flight area to pixel coordinates in the tif file and obtains the corresponding terrain height based on the pixel coordinates. The 3D terrain rendering module is configured to segment the flight path, obtain multiple path points and their latitude and longitude along the flight path, call the encapsulated function to obtain the terrain height of each path point, and render the 3D terrain effect along the flight path on the ground station software. The data response module is configured to upload the flight path to the flight controller. The flight controller requests elevation data of the designated area where the sampling point is located from the ground station software. The ground station software responds to the flight controller with the terrain elevation of the designated area where the sampling point is located by calling the encapsulated function. The flight operation module is configured to instruct the flight controller to execute the flight route task based on the terrain altitude of the designated area where each of the returned sampling points is located and the preset altitude for terrain-following flight.

7. An electronic device, characterized in that, include: One or more processors; A memory for storing one or more programs that, when executed by one or more processors, cause the electronic device to perform the method as described in any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1 to 5.

9. A method for UAV terrain-following flight interaction based on 3D terrain perception, executed in the apparatus as described in claim 6, characterized in that, The method includes: Based on the latitude, longitude, and altitude of the route and waypoints, 3D route and 3D waypoint effects are drawn on the ground station software. Obtain the screen pixel coordinates of the user's click, and convert the screen pixel coordinates into NDC coordinates; Create ray parameter equations in the camera view space of the map software based on the NDC coordinates; The system detects whether there is an intersection between the ray parameter equation and the waypoints in the flight path. If so, it controls the UAV to fly to the waypoint where the ray parameter equation intersects with the waypoint.

10. The method according to claim 9, characterized in that, The step of drawing 3D routes and 3D waypoints on the ground station software based on the latitude, longitude, and altitude of the routes and waypoints includes: Based on the latitude, longitude, and altitude of the route and waypoints, 3D routes and 3D waypoints are drawn on the ground station software using ThreeJS. The 3D flight routes and 3D waypoints are loaded into the map software through the 3D container of the ground station software. Set the pitch and rotation angle parameters of the map software and enable 3D flight paths and 3D waypoints.

11. The method according to claim 9, characterized in that, The step of obtaining the screen pixel coordinates of the user's click and converting the screen pixel coordinates into NDC coordinates includes: The screen pixel coordinates are converted to NDC coordinates using the following conversion formula: NDC_x=(pixel_x*scale / width)*2-1 NDC_y=-(pixel_y*scale / height)*2+1 Where NDC_x and NDC_y are NDC coordinates, ranging from [-1, 1], pixel_x and pixel_y are screen pixel coordinates, width and height are the width and height of the screen canvas, respectively, and scale is the screen scaling ratio.

12. The method according to claim 11, characterized in that, The step of creating ray parameter equations in the camera view space of the map software based on the NDC coordinates includes: Let the clipping space coordinates be (NDC_x, NDC_y, -1, 1); Using the projection matrix and the camera view matrix, the clipping space coordinates are mapped sequentially to the camera space and the world space to obtain the world space point P. The ray direction vector is then obtained as follows: The starting point of the ray is O = (O x O y O z ), the direction vector is D = (D x D y D z ); Then any point R(t) on the ray can be expressed by the ray parametric equation as: R(t) = O + t * D Wherein, the parameter t≥0.

13. The method according to claim 12, characterized in that, The detection of whether the ray parameter equation intersects with waypoints in the flight path includes: Let the center of the 3D waypoint be C = (C x C y C z ), with a radius of r; Define vector ; Calculate the projection: Calculate the square of the shortest distance from the ray to the center of the 3D waypoint: If d 2 ≤r 2 Then the ray intersects with the 3D waypoint.

14. The method according to claim 9, characterized in that, The method further includes: If the ray parameter equation is found to intersect with multiple waypoints in the flight path, the UAV is controlled to fly sequentially from the nearest intersection point to the farthest intersection point.

Images