Terrain route generation method and program product based on three-dimensional real scene

Abstract

The present application relates to a method and program product for generating a terrain route based on a three-dimensional real scene. The method comprises: obtaining real scene model data, generating a safety fence based on the real scene model data and a safety distance, wherein the safety fence is used to cover obstacles in the real scene model so that the drone does not collide with the obstacles; obtaining the number and direction based on preset parameters, obtaining the vertical plane corresponding to the safety fence based on the number and direction, wherein the preset parameters are used to configure the drone, and the vertical plane intersects with the safety fence; obtaining the flight segment based on the intersection line of the vertical plane and the safety fence, and obtaining a terrain route based on the flight segment. The use of this method can improve the consistency between the drone's route and the terrain, and improve the clarity of the image of the real scene taken by the drone.

Description

Technical Field

[0001] The present application relates to the field of unmanned aerial vehicle technology, and in particular to a method, device, electronic device, computer-readable storage medium and computer program product for generating a terrain route based on a three-dimensional real scene. Background Art

[0002] With the development of mechanical technology, drones have emerged. Drones can fly according to planned routes to achieve preset goals. For example, a camera is installed on a drone to obtain images of the target area as the drone flies. At present, when using drones to shoot target areas, such as shooting urban landscapes, the height of buildings is not taken into account. A specific flight altitude is simply set, and the drone's route is inconsistent with the terrain, which leads to inconsistent clarity of the captured images. Summary of the invention

[0003] Based on this, it is necessary to provide a terrain route generation method, device, electronic device, computer-readable storage medium and computer program product based on three-dimensional real scenes, which can improve the consistency between the drone's route and the terrain in order to solve the above technical problems.

[0004] In a first aspect, the present application provides a method for generating a terrain route based on a three-dimensional real scene, comprising:

[0005] Acquire real-scene model data, and generate a safety fence based on the real-scene model data and a safety distance, wherein the safety fence is used to cover obstacles in the real-scene model so that the drone does not collide with the obstacles;

[0006] Obtaining a quantity and a direction based on preset parameters, and obtaining a vertical plane corresponding to the safety fence based on the quantity and the direction, wherein the preset parameters are used to configure the drone, and the vertical plane intersects with the safety fence;

[0007] Flight segments are obtained based on the number of intersection lines of the vertical planes and the safety fences to obtain a terrain route based on the number of flight segments.

[0008] In one embodiment, the preset parameters include camera parameters, lateral overlap rate and route direction, the lateral overlap rate is the probability of overlapping shooting ranges of the drone between different flight segments, the route direction is the driving direction of the drone, and obtaining the quantity and direction based on the preset parameters includes:

[0009] Determine the spacing between adjacent vertical planes based on the lateral overlap ratio and the imaging parameters;

[0010] The direction is determined based on the route orientation, a center point of the safety fence is obtained, and the number is determined based on the center point, the distance and the direction.

[0011] In one embodiment, determining the number based on the center point, the spacing, and the direction comprises:

[0012] Taking the center point as the starting point, horizontally extending along a preset direction toward both ends of the center point to obtain a plurality of vertical points, wherein the distance between the vertical points is the spacing;

[0013] For the vertical point, determining the direction of a candidate plane based on the direction of the route, and determining the candidate plane based on the vertical point and the direction;

[0014] If there are intersection lines between the selected plane and the safety fence, the number of the intersection lines is determined to be the number.

[0015] In one embodiment, the imaging parameters include image resolution, sensor size, and lens focal length, and determining the spacing between adjacent vertical planes based on the lateral overlap rate and the imaging parameters includes:

[0016] Determining an equivalent focal length based on the sensor size and the lens focal length;

[0017] Determine the horizontal distance of the ground range for photographing the real scene model based on the equivalent focal length, the image resolution and the safety distance;

[0018] The spacing is determined based on the horizontal distance and the lateral overlap ratio.

[0019] In one embodiment, there are multiple waypoints on the flight segment, and the waypoints are sorted according to the time when the drone passes through the waypoints. After obtaining the flight segment based on the intersection of the vertical plane and the safety fence, the method includes:

[0020] If the distance between the waypoint that the drone passes first and the waypoint that the drone passes later is less than a preset distance, then determining that the waypoint that the drone passes later is a redundant waypoint;

[0021] If the angle between the line connecting the waypoint that the drone passes first and the waypoint that the drone passes later and the horizontal direction is less than a preset angle, the waypoint that the drone passes later is determined to be the redundant waypoint;

[0022] Eliminating the redundant waypoints from the flight segment to determine the flight segment from which the redundant waypoints are eliminated;

[0023] The obtaining of a terrain route based on the flight segment comprises:

[0024] A terrain route is obtained based on the flight segment excluding the redundant waypoints.

[0025] In one embodiment, generating a safety fence based on the real scene model data and the safety distance includes:

[0026] Adding point indexes to the real scene model data;

[0027] Obtaining a tree data structure and a height map data structure according to the real scene model data and point index;

[0028] Obtaining a height map data set according to the tree data structure and the height map data structure;

[0029] The safety fence is generated according to the height map dataset.

[0030] In a second aspect, the present application also provides a terrain route generation device based on a three-dimensional real scene, comprising:

[0031] An acquisition module, used to obtain real-scene model data, and generate a safety fence based on the real-scene model data and a safety distance, wherein the safety fence is used to cover obstacles in the real-scene model so that the drone does not collide with the obstacles;

[0032] a plane determination module, configured to obtain a quantity and a direction based on preset parameters, and to obtain a vertical plane corresponding to the safety fence based on the quantity and the direction, wherein the preset parameters are used to configure the drone, and the vertical plane intersects the safety fence;

[0033] A route determination module is used to obtain flight segments based on the number of intersection lines of the vertical planes and the safety fences, so as to obtain a terrain route based on the number of flight segments.

[0034] In a third aspect, the present application further provides a drone, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0035] Acquire real-scene model data, and generate a safety fence based on the real-scene model data and a safety distance, wherein the safety fence is used to cover obstacles in the real-scene model so that the drone does not collide with the obstacles;

[0036] Obtaining a quantity and a direction based on preset parameters, and obtaining a vertical plane corresponding to the safety fence based on the quantity and the direction, wherein the preset parameters are used to configure the drone, and the vertical plane intersects with the safety fence;

[0037] Flight segments are obtained based on the number of intersection lines of the vertical planes and the safety fences to obtain a terrain route based on the number of flight segments.

[0038] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the following steps are implemented:

[0039] Acquire real-scene model data, and generate a safety fence based on the real-scene model data and a safety distance, wherein the safety fence is used to cover obstacles in the real-scene model so that the drone does not collide with the obstacles;

[0040] Obtaining a quantity and a direction based on preset parameters, and obtaining a vertical plane corresponding to the safety fence based on the quantity and the direction, wherein the preset parameters are used to configure the drone, and the vertical plane intersects with the safety fence;

[0041] Flight segments are obtained based on the number of intersection lines of the vertical planes and the safety fences to obtain a terrain route based on the number of flight segments.

[0042] In a fifth aspect, the present application further provides a computer program product, including a computer program, which implements the following steps when executed by a processor:

[0043] Acquire real-scene model data, and generate a safety fence based on the real-scene model data and a safety distance, wherein the safety fence is used to cover obstacles in the real-scene model so that the drone does not collide with the obstacles;

[0044] Obtaining a quantity and a direction based on preset parameters, and obtaining a vertical plane corresponding to the safety fence based on the quantity and the direction, wherein the preset parameters are used to configure the drone, and the vertical plane intersects with the safety fence;

[0045] Flight segments are obtained based on the number of intersection lines of the vertical planes and the safety fences to obtain a terrain route based on the number of flight segments.

[0046] The above-mentioned terrain route generation method, device, electronic device, computer-readable storage medium and computer program product based on three-dimensional real scene obtains real scene model data, generates a safety fence based on the real scene model data and the safety distance, and then determines the number and direction of the vertical planes corresponding to the safety fence based on the preset parameters, and then obtains the flight segment based on the intersection line of the vertical plane and the safety fence, so as to obtain the terrain route based on the flight segment. Since the safety fence is at a safe distance from the obstacle in the real scene model, the distance between each point on the intersection line of the vertical plane and the safety fence and the corresponding obstacle is equal, and the terrain route obtained based on the intersection line is consistent with the terrain of the real scene model. When the drone moves on the terrain route obtained based on the intersection line, the distance between the drone and the obstacle in the real scene is unchanged, so that the clarity of each part of the real scene image collected by the drone based on the planned terrain route is consistent, thereby improving the quality of the collected image. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0048] Figure 1 A schematic diagram of a flow chart of a terrain route generation method in one embodiment;

[0049] Figure 2 is a schematic diagram of a terrain route in one embodiment;

[0050] Figure 3 A schematic flow chart of the steps of determining a vertical plane in one embodiment;

[0051] Figure 4 A schematic flow chart of the steps of obtaining the spacing of the vertical planes in one embodiment;

[0052] Figure 5 A schematic diagram of a flow chart of the steps of removing redundant waypoints in one embodiment;

[0053] Figure 6 A schematic diagram of a flow chart of a safety fence generation step in one embodiment;

[0054] Figure 7 A schematic diagram of a process flow of terrain route generation in another embodiment;

[0055] Figure 8 is a structural block diagram of a terrain route generating device in one embodiment;

[0056] Fig. 9 FIG. 1 is a diagram showing the internal structure of a drone in one embodiment. DETAILED DESCRIPTION

[0057] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0058] In an exemplary embodiment, Figure 1 As shown, a method for generating a terrain route based on a three-dimensional real scene is provided. This embodiment uses the method applied to a drone as an example. It can be understood that the method can also be applied to a server, and can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server. In this embodiment, the method includes the following steps 102 to 106. Among them:

[0059] Step 102: obtaining real-scene model data, and generating a safety fence based on the real-scene model data and the safety distance.

[0060] Among them, the safety fence is used to cover the obstacles in the real scene model so that the drone does not collide with the obstacles. Among them, obtaining the real scene model is a three-dimensional virtual display technology that uses a drone or a laser scanner to take a multi-angle panoramic shot of the existing scene (i.e., the three-dimensional real scene) and then uses the three-dimensional real scene modeling software ContextCapture to process and generate it. During browsing, the three-dimensional real scene modeling can zoom in, zoom out, move, and view the model from multiple angles, and it can be checked that the parameters (length, width, height, area, volume) of the objects in the three-dimensional real scene model are consistent with the data information of the real scene. Three-dimensional real scene modeling can be used for site planning, area measurement, and earthwork calculation. In addition, docking with the real scene model progress analysis software can analyze the construction progress of the project, and manage the virtual space operation and maintenance of the real scene model. In this embodiment, close photogrammetry is used to obtain the three-dimensional real scene model data. Close photogrammetry is a measurement method derived from the full combination of the advantages of drone gimbal attitude control and aerial survey high-precision positioning technology. By changing the photography path, the camera is directly aimed at the slope to complete the image acquisition work.

[0061] The safety fence is generated based on the real scene model data and the safety distance. The safety distance is the preset distance between the drone and the obstacle. Therefore, the distance between each point on the safety fence and the corresponding obstacle is the safety distance.

[0062] Optionally, after obtaining the real-scene model data, add a point index to the real-scene model data; obtain a tree data structure and a height map data structure based on the real-scene model data and the point index; obtain a height map data set based on the tree data structure and the height map data structure; and generate a safety fence based on the height map data set.

[0063] Optionally, after obtaining the real scene model data, the height data of the obstacle is obtained, and the safety fence is generated based on the height data of the obstacle and the safety distance.

[0064] Step 104, obtaining a quantity and a direction based on preset parameters, and obtaining a vertical plane corresponding to the safety fence based on the quantity and the direction.

[0065] Wherein, the preset parameters are used to configure the drone, and the vertical plane intersects with the safety fence. The preset parameters may include a lateral overlap rate, a route direction, and a camera parameter of the drone. Wherein, the camera parameter of the drone may include an image resolution, a sensor size, and a lens focal length. The lateral overlap rate is the probability that the shooting range of the drone overlaps between different flight segments. The route direction is the driving direction of the drone. The vertical plane is used to determine the terrain route of the drone. The number and direction may refer to the number and direction of the vertical planes corresponding to the safety fence. After obtaining the number and direction based on the preset parameters, the vertical plane may be obtained based on the obtained number and direction. For example, the corresponding vertical plane is drawn based on the obtained number and direction.

[0066] Exemplarily, when there is more than one vertical plane corresponding to the safety fence, the spacing between the vertical planes can be obtained based on the lateral overlap rate and the camera parameters. And the direction of the vertical plane is determined based on the direction of the route. That is, in this embodiment, the vertical planes are parallel. Since the vertical planes are used to determine the terrain route of the drone, if the spacing between the vertical planes is too narrow or too wide, it may cause the drone to obtain an incomplete image of the real scene. Therefore, by determining the spacing of the vertical planes based on the lateral overlap rate and the camera parameters, the overlap rate of the image when the drone shoots the real scene in different sections can be reduced while ensuring the amount of information of the real scene image.

[0067] Optionally, the distance between the vertical planes may be directly set, and the vertical plane corresponding to the safety fence is determined based on the distance.

[0068] Step 106 , obtaining a flight segment based on the intersection line of the vertical plane and the safety fence, so as to obtain a terrain route based on the flight segment.

[0069] Among them, a flight segment is a part of a terrain route, and a terrain route is formed by connecting multiple flight segments in sequence. Figure 2 As shown, the flight segments 201 in the figure are obtained based on the vertical plane 202, and are sequentially connected to form a terrain route. The terrain route is the flight path of the drone.

[0070] Exemplarily, after the number and direction of the vertical planes are obtained, the number of vertical planes is determined in combination with the preset point. The preset point may be, for example, the center point of the safety fence, and the two ends of the preset direction are extended from the center point. The position of each vertical plane is determined based on the points on the extension line (including the preset point). The flight segment is obtained based on the intersection line of the vertical plane and the safety fence, and the terrain route is obtained based on the flight segment.

[0071] For example, the processor determines based on preset parameters that the number of vertical planes corresponding to the safety fence is 10. The intersection lines of these 10 vertical planes and the safety fence are determined as flight segments, and there are 10 flight segments. These 10 flight segments are connected to obtain a terrain route.

[0072] Optionally, after obtaining the intersection line of the vertical plane and the safety fence, the intersection line may be processed to obtain the flight segment, for example, the intersection line may be translated in the vertical direction to obtain the flight segment.

[0073] By obtaining the real-life model data, generating a safety fence based on the real-life model data and the safety distance, determining the number and direction of the vertical planes corresponding to the safety fence based on preset parameters, and obtaining the flight segment based on the intersection line of the vertical plane and the safety fence, a terrain route is obtained based on the flight segment. Since the safety fence is at a safe distance from the obstacle in the real-life model, the distances between each point on the intersection line of the vertical plane and the safety fence and the corresponding obstacle are equal, and the terrain route obtained based on the intersection line is consistent with the terrain of the real-life model. When the drone moves on the terrain route obtained based on the intersection line, the distance between the drone and the real-life obstacle remains unchanged, so that the clarity of each part of the real-life image collected by the drone based on the planned terrain route is consistent, thereby improving the quality of the collected image.

[0074] In an exemplary embodiment, the preset parameters may include a lateral overlap rate and a flight path direction, so that the processor can determine the number and direction of the vertical planes based on the lateral overlap rate and the flight path direction. Figure 3 As shown, determining the number and direction of vertical planes corresponding to the safety fence based on preset parameters includes steps 302 to 304. Among them:

[0075] Step 302: Determine the distance between adjacent vertical planes based on the lateral overlap ratio and the imaging parameters.

[0076] Among them, the lateral overlap rate is the probability that the shooting range of the drone overlaps between different flight segments. The lateral overlap rate is pre-configured. When the area of ​​the target area that the drone needs to shoot is large, for example, the width of the target area is large, the drone cannot shoot the entire target area in one one-way flight. Therefore, the drone may need to fly back and forth multiple times to shoot the complete target area. In the process of the drone moving back and forth, if the spacing between the flight segments is too small, it may cause the overlap rate of the images shot by the drone between multiple flight segments to be too high, which will cause a waste of resources, and because the spacing between the flight segments is too small, the number of times the drone moves back and forth increases, which will also increase the operating pressure of the drone. If the spacing between the flight segments is too large, it may cause the drone to move to the end point according to the terrain route. The image taken cannot completely cover the target area, and there are images of some areas that are not captured. Since the requirements of each shooting task are different, for example, sometimes it is necessary to shoot a complete real-life image, and sometimes it is necessary for the drone to overlap between the images shot in each flight segment for subsequent synthesis. Therefore, when determining the spacing between the flight segments, that is, the spacing between the vertical planes, it can be determined based on the preset lateral overlap rate. The lower the lateral overlap rate, the larger the spacing between the vertical planes. The higher the lateral overlap rate, the smaller the spacing between vertical planes. When the lateral overlap rate is zero, the images taken by the drone in each flight segment are neither repeated nor a complete image of the target area can be obtained based on the multiple images taken.

[0077] To obtain a specific spacing, the camera parameters of the drone also need to be considered. In some embodiments, the camera parameters may include image resolution, sensor size, and lens focal length. Based on the image resolution, sensor size, and lens focal length, the range that the drone can shoot can be obtained, and then the spacing can be determined based on the range that the drone can shoot and the lateral overlap rate.

[0078] Optionally, the camera parameters may also include the camera shooting range of the drone, and the spacing may be determined based on the camera shooting range and the lateral overlap rate.

[0079] Step 304, determine the direction based on the direction of the route, obtain the center point of the safety fence, and determine the quantity based on the center point, spacing and direction.

[0080] The flight direction refers to the direction of the drone. The projection of the vertical plane to the safety fence is a line, not a plane. The line projected from the vertical plane to the safety fence is parallel to the flight direction.

[0081] Exemplarily, after obtaining the center point corresponding to the safety fence, the number of vertical planes can be determined based on the center point, spacing and direction. For example, after determining the center point, multiple vertical points on the target segment are determined based on the preset direction and spacing. The target segment passes through the center point and the direction of the target segment is the preset direction. The distances between the vertical points on the target segment are equal and equal to the spacing. In some embodiments, if the overlap rate of images taken by the drone in different flight segments is required to be low, the preset direction can be perpendicular to the direction of the route. These vertical points are projected onto the safety fence, and the number of vertical points of the projection points on the safety fence can be the number of vertical planes.

[0082] In some embodiments, each vertical plane can also be determined based on the center point, the spacing and the direction, where the vertical planes are parallel and the distance between two adjacent vertical planes is a value corresponding to the spacing. Since the vertical plane is an infinitely large plane, the vertical plane will intersect with the safety fence, so the number of intersections can be confirmed as the number of vertical planes.

[0083] In this embodiment, the center point of the safety fence is obtained by determining the spacing between adjacent vertical planes based on the lateral overlap rate and the camera parameters, and determining the direction of the vertical plane based on the route heading. The number of vertical planes is determined based on the center point, spacing and direction. In this way, when the drone flies along the terrain route corresponding to the determined number and direction, the corresponding real-life image captured can completely cover the area corresponding to the real-life model data.

[0084] As mentioned above, when determining the number of vertical planes, the number of intersection lines between the vertical plane and the safety fence can be used to determine it. Specifically, starting from the center point, extend horizontally along the preset direction toward both ends of the center point to obtain multiple vertical points, and the distance between the vertical points is the spacing. For the vertical points, the direction of the selected plane is determined based on the direction of the route, and the selected plane is determined based on the vertical point and the direction. If there is an intersection line between the selected plane and the safety fence, the number of intersection lines is determined as the number of vertical planes. Among them, the projection of the selected plane on the safety fence is a straight line. By determining the number of intersection lines as the number of vertical planes, the accuracy of terrain route planning can be improved.

[0085] When determining the spacing between adjacent vertical planes, it can be determined based on the camera parameters of the drone. In some embodiments, the camera parameters may include image resolution, sensor size, or lens focal length. And the spacing can be obtained based on the image resolution, sensor size, and lens focal length. Figure 4 As shown, the step of determining the spacing between adjacent vertical planes based on the lateral overlap ratio and the imaging parameters includes steps 402 to 406, wherein:

[0086] Step 402 : determining an equivalent focal length based on the sensor size and the lens focal length.

[0087] Step 404 : determining the horizontal distance of the ground range of the photographed real scene model based on the equivalent focal length, the image resolution and the safety distance.

[0088] Step 406, determining the spacing based on the horizontal distance and the lateral overlap ratio.

[0089] Among them, sensor size refers to the area size of the drone's sensor photosensitive device. Lens focal length refers to the distance between the lens's optical rear principal point and the focal point, and is an important indicator of the lens. Image resolution refers to the amount of information stored in an image.

[0090] Exemplarily, the equivalent focal length (35mm equivalent focal length) is first determined based on the sensor size and the focal length of the lens, and then the ground range that the camera of the drone can capture is obtained. In this embodiment, the horizontal distance can be selected to obtain the ground range. The ground range that the camera of the drone can capture can be obtained by the distance between the drone and the obstacle, that is, the safety distance, as well as the image resolution and the equivalent focal length. For example, the horizontal distance can be obtained based on the product of the pixel value in the horizontal direction of the safety distance and the image resolution and the ratio between the equivalent focal length. The spacing is then determined based on the horizontal distance and the lateral overlap rate. When the lateral overlap rate is zero, that is, when the overlap rate between the images taken by the drone in different flight segments of the real scene is zero, the distance between the vertical planes is the horizontal distance, that is, the spacing is the horizontal distance. Optionally, the distance between the vertical planes can be obtained by the formula f=d*(1-r), where f is the spacing, d is the horizontal distance, and r is the lateral overlap rate.

[0091] By obtaining the spacing based on the lateral overlap rate and camera parameters including image resolution, lens focal length, and sensor size, the overlap rate between images taken by the UAV on the real scene in the flight segment obtained based on the vertical plane can reach the standard, thereby improving the quality of the captured images and improving the efficiency of UAV shooting.

[0092] Through the above steps, the repetition rate of images taken by the drone between different flight segments can be reduced to improve the image quality. For the images taken by the drone at multiple waypoints in the same flight segment, there may be too much overlap. Therefore, after obtaining the flight segment based on the intersection line, in order to reduce the overlap rate of images taken by the drone in the same flight segment, the waypoints on each flight segment can be judged separately. If the image corresponding to a certain waypoint has a high similarity with the image corresponding to the previous or previous waypoints, the waypoint can be removed. Figure 5 As shown, after obtaining the flight segment based on the intersection of the vertical plane and the safety fence, steps 502 to 508 are included. Among them:

[0093] Step 502: If the distance between the waypoint that the drone passes first and the waypoint that the drone passes later is less than a preset distance, the waypoint that the drone passes later is determined to be a redundant waypoint.

[0094] Step 504: If the angle between the line connecting the waypoint that the drone passes first and the waypoint that the drone passes later and the horizontal direction is less than a preset angle, the waypoint that the drone passes later is determined to be a redundant waypoint.

[0095] Step 506, remove redundant waypoints from the route to determine the flight segment from which the redundant waypoints are removed.

[0096] Step 508, obtaining a terrain route based on the flight segment with redundant waypoints removed.

[0097] In this embodiment, for a single flight segment, redundant waypoints on the flight segment are determined based on the distances between the waypoints on the flight segment and the angles between the waypoints, and then the redundant waypoints are removed from the route. The overlap rate of the captured images corresponding to the waypoints on the terrain route obtained in this way is low, which can reduce the camera operation time and thus reduce the processing pressure of the drone.

[0098] For example, when judging the waypoints on the route to determine redundant waypoints, if the distance between the waypoint that the drone passes first and the waypoint that the drone passes later is less than a preset distance, the waypoint that the drone passes later is determined to be a redundant waypoint. If the angle between the line between the waypoint that the drone passes first and the waypoint that the drone passes later and the horizontal direction is less than a preset angle, the waypoint that the drone passes later is determined to be a redundant waypoint. After determining the redundant waypoints on the route, the redundant waypoints are removed from the route to obtain a segment from which the redundant waypoints are removed. At this time, the overlap rate between the real-life images taken by the drone at each waypoint on the segment is low. Thereafter, a terrain route can be obtained based on the segment from which the redundant waypoints are removed. In some embodiments, the segments from which the redundant waypoints are removed can be sequentially connected to obtain a terrain route (such as Figure 2 as shown).

[0099] In this embodiment, by eliminating redundant waypoints on the flight segment, the distribution of waypoints on the flight segment can be reasonably optimized, the operation efficiency of the drone can be accelerated, and the operation cost of the drone can be reduced.

[0100] In one possible implementation, Figure 6 As shown, the step of generating a safety fence based on the real scene model data and the safety distance may include steps 602 to 608. Among them:

[0101] Step 602: Add point indexes to the real scene model data.

[0102] By adding point indexes to divide the real-scene model into blocks, that is, adding a suitable index to the points of the real-scene model and being able to sort them, a method similar to binary search can be used for fast query. In one implementation method, the GeoHash spatial index method can be used to add point indexes to the real-scene model data.

[0103] Step 604: Obtain a tree data structure and a height map data structure according to the real scene model data and the point index.

[0104] Specifically, this embodiment constructs a data model for obstacles in the real scene, and needs to construct a tree data structure and a height map data structure. The tree data structure can construct a point index structure, and the height map data structure can reflect the height of the obstacle, so as to detect whether the drone route interferes with the boundary of the safety fence when planning the drone route, so that the drone does not collide with the building during flight, ensuring the flight safety of the drone. In one implementation, the steps of obtaining the tree data structure and the height map data structure according to the real scene model data and the point index can be implemented by the following steps: preset the maximum recursive depth. According to the real scene model data, a parent cube is established, and the point index is placed in the parent cube that can be contained and has no child nodes in sequence. According to the maximum recursive depth, the parent cube is recursively divided using the octree method to obtain the index relationship of the child cube. According to the index relationship of the child cube, the tree data structure and the octree are obtained. According to the real scene model data, the outer layer of the real scene model is evenly cut into a specified number of cells of the same size, and the height coordinate and the horizontal axis coordinate of each cell are obtained. According to the height coordinate and the horizontal axis coordinate of each cell, the height map data structure is obtained.

[0105] In some embodiments, according to the maximum recursive depth, the cube is recursively divided using the octree method to obtain the index relationship of the child cubes, and the step can be achieved by the following steps: if the recursive division does not reach the maximum recursive depth, the parent cube is subdivided into eight equal parts to obtain eight child cubes; the point indexes in the parent cube are allocated to the eight child cubes according to the containment relationship to obtain child cubes with containment index relationships; the allocation relationship of each child cube with containment index relationship is determined to obtain an allocation result; according to the allocation result, the parent cube is continued to be recursively divided until the maximum recursive depth is reached to obtain the index relationship of the child cubes.

[0106] Specifically, in this embodiment, the octree divides the geometric entity of the parent cube into voxels, each voxel has the same time and space complexity, and divides the parent cube in three-dimensional space through a cyclic recursive division method, thereby forming a direction graph with a root node. In the octree structure, if the divided voxels have the same attributes, the voxel constitutes a leaf node; otherwise, the voxel continues to be divided into 8 sub-cubes, and the division is recursive in sequence. In this way, all point indexes are assigned to the sub-cubes, forming an index relationship of the sub-cubes. The obstacle surface in the real scene data can be modeled through the index relationship of the sub-cubes.

[0107] Step 606: Obtain a height map data set according to the tree data structure and the height map data structure.

[0108] Exemplarily, a line segment is obtained according to the height coordinate of each cell and a preset designated height. The line segment and the octree are intersected to obtain an intersection result. According to the intersection result, a height map data set is obtained.

[0109] Specifically, the first cell A and a point B determined by the coordinate height plus the specified height are connected into a line segment, and the intersection is judged with the octree to obtain a height map dataset. Among them, the specified height is preset and is the safe distance between the drone and the obstacle.

[0110] Step 608: Generate a safety fence based on the height map data set.

[0111] Exemplarily, a point set of the safety fence is obtained according to the height map data set. The point set of the safety fence is triangulated to obtain a surface set of the safety fence. The safety fence is generated according to the surface set of the safety fence.

[0112] Specifically, in this embodiment, the finite difference method triangulation meshing, the CVT algorithm-based triangulation meshing, and the gradual insertion triangulation meshing method can be used to connect the acquired safety fence point set into a safety fence surface set.

[0113] The safety fence face set in this embodiment not only considers the height of obstacles in the real scene model, but also reserves a safety distance and generates structural data for the index points of obstacles. Finally, the obstacles in the real scene model data are wrapped by the safety fence face set, so that the drone does not break through the safety fence during route planning to avoid collision, ensure the flight safety of the drone, and obtain a safety fence that keeps a safe distance from obstacles, which is conducive to a terrain route consistent with the terrain features.

[0114] It is understandable that if the terrain route obtained in the embodiment of the present application can be converted into coordinates in the world coordinate system before application, the drone can then fly according to the converted terrain route to ensure the accuracy of the drone's moving route.

[0115] In an exemplary embodiment, Figure 7 As shown, the drone can obtain the real-life model data, and generate a safety fence based on the real-life model data and the safety distance. To obtain the terrain route, the vertical plane corresponding to the safety fence can be determined based on the preset parameters. The preset parameters may include the lateral overlap rate, the route direction, and the camera parameters. Therefore, the spacing of the vertical planes can be determined based on the lateral overlap rate, the route direction, and the camera parameters. After determining the spacing of the vertical planes, the number of vertical planes corresponding to the safety fence can be obtained, and then the vertical planes corresponding to the number can be obtained. Optionally, the center point of the safety fence is first determined. Starting from the center point, extend horizontally along the preset direction to both ends of the center point to obtain multiple vertical points, and the distance between the vertical points is the spacing. For the vertical point, the direction of the selected plane is determined based on the route direction, and the selected plane is determined based on the vertical point and the direction. If there is an intersection line between the selected plane and the safety fence, the number of intersection lines is determined as the number of vertical planes. Among them, the projection of the selected plane on the safety fence is a straight line. By determining the number of intersection lines as the number of vertical planes, the accuracy of terrain route planning can be improved.

[0116] When determining the spacing between vertical planes based on preset camera parameters and lateral overlap rate, the equivalent focal length can be determined based on the sensor size and lens focal length, and then the ground range that the camera of the drone can shoot is obtained. In this embodiment, the horizontal distance can be selected to obtain the ground range. The ground range that the camera of the drone can shoot can be obtained by the distance between the drone and the obstacle, that is, the safety distance, and the image resolution and the equivalent focal length. For example, the horizontal distance can be obtained based on the product of the pixel value in the horizontal direction of the safety distance and the image resolution and the ratio between the equivalent focal length. The spacing is then determined based on the horizontal distance and the lateral overlap rate. When the lateral overlap rate is zero, that is, the overlap rate between the images taken by the drone in different flight segments of the real scene is zero, the distance between the vertical planes is the horizontal distance, that is, the spacing is the horizontal distance. Optionally, the distance between the vertical planes can be obtained by the formula f=d*(1-r), where f is the spacing, d is the horizontal distance, and r is the lateral overlap rate.

[0117] After determining each vertical plane, determine the flight segment based on the intersection line of the vertical plane and the safety fence. In order to reasonably arrange the waypoints on the flight segment, the waypoint is the location where the drone needs to shoot the real scene. The waypoints on the flight segment can be screened based on the limit distance and limit angle to filter out redundant waypoints. The distance between the waypoints on the route must be greater than or equal to the limit distance. The angle between the line between two adjacent waypoints on the route and the horizontal line must be greater than or equal to the limit angle. Optionally, if the distance between the waypoint that the drone passes first and the waypoint that the drone passes later is less than a preset distance, the waypoint that the drone passes later is determined to be a redundant waypoint. If the angle between the line between the waypoint that the drone passes first and the waypoint that the drone passes later and the horizontal direction is less than a preset angle, the waypoint that the drone passes later is determined to be a redundant waypoint. The redundant waypoints are then removed from the flight segment to obtain a final terrain route.

[0118] Through the above steps, the terrain route can be made to approximate the terrain features of the real scene, and the distance between the waypoints and the corresponding ground observation points can be consistent, and the height changes with the terrain. In addition, the distribution of waypoints between routes is reasonably optimized, the operating efficiency of drones and other flight equipment is accelerated, and the operating costs of drones and other flight equipment are reduced.

[0119] It should be understood that, although the steps in the flowcharts involved in the above embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0120] Based on the same inventive concept, the embodiment of the present application also provides a terrain route generation device based on a three-dimensional real scene for implementing the terrain route generation method based on a three-dimensional real scene involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations of one or more embodiments of the terrain route generation device based on a three-dimensional real scene provided below can refer to the limitations of the terrain route generation method based on a three-dimensional real scene above, and will not be repeated here.

[0121] In an exemplary embodiment, Figure 8As shown, a terrain route generation device 800 based on a three-dimensional real scene is provided, comprising: an acquisition module 801, a plane determination module 802 and a route determination module 803, wherein:

[0122] The acquisition module 801 is used to obtain the real scene model data, and generate a safety fence based on the real scene model data and the safety distance. The safety fence is used to cover obstacles in the real scene model to prevent the drone from colliding with the obstacles.

[0123] The plane determination module 802 is used to obtain the number and direction based on preset parameters, and obtain the vertical plane corresponding to the safety fence based on the number and the direction. The preset parameters are used to configure the drone, and the vertical plane intersects with the safety fence.

[0124] The route determination module 803 is used to determine the intersection of the vertical plane and the safety fence as a flight segment, so as to obtain a terrain route based on the flight segment.

[0125] In some embodiments, the plane determination module 802 is also used to determine the spacing between adjacent vertical planes based on the lateral overlap rate and camera parameters; determine the direction of the vertical plane based on the route heading, obtain the center point of the safety fence, and determine the number of vertical planes based on the center point, spacing and direction.

[0126] In some embodiments, the plane determination module 802 is also used to take the center point as the starting point, extend horizontally toward both ends of the center point along a preset direction to obtain multiple vertical points, and the distance between the vertical points is the spacing; for the vertical points, the direction of the selected plane is determined based on the direction of the route, and the selected plane is determined based on the vertical point and the direction; if there is an intersection line between the selected plane and the safety fence, the number of intersection lines is determined as the number of vertical planes.

[0127] In some embodiments, the plane determination module 802 is further used to determine the equivalent focal length based on the sensor size and the lens focal length; determine the horizontal distance of the ground range of the photographed real scene model based on the equivalent focal length, image resolution and safety distance; and determine the spacing based on the horizontal distance and the lateral overlap rate.

[0128] In some embodiments, the route determination module 803 is further used to determine that the waypoint that the drone passes through first and the waypoint that the drone passes through later are redundant waypoints if the distance between the waypoint that the drone passes through first and the waypoint that the drone passes through later is less than a preset distance; determine that the waypoint that the drone passes through later is redundant waypoints if the angle between the line between the waypoint that the drone passes through first and the waypoint that the drone passes through later and the horizontal direction is less than a preset angle; remove the redundant waypoints from the route to determine the flight segment from which the redundant waypoints are removed. A terrain route is obtained based on the flight segment from which the redundant waypoints are removed.

[0129] In some embodiments, the acquisition module 801 is also used to add point indexes to the real-life model data; obtain a tree data structure and a height map data structure based on the real-life model data and the point index; obtain a height map data set based on the tree data structure and the height map data structure; and generate a safety fence based on the height map data set.

[0130] Each module in the above-mentioned terrain route generation device based on three-dimensional real scene can be fully or partially implemented by software, hardware and their combination. Each of the above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the corresponding operations of each of the above modules.

[0131] In an exemplary embodiment, an electronic device is provided, which may be a drone, and its internal structure diagram may be as shown in FIG. Fig. 9 As shown. The electronic device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. The processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the electronic device is used to exchange information between the processor and an external device. The communication interface of the electronic device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, near field communication (NFC) or other technologies. When the computer program is executed by the processor, a terrain route generation method is implemented. The display unit of the electronic device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the electronic device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the electronic device casing, or an external keyboard, touchpad or mouse.

[0132] Those skilled in the art will understand that Fig. 9 The structure shown in the figure is merely a block diagram of a partial structure related to the scheme of the present application, and does not constitute a limitation on the electronic device to which the scheme of the present application is applied. The specific electronic device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0133] In an exemplary embodiment, an electronic device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0134] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0135] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0136] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0137] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0138] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for generating a terrain route based on a three-dimensional real scene, characterized in that: The method comprises: Acquire real-scene model data, and generate a safety fence based on the real-scene model data and a safety distance, wherein the safety fence is used to cover obstacles in the real-scene model so that the drone does not collide with the obstacles; Determine the spacing between adjacent vertical planes based on the lateral overlap rate and the camera parameters, wherein the lateral overlap rate is the probability that the shooting ranges of the drone overlap between different flight segments; Determine the direction of the selected plane based on the route direction, obtain the center point of the safety fence, and determine the number of intersections between the selected plane and the safety fence based on the center point, the distance and the direction, wherein the route direction is the driving direction of the UAV; A vertical plane corresponding to the safety fence is obtained based on the number and the direction, and the vertical plane intersects with the safety fence; a flight segment is obtained based on an intersection line between the vertical plane and the safety fence, so as to obtain a terrain route based on the flight segment.

2. The method according to claim 1, characterized in that The camera parameters include a camera shooting range of the drone, and the method further includes: The distance is determined based on the camera shooting range and the lateral overlap ratio.

3. The method according to claim 1, characterized in that The determining the number based on the center point, the distance and the direction comprises: Taking the center point as the starting point, horizontally extending along a preset direction toward both ends of the center point to obtain a plurality of vertical points, wherein the distance between the vertical points is the spacing; For the vertical point, determining the direction of a candidate plane based on the direction of the route, and determining the candidate plane based on the vertical point and the direction; If there are intersection lines between the selected plane and the safety fence, the number of the intersection lines is determined to be the number.

4. The method according to claim 1, characterized in that: The camera parameters include image resolution, sensor size, and lens focal length. The determining the spacing between adjacent vertical planes based on the lateral overlap rate and the camera parameters includes: Determining an equivalent focal length based on the sensor size and the lens focal length; Determine the horizontal distance of the ground range for photographing the real scene model based on the equivalent focal length, the image resolution and the safety distance; The spacing is determined based on the horizontal distance and the lateral overlap ratio.

5. The method according to claim 1, characterized in that: There are multiple waypoints on the flight segment, and the waypoints are sorted according to the time when the drone passes through the waypoints. The flight segment is obtained based on the intersection line of the vertical plane and the safety fence, and then the following steps are performed: If the distance between the waypoint that the drone passes first and the waypoint that the drone passes later is less than a preset distance, then determining that the waypoint that the drone passes later is a redundant waypoint; If the angle between the line connecting the waypoint that the drone passes first and the waypoint that the drone passes later and the horizontal direction is less than a preset angle, the waypoint that the drone passes later is determined to be the redundant waypoint; Eliminating the redundant waypoints from the flight segment to determine the flight segment from which the redundant waypoints are eliminated; The obtaining of a terrain route based on the flight segment comprises: A terrain route is obtained based on the flight segment excluding the redundant waypoints.

6. The method according to claim 1, characterized in that The generating of the safety fence based on the real scene model data and the safety distance includes: Adding point indexes to the real scene model data; Obtaining a tree data structure and a height map data structure according to the real scene model data and point index; Obtaining a height map data set according to the tree data structure and the height map data structure; The safety fence is generated according to the height map dataset.

7. A terrain route generation device based on three-dimensional real scene, characterized in that: The device comprises: An acquisition module, used to obtain real-scene model data, and generate a safety fence based on the real-scene model data and a safety distance, wherein the safety fence is used to cover obstacles in the real-scene model so that the drone does not collide with the obstacles; A plane determination module is used to determine the spacing between adjacent vertical planes based on a lateral overlap rate and a camera parameter, wherein the lateral overlap rate is the probability that the shooting ranges of the drone overlap between different flight segments; determine the direction of a candidate plane based on the route orientation, obtain the center point of the safety fence, and determine the number of intersection lines between the candidate plane and the safety fence based on the center point, the spacing, and the direction, wherein the route orientation is the driving direction of the drone; obtain a vertical plane corresponding to the safety fence based on the number and the direction, wherein the vertical plane intersects with the safety fence; The route determination module is used to obtain a flight segment based on an intersection line of the vertical plane and the safety fence, so as to obtain a terrain route based on the flight segment.

8. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images