Low-altitude route planning method and system based on image processing

Abstract

The invention relates to the technical field of image processing, and provides a low-altitude route planning method and system based on image processing, and the method comprises the steps: obtaining an urban GIS model, point cloud data and a remote sensing image, generating an urban environment high-precision three-dimensional model, and obtaining an urban discrete graph; acquiring a plurality of neighborhood unmanned aerial vehicles of the target unmanned aerial vehicle at the current node; obtaining the dynamic interaction strength and static interaction strength of the target unmanned aerial vehicle and each neighborhood unmanned aerial vehicle in the verification flight stage, and further obtaining the interaction strength of the target unmanned aerial vehicle at the current node; judging whether the target unmanned aerial vehicle needs to adjust a preset node at the current node through the threshold value, and obtaining a plurality of candidate nodes of the target unmanned aerial vehicle at the current node; and acquiring a flight target node of the target unmanned aerial vehicle at the current node to realize low-altitude route real-time planning of the unmanned aerial vehicle. The method aims at solving the problem that unmanned aerial vehicle low-altitude route planning does not consider task scheduling interference of other unmanned aerial vehicles and is difficult to adapt to dynamic environment changes.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to a low-altitude air route planning method and system based on image processing. BACKGROUND

[0002] In the research of low-altitude unmanned aerial vehicle air route planning, the existing technology usually relies on the combination of image processing and path planning algorithm to realize obstacle avoidance and flight path generation; through the environment image obtained by airborne or ground, the terrain features and obstacle information in the flight area are extracted by using image processing method, and are converted into map data which can be used for path planning, on this basis, the RRT algorithm is used to plan the air route of unmanned aerial vehicle.

[0003] However, in the actual low-altitude environment, the unmanned aerial vehicle often needs to face the dynamic and variable flight environment, such as path interference caused by other unmanned aerial vehicle task scheduling, etc., simply relying on the way of generating leaf nodes by random sampling cannot fully capture these dynamic changes, resulting in that the generated air route may lag behind the actual situation of the environment, ignoring the interaction risk between multiple unmanned aerial vehicles under real-time changes of the environment, and further affecting the safety and adaptability of the low-altitude planning air route. SUMMARY

[0004] The present application provides a low-altitude air route planning method and system based on image processing, to solve the problem that the existing unmanned aerial vehicle low-altitude air route planning does not consider the interference of other unmanned aerial vehicle task scheduling and is difficult to adapt to dynamic environmental changes, the technical scheme adopted is as follows: The present application provides a low-altitude air route planning method and system based on image processing, to solve the problem that the existing unmanned aerial vehicle low-altitude air route planning does not consider the interference of other unmanned aerial vehicle task scheduling and is difficult to adapt to dynamic environmental changes, the technical scheme adopted is as follows: Obtain city GIS model, point cloud data and remote sensing image, generate city environment high-precision three-dimensional model, and then obtain city discrete graph; Generate a preset planning air route for the target unmanned aerial vehicle based on the city discrete graph, obtain a plurality of neighborhood unmanned aerial vehicles of the target unmanned aerial vehicle at the current node in combination with the neighborhood performance of time domain and space domain, and record the verification flight stage of the target unmanned aerial vehicle from the current node to the preset node; analyze the position difference changes of the same feature points in the shooting images of the target unmanned aerial vehicle and each neighborhood unmanned aerial vehicle in the verification flight stage, and obtain the dynamic interaction intensity of the target unmanned aerial vehicle and each neighborhood unmanned aerial vehicle in the verification flight stage; Based on the similarity relationship between the shooting images of the target unmanned aerial vehicle and each neighborhood unmanned aerial vehicle in the verification flight stage, obtain the static interaction intensity of the target unmanned aerial vehicle and each neighborhood unmanned aerial vehicle in the verification flight stage, and then obtain the interaction intensity of the target unmanned aerial vehicle at the current node; judge whether the target unmanned aerial vehicle needs to adjust the preset node at the current node through threshold value, and obtain a plurality of candidate nodes of the target unmanned aerial vehicle at the current node in combination with the interaction intensity and the direction of the current node to the preset node. The similarity relationship between the remote sensing image of the candidate node and the remote sensing image of the preset node is analyzed, and a city discrete graph is combined to obtain a flight target node of the target UAV at the current node.

[0005] Optionally, the method for generating the preset planned route for the target UAV based on the city discrete graph comprises the following specific method: The current UAV to be subjected to low-altitude route planning is taken as the target UAV, and a fast expanding random tree algorithm is used to generate a preset planned route for the target UAV based on the city discrete graph.

[0006] Optionally, the method for obtaining the several neighbor UAVs of the target UAV at the current node and recording a verification flight phase of the target UAV from the current node to the preset node comprises the following specific method: Each point in the preset planned route of the target UAV is taken as a node, including a starting point, an ending point and a point where the flight direction changes in the preset planned route, the node where the target UAV is located at the current time is taken as the current node, and the next node of the current node in the preset planned route is taken as the preset node; The voxel where the current node is located in the city discrete graph is taken as the neighbor airspace of the target UAV at the current node, a neighbor time range before and after the current time is taken as the neighbor time range of the current time, and several UAVs whose flight positions are located in the neighbor airspace of the current node in the neighbor time range of the current time are taken as the neighbor UAVs of the target UAV; The distance of the target UAV from the current node to the preset node is obtained, a preset verification ratio is obtained, the distance of the target UAV from the current node to the preset node within the verification ratio is taken as the verification flight phase of the target UAV from the current node to the preset node, the verification flight time length of the target UAV at the current node is obtained based on the verification flight phase and the flight speed of the UAV, and the verification flight time range of the target UAV at the current node is obtained in combination with the current time.

[0007] Optionally, the method for obtaining the dynamic interaction intensity between the target UAV and each neighbor UAV in the verification flight phase comprises the following specific method: Several images taken by the target UAV in the verification flight time range of the current node are taken as the several collected images of the target UAV in the verification flight phase, the several collected images of any neighbor UAV of the target UAV in the verification flight phase of the target UAV are obtained, for any collected image of the target UAV in the verification flight phase, the collected image corresponding to the neighbor UAV is obtained, wavelet denoising and SIFT feature point detection are performed on the two collected images respectively to obtain several feature points of the two collected images, the approximate nearest neighbor search algorithm is used to match the feature points in the two collected images to obtain several feature point pairs in the two collected images, and the two feature points in the feature point pair are taken as the same feature points in the images taken by different UAVs. based on the position difference variation of the same feature points in the collected images of the adjacent time points of the target UAV and the neighbor UAVs in the verification flight time range of the current node obtain the convergence degree of each same feature point at each time point in the verification flight time range; obtain the convergence degree of each same feature point at each time point in the verification flight time range; obtain the convergence degree of each same feature point at each time point in the verification flight time range;

[0008] Optionally, the specific method for obtaining the convergence degree of each same feature point at each time point in the verification flight time range comprises: for the target UAV and the neighbor UAVs at any same feature point corresponding to the two collected images at time points , the relative position of the feature point at time point is calculated as follows:

[0009] wherein, represents the coordinate of the feature point in the collected image of the target UAV at time point represents the coordinate of the feature point in the collected image of the neighbor UAV of the target UAV at time point the relative motion trend of the same feature point in the consecutive collected images is obtained by the optical flow method, and the calculation method of the relative motion trend of the feature point at time point is as follows:

[0010] wherein, represents the relative position of the feature point at time point , and represents the relative position of the feature point at time point ; obtain the product of the relative position and the relative motion trend, and take the ratio of the module length of the relative position to the product as the convergence factor of the feature point at time point ; obtain the convergence factor of the corresponding same feature point of each feature point pair in the verification flight time range of the target UAV at the current node, perform linear normalization on all convergence factors, and take the obtained result as the convergence degree of the corresponding feature point at each time point.

[0011] ​​Optionally, the specific method for obtaining the static interaction strength between the target UAV and each neighboring UAV during the verification flight phase is as follows: For any captured image of the target UAV during the verification flight phase, obtain the corresponding captured image of any neighboring UAV, obtain the number of feature point pairs between the two captured images, and the maximum number of feature points in each of the two captured images, and obtain the ratio of the number of feature point pairs to the maximum value; Obtain the distance between two feature points in each feature point pair, obtain the ratio of the mean of all distances to the maximum value of all distances, and multiply the difference obtained by subtracting the ratio from 1 and the ratio of the number of feature point pairs to the maximum value as the image similarity of the two captured images; The mean value of the image similarity between the target UAV and the neighboring UAV at all times within the verification flight time range is taken as the static interaction intensity between the target UAV and the neighboring UAV during the verification flight phase.

[0012] Optionally, the specific method of obtaining the interaction strength of the target UAV at the current node includes: The product of the static interaction intensity and the dynamic interaction intensity of the target UAV with the neighboring UAVs during the verification flight phase is taken as the comprehensive interaction intensity of the target UAV with the neighboring UAVs during the verification flight phase; the average of the comprehensive interaction intensity of the target UAV with each neighboring UAV during the verification flight phase is taken as the interaction intensity of the target UAV at the current node.

[0013] Optionally, the specific method of obtaining several candidate nodes of the target UAV at the current node includes: A candidate leaf node is randomly generated at the position corresponding to the current node. The direction of each candidate leaf node is from the current node to the corresponding candidate leaf node. The interaction strength is used as the modulus. Combined with the direction of the candidate leaf node, the interaction vector of each candidate leaf node is obtained. Based on the interaction strength and the direction of the current node to the preset node, the interaction strength vector is constructed. Obtain a projection component of the interaction vector of each candidate leaf node on the interaction strength vector, obtain a ratio of the projection component to the interaction strength, obtain a difference obtained by subtracting the ratio from 1, and multiply the difference by the angle between the direction of the candidate leaf node and the direction of the current node pointing to the preset node, and obtain a corrected direction of the candidate leaf node based on the direction of the current node pointing to the preset node and the product; Based on the corrected direction of the candidate leaf node and the distance between the current node and the preset node, the position corresponding to the candidate leaf node is obtained and used as a candidate node for the current node.

[0014] Optionally, the target unmanned aerial vehicle is a flight target node of the current node, and the specific acquisition method is as follows: For any candidate node of the target unmanned aerial vehicle at the current node, a remote sensing image of the candidate node is acquired through remote sensing, and the remote sensing image is compared with a remote sensing image corresponding to the preset node; a structural similarity index SSIM is used to quantify the similarity of the two remote sensing images as an environmental invariant factor of the candidate node; Based on the city discrete graph, whether there is an obstacle between the current node and the candidate node is acquired, and if there is, it is marked as 0, and if not, it is marked as 1, and is recorded as an obstacle factor of the candidate node; The product of the environmental invariant factor and the obstacle factor is used as the preferred degree of the candidate node, and the candidate node corresponding to the maximum preferred degree among all candidate nodes is used as the flight target node of the target unmanned aerial vehicle at the current node.

[0015] The application also provides a low-altitude air route planning system based on image processing, which comprises a memory, a processor, and a computer program stored in the memory and running on the processor, and the processor implements the steps of the above method when executing the computer program.

[0016] The application has the following advantages: based on the traditional unmanned aerial vehicle low-altitude air route planning based on the city three-dimensional model, the leaf nodes generated by the RRT algorithm are combined with the interaction information between unmanned aerial vehicles for guidance; wherein the preset planning route is generated, and the target unmanned aerial vehicle at the current time corresponding to the current node and the preset node of the next node on the route are verified and flown, the perspective change between the feature points of the neighboring unmanned aerial vehicle and the target unmanned aerial vehicle is considered, and the perspective change difference of the feature points in different unmanned aerial vehicles can reflect whether the two unmanned aerial vehicles are intersecting, and then the dynamic interaction relationship between the target unmanned aerial vehicle and the neighboring unmanned aerial vehicle in the verification flight stage is quantitatively verified; through the image similarity relationship between the two unmanned aerial vehicles in the verification flight stage, the interaction relationship between the two unmanned aerial vehicles as a whole is further analyzed, and the interaction intensity of the target unmanned aerial vehicle at the current node is obtained, which is combined with the preset node to generate candidate nodes, adjust the flight direction of the unmanned aerial vehicle to reduce the interaction intensity; and the optimal candidate node is selected through the evaluation mechanism as the flight target node to realize the dynamic correction and optimization of the route, which can improve the safety and adaptability of the route, and make the unmanned aerial vehicle group have stronger robustness and cooperativeness in the complex low-altitude environment. BRIEF DESCRIPTION OF DRAWINGS

[0017] In order to make the technical solutions in the embodiments of the present application or the prior art clearer, the accompanying drawings needed in the embodiments or prior art description will be briefly introduced. Obviously, the accompanying drawings in the following description only only some embodiments of the present application, and for those skilled in the art, other drawings can be obtained without creative labor under the premise of the accompanying drawings.

[0018] Figure 1 The flowchart of the low-altitude route planning method based on image processing provided by an embodiment of the present application; Figure 2 The city discrete graph provided by the present application; Figure 3 The schematic diagram of the position difference of the same feature point for two unmanned aerial vehicles. DETAILED DESCRIPTION

[0019] The technical solutions in the embodiments of the present application will be clearly and completely described with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments only only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of the present application.

[0020] Please refer to Figure 1 which shows the flowchart of the low-altitude route planning method based on image processing provided by an embodiment of the present application. The method comprises the following steps: Step S001, obtaining city GIS model, point cloud data and remote sensing image, generating city environment high-precision three-dimensional model, and then obtaining city discrete graph.

[0021] It should be noted that the city environment data required for low-altitude route planning is mainly from various channels to ensure coverage and accuracy; static terrain and building information is obtained through city geographic information system (GIS) database, including building contour, elevation data, road network and green land distribution, etc.; city low-altitude airspace restriction and no-fly zone information comes from public data of civil aviation management department and city planning department; in order to capture dynamic obstacles and real-time environment changes, unmanned aerial vehicle low-altitude aerial image and traffic monitoring data are also needed; these data are different in spatial resolution, time update frequency and data format, so it is necessary to integrate through unified coordinate system and time mark.

[0022] Specifically, obtain the city GIS model and the city aerial image, obtain the city point cloud data through the laser radar, and obtain the city remote sensing image, filter and remove the noise of the point cloud data and the remote sensing image, and use the point cloud clustering and object segmentation algorithm to extract key features such as buildings, roads, and trees; resample and vectorize the GIS model and the remote sensing image so that their geometric information corresponds to the point cloud data; then, through the spatial alignment and fusion algorithm, uniformly map the three types of data to the same three-dimensional coordinate system to form a high-precision three-dimensional model of the urban environment. The point cloud clustering and object segmentation algorithm, as well as the spatial alignment and fusion algorithm, are all existing technologies in the three-dimensional model construction process and will not be repeated in this embodiment.

[0023] Furthermore, the voxelization method is used to divide the continuous urban space into small cubes with the same side length. ,like Figure 2 As shown in the figure, each cubic unit (voxel) is marked as passable or impassable according to its occupancy status; the passable area is the space where drones can fly, which is the white square in the figure; the impassable area corresponds to buildings, obstacles or no-fly airspace, which is the black square in the figure; the city discrete map is obtained.

[0024] Furthermore, drawing on the practice of setting discrete waypoints for aircraft in the civil aviation air traffic control system, the waypoints for drones are also set in discrete form, and can only be selected from the vertices of the small white cubes in the figure. It is also stipulated that the drone can only fly in a straight line between two adjacent waypoints. The drone can choose an edge, a face diagonal or a body diagonal of the cube as the next flight segment.

[0025] Step S002: Generate a preset planned route for the target UAV based on the city scatter map, obtain several neighboring UAVs of the target UAV at the current node by combining the proximity performance in the time domain and airspace, and record the verification flight phase of the target UAV from the current node to the preset node; analyze the position difference changes of the same feature points in the images taken by the target UAV and each neighboring UAV during the verification flight phase, and obtain the dynamic interaction intensity of the target UAV with each neighboring UAV during the verification flight phase.

[0026] Preferably, in one embodiment of the present invention, a preset planned route is generated for the target UAV based on the city scatter graph, and several neighboring UAVs of the target UAV at the current node are obtained by combining the proximity performance in the time domain and airspace, and the verification flight phase of the target UAV from the current node to the preset node is recorded. The specific method includes: The UAV currently undergoing low-altitude route planning is taken as the target UAV. Based on the city discrete graph, the rapidly expanding random tree (RRT) algorithm is used to generate a preset planned route for the target UAV. That is, the algorithm first takes the starting point as the root node and gradually randomly expands the tree structure in the discrete space. During the expansion process, obstacle areas are dynamically detected to prevent the path from entering the non-flyable area. As the iteration proceeds, the RRT gradually generates a feasible path from the starting point to the target point, which is used as the preset planned route.

[0027] It should be noted that when performing a mission, the UAV will fly and operate strictly according to the preset route to ensure the feasibility of path planning and the coverage integrity of the operation area; however, in actual low-altitude environments, UAVs often need to face dynamic and changeable flight environments, such as path interference caused by other UAV task scheduling, which makes the preset planned route inaccurate and may cause flight accidents; in addition, in traditional RRT algorithms, the generation of leaf nodes is usually achieved by expanding in a random direction. In this process, the algorithm does not consider the existence and behavior of other UAVs in the same mission environment, that is, the generation of each node is independent of the single machine, ignoring the impact of multi-UAV collaboration or potential interaction, which may cause path conflicts or poor efficiency in actual flight.

[0028] Furthermore, each point in the preset planned route of the target UAV is taken as a node, including the starting point, the end point and the point where the flight direction changes in the preset planned route. The node where the target UAV is currently located is taken as the current node, and the next node of the current node in the preset planned route is taken as the preset node; the voxel where the current node is located in the city discrete map is taken as the neighborhood airspace of the target UAV at the current node, and the neighborhood time range is preset. In this embodiment, the neighborhood time range adopts The neighborhood time range before and after the current moment is described in seconds, and the neighborhood time range before and after the current moment is regarded as the neighborhood time domain of the current moment; in the neighborhood time domain of the current moment, several drones whose flight positions are located in the neighborhood airspace of the current node are regarded as the neighborhood drones of the target drone.

[0029] Furthermore, the distance of the target UAV from the current node to the preset node is obtained, and a verification ratio is preset. In this embodiment, the verification ratio is described as 1 / 10. The distance of the target UAV from the current node to the preset node before the verification ratio is used as the verification flight stage of the target UAV from the current node to the preset node. Based on the verification flight stage and the flight speed of the UAV, the verification flight duration of the target UAV at the current node is obtained. Then, combined with the current moment, the verification flight time range of the target UAV at the current node is obtained.

[0030] It is required to be explained that a section of the path when the target UAV flies from the current node to the preset node is called the verification flight phase, and the purpose is to determine the feasibility of the preset node by using the image data collected in the phase, and the image data collected in the verification flight phase is used to analyze the interaction with the neighboring UAVs.

[0031] It is further required to be explained that the coupling effect between individuals due to the mutual influence of spatial position, flight path trend and task target in the multi-UAV flight process is called interaction, which is reflected in whether the flight paths are likely to collide or approach, and also reflected in the degree of overlap of the task area coverage; in order to quantify the size of the interaction, the interaction intensity is the degree of mutual influence between two UAVs reflected by the image feature position relationship; and the interaction intensity is jointly determined by dynamics and statics, if the position difference of the feature points from the same scene in the images collected at consecutive time points changes rapidly with time in the screens of UAV A and UAV B, it indicates that the trajectories of the two UAVs tend to converge, and the interaction intensity is larger; in statics, if the images collected by the two UAVs have high feature point overlap, and their spatial positions are relatively close, it also means that there is a strong spatial interference relationship between them.

[0032] Preferably, in an embodiment of the present application, the position difference change of the same feature points in the images taken by the target UAV and each neighboring UAV in the verification flight phase is analyzed to obtain the dynamic interaction intensity between the target UAV and each neighboring UAV in the verification flight phase, and the specific method includes: A plurality of images taken by the target UAV in the verification flight time range at the current node are obtained as a plurality of collected images of the target UAV in the verification flight phase; a plurality of collected images of any neighboring UAV of the target UAV in the verification flight phase of the target UAV are obtained (the sampling time intervals of image taking are consistent, and the starting sampling times are also the same, so the corresponding time points can be completely the same); for any collected image of the target UAV in the verification flight phase, the corresponding collected image of the neighboring UAV is obtained, and wavelet denoising and SIFT feature point detection are performed on the two collected images (both are existing technologies in the art of image processing, and will not be described here), a plurality of feature points of the two collected images are obtained, the feature points in the two collected images are matched by using the approximate nearest neighbor search algorithm, a plurality of feature point pairs in the two collected images are obtained, and the two feature points in the feature point pair are taken as the same feature points in the images taken by different UAVs.

[0033] It is required to be explained that when the UAV is flying, the images taken by the on-board camera will contain feature points in the environment, if two UAVs are far apart, the position difference of the feature points they take will be relatively stable; if two UAVs are gradually approaching, the relative positions of the common feature points they take in the images will change rapidly.

[0034] It is further necessary to explain that if Figure 3 As shown in the figure, two drones A and B are flying in the air. They have different directions, but they can both see the same feature point P on the ground. At a certain moment, P is on the left side of the screen in A's captured image, and on the right side of the screen in B's captured image. For a single captured image, the relative position Δp is very large. Then the two drones continue to fly, and each frame records the position of P in their respective captured images. As time goes by, A and B move along their respective trajectories, and the projection difference Δu continues to shrink in consecutive frames, indicating that the lines of sight of the two drones are moving towards the same point. The obvious convergence speed of Δu indicates that their spatial trajectories tend to approach or even intersect. Therefore, the convergence trend and direction of Δu can be used to quantify the dynamic interaction intensity between the target drone and the neighboring drones.

[0035] Specifically, for the target drone and its neighboring drones At the moment Any identical feature point corresponding to the two captured images of Relative position The calculation method is:

[0036] in, Indicates that the target drone is at time The coordinates of the feature point in the captured image, Indicates the neighboring drones of the target drone At the moment The coordinates of the feature point in the captured image; the same feature point in consecutive captured images is obtained by the optical flow method, then the feature point at time Relative movement trend The calculation method is:

[0037] in, Indicates that the feature point is at time The relative position of The feature point at time The relative position of the feature point at time t; Since the relative position and relative motion trend are actually expressed as coordinates, they can be regarded as vectors pointing from the coordinate origin to the corresponding coordinates, and the degree of convergence is calculated based on the vector; the product of the relative position and the relative motion trend (the product of the vectors) is obtained, and the ratio of the modulus of the relative position to the product is used as the feature point at time convergence factors of the corresponding same feature points of each feature point pair in the verification flight time range of the target UAV at the current node are obtained according to the above method, all the convergence factors are linearly normalized, and the obtained result is taken as the convergence degree of the corresponding feature points at each time; wherein the same feature point does not necessarily exist at all times in the verification flight time range, and the feature point matching at different times is specifically performed according to the optical flow method; the average of the convergence degrees of the corresponding feature points of all feature point pairs at all times in the verification flight time range is taken as the dynamic interaction intensity of the target UAV and the neighbor UAVs in the verification flight phase .

[0038] It should be noted that, according to the convergence analysis of the direction vector represented by the coordinates, the greater the difference between the relative motion trend and the relative position, the greater the relative motion trend, which will lead to a greater convergence degree, that is, a condition that is less consistent with a smaller change in the relative position, and then the convergence degree reflects the interaction intensity, so the greater the dynamic interaction intensity, and vice versa, the more dispersed the relative position, and the smaller the change in the relative motion trend, the smaller the convergence degree.

[0039] At this point, the dynamic interaction intensity of the target UAV and each neighbor UAV in the verification flight phase is obtained.

[0040] Step S003, based on the similarity relationship between the photographed images of the target UAV and each neighbor UAV in the verification flight phase, the static interaction intensity of the target UAV and each neighbor UAV in the verification flight phase is obtained, and then the interaction intensity of the target UAV at the current node is obtained; whether the target UAV needs to adjust the preset node at the current node is determined through a threshold value, and a plurality of candidate nodes of the target UAV at the current node are obtained in combination with the interaction intensity and the direction of the current node to the preset node.

[0041] It should be noted that when the number of feature points in the overlapping area of the images photographed by the two UAVs is large and the position consistency is high, it indicates that their positions in space are relatively close, and the observed scenes are highly overlapped; such high overlap not only reflects the proximity of the spatial positions, but also means that there may be a strong interaction or mutual influence between the two UAVs, that is, the static nature is strong.

[0042] Preferably, in an embodiment of the present application, based on the similarity relationship between the photographed images of the target UAV and each neighbor UAV in the verification flight phase, the static interaction intensity of the target UAV and each neighbor UAV in the verification flight phase is obtained, and then the interaction intensity of the target UAV at the current node is obtained, which includes the following specific method: For any acquisition image of the target UAV in the verification flight phase, the acquisition image corresponding to the acquisition image of any neighborhood UAV is obtained, the number of feature point pairs of the two acquisition images is obtained, and the maximum value of the number of feature points in the two acquisition images is obtained. The ratio of the number of feature point pairs to the maximum value is obtained; the distance between the two feature points in each feature point pair is obtained, the ratio of the average value of all distances (the distance between the two feature points in the same feature point pair) to the maximum value of all distances is obtained, and the product of the difference obtained by subtracting 1 from the ratio and the ratio of the number of feature point pairs to the maximum value is taken as the image similarity of the two acquisition images. The average value of the image similarity between the acquisition images corresponding to all time ranges of the target UAV and the neighborhood UAV in the verification flight time is taken as the static interaction strength of the target UAV in the verification flight phase with the neighborhood UAV.

[0043] Further, the product of the static interaction strength and the dynamic interaction strength of the target UAV in the verification flight phase with the neighborhood UAV is taken as the comprehensive interaction strength of the target UAV in the verification flight phase with the neighborhood UAV. The average value of the comprehensive interaction strength of the target UAV in the verification flight phase with each neighborhood UAV is taken as the interaction strength of the target UAV at the current node.

[0044] Preferably, in an embodiment of the present application, whether the target UAV needs to adjust the preset node at the current node is determined by a threshold value, and a plurality of candidate nodes of the target UAV at the current node are obtained in combination with the interaction strength and the direction of the current node to the preset node. The specific method comprises: A preset interaction threshold value is adopted in the embodiment, and the interaction threshold value is 0.4. If the interaction strength of the target UAV at the current node is greater than or equal to the interaction threshold value, the verification flight phase cannot meet the flight safety or UAV route separation requirements when reaching the preset node, and node expansion is required, that is, candidate nodes are obtained. If the interaction strength of the target UAV at the current node is less than the interaction threshold value, the preset node meets the requirements, and there is no need to adjust, and the target UAV flies to the preset node along the current node and performs subsequent route planning.

[0045] It should be noted that when generating a new leaf node, the original RRT algorithm generates in a random direction without considering the relationship with other UAVs, while the generation of the new candidate leaf node is based on the interaction distribution characteristics of the neighborhood UAV, and the interaction strength vector is introduced to dynamically correct the node generation direction.

[0046] It needs to be further explained that when the interaction intensity is large, it means that the target UAV has a high possibility of potential safety conflict with the neighbor UAV in the preset node direction, so the randomly generated node direction should be as far as possible from the interaction direction to reduce the risk; on the contrary, when the interaction intensity is small, it means that the target UAV has a low risk of potential conflict with the neighbor UAV in the direction, so the direction of the random node can be closer to the interaction direction, so as to maintain the efficiency and feasibility of the flight path planning; so that the random node can adaptively avoid high-risk areas while maintaining the random exploration ability, and realize the safe and efficient expansion of the UAV in the complex environment.

[0047] Specifically, a candidate leaf node is randomly generated at the position corresponding to the current node, and 50 candidate leaf nodes are generated in this embodiment. The direction of each candidate leaf node is the direction of the current node pointing to the corresponding candidate leaf node. The interaction intensity is taken as the module length, and the interaction vector of each candidate leaf node is obtained in combination with the direction of the candidate leaf node. Meanwhile, the interaction intensity vector is constructed based on the interaction intensity and the direction of the current node pointing to the preset node. The projection component of each candidate leaf node on the interaction intensity vector is obtained, and the ratio (module length value) of the projection component (length value) to the interaction intensity is obtained. The difference obtained by subtracting the ratio from 1 is multiplied by the included angle between the direction of the candidate leaf node and the direction of the current node pointing to the preset node. The modified direction of the candidate leaf node is obtained based on the direction of the current node pointing to the preset node and the product, that is, the included angle is adjusted and modified. Then, the modified direction is obtained again based on the adjusted and modified included angle. The modified direction, the original direction of the candidate leaf node, and the direction of the current node pointing to the preset node are located in the same plane, and the modified direction and the original direction are the same rotation direction based on the direction of the current node pointing to the preset node, that is, both are clockwise or both are counterclockwise.

[0048] Further, the position corresponding to the candidate leaf node is obtained based on the modified direction of the candidate leaf node and the distance between the current node and the preset node, and is taken as a candidate node of the current node. The candidate nodes of each candidate leaf node are obtained according to the above method, and then the target UAV has several candidate nodes at the current node.

[0049] At this point, the target UAV has several candidate nodes at the current node.

[0050] Step S004, analyze the similarity relationship between the remote sensing image of the candidate node and the remote sensing image of the preset node, and obtain the flight target node of the target UAV at the current node in combination with the city discrete graph, so as to realize the real-time planning of the low-altitude route of the UAV.

[0051] It should be noted that after determining the candidate node generation process, the optimal candidate node needs to be found as the next ideal position of the target UAV in the new route, and the path planning not only needs to explore new possible positions, but also must maintain the continuity and rationality of the planning, and the image difference measurement can intuitively reflect the degree of deviation of the node from the expected path; at the same time, the node selection is combined with the obstacle information, so that the environmental constraints can be considered in the planning process, and the node generation process can consider the rationality and safety of the path; through the similarity of the candidate node image and the preset node image, and the obstacle information between the current node and the candidate node, the optimal candidate node is screened out.

[0052] Specifically, for any candidate node of the target UAV at the current node, the remote sensing image of the candidate node is obtained through remote sensing, and is compared with the remote sensing image corresponding to the preset node, the structural similarity index SSIM (prior art, this embodiment will not be repeated) is used to quantify the similarity of the two remote sensing images as the environmental invariant factor of the candidate node; at the same time, based on the city discrete graph, whether there is an obstacle between the current node and the candidate node is obtained, and if there is, it is marked as 0, and if there is not, it is marked as 1, and is marked as the obstacle factor of the candidate node; the product of the environmental invariant factor and the obstacle factor is used as the preferred degree of the candidate node, and the candidate node corresponding to the maximum value of the preferred degree in all candidate nodes is used as the flight target node of the target UAV at the current node, then the target UAV generates a new route based on the flight target node, and goes to the next predicted position along the route; in this process, the route can be fine-tuned in combination with the path smoothing and dynamic obstacle avoidance strategy to ensure the continuity and safety of the flight, so that the UAV can efficiently and stably advance the task according to the generated route in a complex environment, and at the same time, provides a real-time state basis for the next leaf node generation, so as to realize the real-time planning of the low-altitude route of the UAV.

[0053] Thus, the embodiment is completed.

[0054] Another embodiment of the application provides a low-altitude route planning system based on image processing, which comprises a memory, a processor and a computer program stored in the memory and running on the processor, and the processor implements the method steps S001 to S004 when executing the computer program.

[0055] The above only describes the preferred embodiments of the application and is not intended to limit the application, and any modification, equivalent replacement, improvement, etc. made within the principles of the application shall be included in the protection scope of the application.

Claims

1. A low-altitude route planning method based on image processing, characterized in that: The method comprises the following steps: Obtain urban GIS models, point cloud data, and remote sensing images to generate high-precision three-dimensional models of the urban environment, and then obtain urban discrete maps; Based on the city scatter map, a preset planned route is generated for the target UAV. Combining the proximity performance in the time domain and airspace, several neighboring UAVs of the target UAV at the current node are obtained, and the verification flight phase of the target UAV from the current node to the preset node is recorded. The position difference changes of the same feature points in the images taken by the target UAV and each neighboring UAV during the verification flight phase are analyzed to obtain the dynamic interaction intensity between the target UAV and each neighboring UAV during the verification flight phase. Based on the similarity relationship between the target UAV and the images taken by each neighboring UAV during the verification flight phase, the static interaction strength between the target UAV and each neighboring UAV during the verification flight phase is obtained, and then the interaction strength of the target UAV at the current node is obtained; a threshold is used to determine whether the target UAV needs to adjust the preset node at the current node, and a number of candidate nodes for the target UAV at the current node are obtained by combining the interaction strength and the direction of the current node pointing to the preset node; The similarity relationship between the remote sensing images of the candidate nodes and the remote sensing images of the preset nodes is analyzed, and the flight target node of the target UAV at the current node is obtained by combining the city discrete graph.

2. The low-altitude route planning method based on image processing according to claim 1, characterized in that: The specific method of generating a preset planned route for the target UAV based on the city discrete map includes: The UAV currently undergoing low-altitude route planning is taken as the target UAV. Based on the city discrete graph, a rapidly expanding random tree algorithm is used to generate a preset planned route for the target UAV.

3. The low-altitude route planning method based on image processing according to claim 1, characterized in that: The specific method of obtaining several neighboring drones of the target drone at the current node and recording the verification flight phase of the target drone from the current node to the preset node includes: Each point in the preset planned route of the target UAV is regarded as a node, including the starting point, the end point and the point where the flight direction changes in the preset planned route. The node where the target UAV is currently located is regarded as the current node, and the next node of the current node in the preset planned route is regarded as the preset node; The voxel where the current node is located in the city discrete map is regarded as the neighborhood airspace of the target UAV at the current node, and the neighborhood time range before and after the current moment is regarded as the neighborhood time domain at the current moment; in the neighborhood time domain at the current moment, several UAVs whose flight positions are located in the neighborhood airspace of the current node are regarded as the neighboring UAVs of the target UAV; Get the distance of the target UAV from the current node to the preset node, preset the verification ratio, and use the distance of the target UAV from the current node to the preset node before the verification ratio as the verification flight stage of the target UAV from the current node to the preset node. Based on the verification flight stage and the UAV flight speed, get the verification flight duration of the target UAV at the current node. Combined with the current moment, get the verification flight time range of the target UAV at the current node.

4. The low-altitude route planning method based on image processing according to claim 3, characterized in that: The specific method for obtaining the dynamic interaction strength between the target UAV and each neighboring UAV during the verification flight phase is as follows: Acquire several images taken by the target UAV during the verification flight time range of the current node as several collected images of the target UAV during the verification flight phase; acquire several collected images of any neighboring UAV of the target UAV during the verification flight phase of the target UAV; for any collected image of the target UAV during the verification flight phase, acquire the collected image corresponding to the collected image of the neighboring UAV, perform wavelet denoising and SIFT feature point detection on the two collected images respectively, obtain several feature points of the two collected images, use the approximate nearest neighbor search algorithm to match the feature points in the two collected images, obtain several feature point pairs in the two collected images, and use the two feature points in the feature point pair as the same feature points in the images taken by different UAVs; Based on the target UAV’s interaction with neighboring UAVs at adjacent moments in the verification flight time range of the current node The position difference change of the same feature point in the collected image is obtained to obtain the convergence degree of each same feature point at each moment in the verification flight time range; The average convergence degree of all feature points corresponding to the feature points at all moments in the verification flight time range is used as the average convergence degree of the target UAV and its neighboring UAVs in the verification flight phase. Dynamic interaction strength.

5. The low-altitude route planning method based on image processing according to claim 4, characterized in that: The specific method of obtaining the convergence degree of each identical feature point at each moment in the verification flight time range includes: For the target drone and its neighboring drones At the moment Any identical feature point corresponding to the two captured images of Relative position The calculation method is: in, Indicates that the target drone is at time The coordinates of the feature point in the captured image, Indicates the neighboring drones of the target drone At the moment The coordinates of the feature point in the captured image; The same feature point in the continuous captured images is obtained by the optical flow method, and the feature point is at the time Relative movement trend The calculation method is: in, Indicates that the feature point is at time The relative position of The feature point at time relative position; Obtain the product of the relative position and the relative motion trend, and use the ratio of the modulus of the relative position to the product as the characteristic point at time The convergence factor of Obtain the convergence factor of each feature point corresponding to the same feature point within the verification flight time range of the target UAV at the current node, perform linear normalization on all convergence factors, and use the result as the convergence degree of the corresponding feature point at each moment.

6. The low-altitude route planning method based on image processing according to claim 4, characterized in that: The specific method for obtaining the static interaction strength between the target UAV and each neighboring UAV during the verification flight phase is as follows: For any captured image of the target UAV during the verification flight phase, obtain the corresponding captured image of any neighboring UAV, obtain the number of feature point pairs between the two captured images, and the maximum number of feature points in each of the two captured images, and obtain the ratio of the number of feature point pairs to the maximum value; Obtain the distance between two feature points in each feature point pair, obtain the ratio of the mean of all distances to the maximum value of all distances, and multiply the difference obtained by subtracting the ratio from 1 and the ratio of the number of feature point pairs to the maximum value as the image similarity of the two captured images; The mean value of the image similarity between the target UAV and the neighboring UAV at all times within the verification flight time range is taken as the static interaction intensity between the target UAV and the neighboring UAV during the verification flight phase.

7. The low-altitude route planning method based on image processing according to claim 1, characterized in that: The specific method for obtaining the interaction strength of the target UAV at the current node is as follows: The product of the static interaction intensity and the dynamic interaction intensity of the target UAV with the neighboring UAVs during the verification flight phase is taken as the comprehensive interaction intensity of the target UAV with the neighboring UAVs during the verification flight phase; the average of the comprehensive interaction intensity of the target UAV with each neighboring UAV during the verification flight phase is taken as the interaction intensity of the target UAV at the current node.

8. The low-altitude route planning method based on image processing according to claim 3, characterized in that: The specific method of obtaining several candidate nodes of the target UAV at the current node includes: A candidate leaf node is randomly generated at the position corresponding to the current node. The direction of each candidate leaf node is from the current node to the corresponding candidate leaf node. The interaction strength is used as the modulus. Combined with the direction of the candidate leaf node, the interaction vector of each candidate leaf node is obtained. Based on the interaction strength and the direction of the current node to the preset node, the interaction strength vector is constructed. Obtain a projection component of the interaction vector of each candidate leaf node on the interaction strength vector, obtain a ratio of the projection component to the interaction strength, obtain a difference obtained by subtracting the ratio from 1, and multiply the difference by the angle between the direction of the candidate leaf node and the direction of the current node pointing to the preset node, and obtain a corrected direction of the candidate leaf node based on the direction of the current node pointing to the preset node and the product; Based on the corrected direction of the candidate leaf node and the distance between the current node and the preset node, the position corresponding to the candidate leaf node is obtained and used as a candidate node for the current node.

9. The low-altitude route planning method based on image processing according to claim 1, characterized in that: The target node of the target UAV at the current node is obtained by: For any candidate node of the target UAV at the current node, the remote sensing image of the candidate node is obtained through remote sensing and compared with the remote sensing image corresponding to the preset node. The structural similarity index SSIM is used to quantify the similarity of the two remote sensing images as the environmental invariant factor of the candidate node; Based on the city discrete graph, obtain whether there is an obstacle between the current node and the candidate node. If there is an obstacle, mark it as 0; if not, mark it as 1, and record it as the obstacle factor of the candidate node; The product of the environmental invariant factor and the obstacle factor is used as the preference degree of the candidate node, and the candidate node corresponding to the maximum preference degree among all candidate nodes is used as the flight target node of the target UAV at the current node.

10. A low-altitude route planning system based on image processing, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the steps of the low-altitude route planning method based on image processing as described in any one of claims 1 to 9 are implemented.

Images