Aerial photography unmanned aerial vehicle route planning method for smart city modeling

Abstract

The present application relates to the technical field of unmanned aerial vehicle control, in particular to an aerial photography unmanned aerial vehicle route planning method for smart city modeling, comprising: identifying feature points of a current image in the process that an unmanned aerial vehicle executes a shooting task according to a planned route; determining target importance of each feature point according to single importance of each feature point in the current image and comparative importance relative to position movement in a previous image; determining an actual overlap rate of the current image and the previous image according to a ratio of a total importance sum of matching points and a total importance sum; comparing the actual overlap rate with a preset overlap rate and adjusting a flight speed of the unmanned aerial vehicle according to a comparison result. The method can accurately achieve the preset overlap rate, improve the quality of aerial photography data required for smart city modeling, and further improve the effect of smart city modeling.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of unmanned aerial vehicle control, and particularly relates to an aerial photography unmanned aerial vehicle route planning method for smart city modeling. BACKGROUND

[0002] Smart city modeling refers to using modern information technologies such as big data, Internet of Things, cloud computing and remote sensing technology to build a digital, visual and operable city model. Using aerial photography unmanned aerial vehicles for smart city modeling has significant advantages. First, unmanned aerial vehicles can efficiently obtain large-scale, high-resolution image data of the city, especially in densely populated and heavily trafficked areas, where traditional ground data collection methods are difficult to complete. Through unmanned aerial vehicles, data collection can be more convenient. Second, unmanned aerial vehicles have wide coverage and fast shooting speed, which can complete a large amount of data collection in a short time, reducing the time cost of manual measurement. At the same time, unmanned aerial vehicles can flexibly adjust the route to obtain image data at different angles and heights, providing diverse perspectives for three-dimensional modeling and city planning.

[0003] Traditional aerial photography unmanned aerial vehicle route planning methods usually use an "open loop" mode. First, the user inputs the target area, ground sampling distance (GSD, Ground Sample Distance) and preset image overlap rate and other parameters. Then, the path planning algorithm generates the route of the unmanned aerial vehicle according to these parameters to ensure that the route covers the entire target area and meets the preset image overlap rate requirement. Finally, the unmanned aerial vehicle executes the flight task according to the planned route and completes the collection of city image data.

[0004] Image overlap rate is crucial for later stitching and modeling. By setting an appropriate image overlap rate, it can be ensured that the images taken have enough overlapping areas, which facilitates later stitching and three-dimensional modeling, thereby improving the accuracy of aerial photography data and ensuring high-quality model reconstruction.

[0005] However, the traditional aerial photography unmanned aerial vehicle route planning method cannot well meet the requirements of image overlap rate. Traditional aerial photography unmanned aerial vehicle route planning methods are usually based on the assumption that the unmanned aerial vehicle flies at a constant speed, but in reality, factors such as headwind, tailwind, acceleration or deceleration can cause the actual photographing point interval of the unmanned aerial vehicle to be uneven during flight, thereby affecting the realization of the image overlap rate. For example, in the case of headwind, the ground speed of the unmanned aerial vehicle decreases, the interval between photographing points shortens, and the image overlap rate increases. When tailwind, the ground speed of the unmanned aerial vehicle increases, the interval between photographing points becomes larger, and the image overlap rate decreases. Failure to accurately achieve the preset image overlap rate can affect the quality of aerial photography data required for smart city modeling and the modeling effect of the later smart city. SUMMARY

[0006] In order to solve the technical problem of inaccurate image overlap rate of image collected by the unmanned aerial vehicle, the present application aims to provide an aerial photography unmanned aerial vehicle route planning method for smart city modeling, and the technical solution is as follows:

[0007] The present application provides an aerial photography unmanned aerial vehicle route planning method for smart city modeling, and the method comprises:

[0008] In the process that the unmanned aerial vehicle performs the smart city modeling shooting task according to the planned route, the feature points of the current image are identified;

[0009] According to the single importance of each feature point in the current image and the comparative importance relative to the position movement in the previous image, the target importance of each feature point is determined;

[0010] According to the ratio of the matching point importance sum to the total importance sum, the actual overlap rate of the current image and the previous image is determined; the matching point importance sum is the sum of the target importance of the feature points matched with the previous image in the current image; the total importance sum is the sum of the target importance of all feature points in the current image;

[0011] The actual overlap rate is compared with the preset overlap rate, and the flight speed of the unmanned aerial vehicle is adjusted according to the comparison result.

[0012] According to the aerial photography unmanned aerial vehicle route planning method for smart city modeling provided by the present application, the target importance of each feature point is determined according to the single importance of each feature point in the current image and the comparative importance relative to the position movement in the previous image, which comprises:

[0013] The multi-dimensional features of each feature point in the current image are analyzed to determine the single importance of each feature point in the current image;

[0014] For each feature point, the comparative importance of the feature point relative to the position movement in the previous image is determined according to the change of the moving distance of the feature point and the remaining feature points in the current image and the previous image;

[0015] The target importance of the feature point is determined according to the single importance and the comparative importance of the feature point.

[0016] According to the aerial photography unmanned aerial vehicle route planning method for smart city modeling provided by the present application, the multi-dimensional features comprise the magnitude of the object to which the feature point belongs.

[0017] The analysis of the multi-dimensional features of each feature point in the current image comprises:

[0018] For each feature point in the current image, the gray value of the feature point is compared with the gray value of the remaining pixel points in the image, and the magnitude of the object to which the feature point belongs is determined.

[0019] According to the present application, an aerial photography unmanned aerial vehicle route planning method for smart city modeling is provided, and the comparison of the gray value of the feature point with the gray value of the remaining pixel points in the image and the determination of the magnitude of the object to which the feature point belongs include:

[0020] According to the difference between the gray value of the feature point and the gray value of each of the remaining pixel points in the image, a magnitude contribution value corresponding to each of the remaining pixel points is determined, and the difference between the magnitude contribution value and the gray value is negatively correlated.

[0021] According to the distance between each of the remaining pixel points and the feature point, the magnitude contribution values corresponding to each of the remaining pixel points are weighted and summed, and the magnitude of the object to which the feature point belongs is determined according to the weighted summation result.

[0022] According to the present application, an aerial photography unmanned aerial vehicle route planning method for smart city modeling is provided, and the multi-dimensional features include the possibility that the feature point belongs to the top of the object.

[0023] The analysis of the multi-dimensional features of each of the feature points in the current image includes:

[0024] According to the gradient of the pixel point corresponding to each of the feature points in the current image, the possibility that each of the feature points belongs to the top of the object is determined.

[0025] According to the present application, an aerial photography unmanned aerial vehicle route planning method for smart city modeling is provided, and the multi-dimensional features include the possibility that the feature point belongs to the edge of the image.

[0026] The analysis of the multi-dimensional features of each of the feature points in the current image includes:

[0027] For each feature point in the current image, the possibility that the feature point belongs to the edge of the image is determined according to the difference between the coordinates of the feature point and the coordinates of the center point of the image.

[0028] According to the present application, an aerial photography unmanned aerial vehicle route planning method for smart city modeling is provided, and the determination of the comparative importance of the position movement of the feature point relative to the previous image according to the change of the movement distance of the feature point and the remaining feature points in the current image and in the previous image includes:

[0029] The possibility that the feature point and the remaining feature points in the current image belong to the same object is determined.

[0030] According to the change of the moving distance of the feature point and the remaining feature points in the current image and in the previous image, the moving consistency of the feature point relative to the remaining feature points is determined.

[0031] According to the possibility that the feature point and each of the remaining feature points belong to the same object, the moving consistency of the feature point relative to each of the remaining feature points is weighted and summed, and the comparative importance of the feature point relative to the position moving condition in the previous image is determined according to the weighted and summed result.

[0032] According to the present application, the possibility that the feature point and the remaining feature points in the current image belong to the same object is determined, and the method comprises the following steps:

[0033] According to the discrete degree of the gray value distribution of the pixel points between the feature point and the remaining feature points in the current image, the possibility that the feature point and the remaining feature points in the current image belong to the same object is determined.

[0034] According to the present application, the moving consistency of the feature point relative to the remaining feature points is determined according to the change of the moving distance of the feature point and the remaining feature points in the current image and in the previous image, and the method comprises the following steps:

[0035] The moving distance of the position of the feature point in the current image relative to the position in the previous image, and the moving distance of the position of the remaining feature points in the current image relative to the position in the previous image are determined.

[0036] According to the difference between the moving distance of the feature point and the moving distance of the remaining feature points, and the single importance of the remaining feature points, the moving consistency of the feature point relative to the remaining feature points is determined.

[0037] According to the present application, the actual overlap rate is compared with the preset overlap rate, and the flight speed of the unmanned aerial vehicle is adjusted according to the comparison result, and the method comprises the following steps:

[0038] If the actual overlap rate is less than the preset overlap rate, the flight speed of the unmanned aerial vehicle is reduced.

[0039] If the actual overlap rate is greater than the preset overlap rate, the flight speed of the unmanned aerial vehicle is increased.

[0040] The present application has the following advantages: in the process that the unmanned aerial vehicle performs the intelligent city modeling shooting task according to the planned route, the feature points of the current image are identified, the target importance of each feature point is determined according to the single importance of each feature point in the current image and the comparative importance relative to the position movement in the previous image, the actual overlap rate of the current image and the previous image is determined according to the ratio of the target importance sum of the matching points and the target importance sum of all points, the actual overlap rate is compared with the preset overlap rate, and the flight speed of the unmanned aerial vehicle is adjusted according to the comparison result. Compared with the "open loop" control of the traditional method, the factors such as headwind, tailwind, acceleration or deceleration will cause the actual shooting point interval of the unmanned aerial vehicle to be uneven in the flight process, so that the preset image overlap rate is difficult to be accurately realized, and the final modeling effect is affected. The present application performs "closed loop" control, the actual overlap rate of the current image and the previous image is calculated in real time when the unmanned aerial vehicle performs the shooting task, the flight speed of the unmanned aerial vehicle is fed back and adjusted according to the comparison result of the actual overlap rate and the preset overlap rate, so that the preset overlap rate can be accurately realized, the quality of the aerial photography data required for intelligent city modeling is improved, and the effect of intelligent city modeling is improved. BRIEF DESCRIPTION OF DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, and the advantages thereof, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative effort.

[0042] Figure 1 A flowchart of a route planning method of an aerial photography unmanned aerial vehicle for intelligent city modeling provided by an embodiment of the present application;

[0043] Figure 2 A flowchart of determining the target importance of a feature point provided by an embodiment of the present application;

[0044] Figure 3 A flowchart of determining the comparative importance of a feature point provided by an embodiment of the present application;

[0045] Figure 4 A flowchart of determining the movement consistency of a feature point relative to the remaining feature points provided by an embodiment of the present application;

[0046] Figure 5 A flowchart of a route planning method of an aerial photography unmanned aerial vehicle for intelligent city modeling provided by an embodiment of the present application. DETAILED DESCRIPTION

[0047] In order to further clarify the technical means and effects taken by the present application to achieve the predetermined object of the present application, the following describes in detail the specific implementation, structure, features and effects of the aerial photography unmanned aerial vehicle route planning method for smart city modeling according to the present application, in combination with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. In addition, the specific features, structures or characteristics in one or more embodiments can be combined in any suitable form.

[0048] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs.

[0049] The specific scheme of the aerial photography unmanned aerial vehicle route planning method for smart city modeling provided by the present application is described in detail below in combination with the accompanying drawings.

[0050] Please refer to Figure 1 which shows the flowchart of the aerial photography unmanned aerial vehicle route planning method for smart city modeling provided by one embodiment of the present application, including the following steps:

[0051] Step 101, in the process of the unmanned aerial vehicle performing the smart city modeling shooting task according to the planned route, the feature points of the current image are identified.

[0052] Wherein, the feature point is a basic concept in the field of computer vision, and the feature point refers to a significant position point in an image with uniqueness, stability and repeatability. The feature point usually corresponds to a special geometric structure or texture pattern in the image. For example: the feature point can include the corner point of the object in the image, the edge point and the point in the area with complex texture pattern, etc. The feature point in the current image can be identified by any feature point detection method. For example: the feature point detection method can include Harris corner detection, SIFT (Scale-Invariant Feature Transform), SURF (Speeded-Up Robust Features) and LBP (Local Binary Patterns), etc. In one embodiment, in the dynamic route planning of smart city modeling, first, the boundary and geographic information of the target area to be photographed are defined to ensure that the route planning covers the entire target area and considers the terrain, obstacles and flight restrictions of the target area. Then, according to the size of the target area and the required image resolution, the ground resolution (ground sampling distance) required for each image is determined. Next, the desired front and rear image overlap ratio (preset overlap rate) is set to ensure that each image collected can be effectively aligned and spliced. For example: the preset overlap rate is usually 70%~80%. Finally, using a path planning algorithm: combining the target area, ground sampling distance (GSD, Ground Sample Distance) and preset overlap rate, a suitable route is calculated, including the starting point, ending point and route spacing of the flight path, to meet the image overlap requirement. This embodiment ensures that the route is reasonable and maximizes the shooting efficiency. Then the unmanned aerial vehicle performs the shooting task on this planned route to shoot the required images of smart city modeling. During the execution of the task, each time a current image is obtained, steps 101 to 104 are executed to adjust the flight speed of the unmanned aerial vehicle according to the current image.

[0053] Step 102, determine the target importance of each feature point according to the single importance of each feature point in the current image and the comparative importance relative to the position movement in the previous image.

[0054] Wherein, the single importance refers to the importance degree of the feature point in a single image. The comparative importance refers to the importance degree of the feature point according to the position comparison of the feature point in the current image and the previous image. The target importance is the comprehensive importance degree of the feature point obtained by comprehensively considering the single importance and the comparative importance of the feature point.

[0055] In one embodiment, the target importance of the feature point is determined according to the product of the single importance of the feature point in the current image and the comparative importance of the position movement relative to the previous image.

[0056] In step 103, the actual overlap rate of the current image and the previous image is determined according to the ratio of the matching point importance sum and the total importance sum; the matching point importance sum is the sum of the target importance of the feature points in the current image that are matched with the previous image; and the total importance sum is the sum of the target importance of all the feature points in the current image.

[0057] It can be understood that in the conventional wisdom city modeling, the actual overlap rate is usually calculated by comparing the proportion of the number of the feature points that can be matched in the current image and the previous image and the number of all the feature points in the current image. However, the target importance of each feature point is different, and only the feature points with higher target importance have actual significance for modeling. These important feature points can provide more stable and reliable geometric information, thereby ensuring the spatial consistency and accuracy in the modeling process. Therefore, the calculated actual overlap rate can better reflect the spatial matching degree between the two images and effectively guide the subsequent three-dimensional reconstruction and city model optimization only on the premise of these important feature points. Therefore, the actual overlap rate of the current image and the previous image can be determined according to the ratio of the matching point importance sum and the total importance sum.

[0058] In one embodiment, the actual overlap rate of the current image and the previous image can be determined according to the following formula:

[0059]

[0060] wherein, represents the actual overlap rate of the current image and the previous image. represents the target importance of the feature point in the current image that is matched with the previous image. represents the total number of the feature points in the current image that are matched with the previous image. represents the matching point importance sum. represents the total importance sum of all the feature points in the current image. represents the total number of the feature points in the current image. represents the total importance sum.

[0061] In step 104, the actual overlap rate is compared with the preset overlap rate, and the flight speed of the unmanned aerial vehicle is adjusted according to the comparison result.

[0062] In one embodiment, the actual overlap rate is compared with the preset overlap rate, an error signal is generated according to the comparison result, the error signal is transmitted to the flight control algorithm, and a UAV speed adjustment instruction is generated by the flight control algorithm according to the error signal. For example, the flight control algorithm can use a PID (Proportional Integral Derivative) controller.

[0063] In one embodiment, the UAV speed adjustment instruction generated by the decision is transmitted to the flight control system in real time, the flight control system adjusts the motor power to accelerate or decelerate the UAV until the new target speed is reached. The UAV flies to the next shooting point at the new target speed and takes pictures, and the actual overlap rate after shooting is fed back to the system again to start a new round of adjustment cycle.

[0064] The aerial photography UAV route planning method for smart city modeling described above, by identifying the feature points of the current image during the UAV performing the smart city modeling shooting task according to the planned route, determining the target importance of each feature point according to the single importance of each feature point in the current image and the comparative importance relative to the position movement in the previous image, determining the actual overlap rate of the current image and the previous image according to the ratio of the target importance sum of the matching points to the target importance sum of all points, comparing the actual overlap rate with the preset overlap rate, and adjusting the flight speed of the UAV according to the comparison result. Compared with the "open loop" control of the traditional method, factors such as headwind, tailwind, acceleration or deceleration will cause the actual shooting point interval of the UAV to be uneven during flight, making it difficult to accurately achieve the preset image overlap rate, and thus affecting the final modeling effect. The present application performs "closed loop" control, calculates the actual overlap rate of the current image and the previous image in real time when the UAV performs the shooting task, feeds back and adjusts the flight speed of the UAV according to the comparison result of the actual overlap rate and the preset overlap rate, so as to accurately achieve the preset overlap rate, improve the quality of the aerial photography data required for smart city modeling, and thus improve the effect of smart city modeling. This fine-tuning mechanism effectively offsets the influence of external factors such as wind speed changes, ensuring that the image overlap rate is stable around the preset overlap rate, thereby ensuring the uniformity of the data. For smart city modeling, uniform and high-quality data is crucial, avoiding model holes or redundant data caused by insufficient image overlap, significantly improving the accuracy and efficiency of subsequent modeling. Ultimately, this closed-loop system realizes dynamic adaptive route planning, providing reliable protection for the generation of high-precision smart city base maps.

[0065] In one embodiment, referring to Figure 2 , the target importance of each feature point is determined according to the single importance of each feature point in the current image and the comparative importance relative to the position movement in the previous image, including the following steps:

[0066] Step 201, analyze the multi-dimension features of each feature point in the current image to determine the single importance of each feature point in the current image.

[0067] In one embodiment, the multi-dimension features include several of the following: the size of the object to which the feature point belongs, the likelihood that the feature point belongs to the top of the object, and the likelihood that the feature point belongs to the edge of the image. Among them, the size of the object to which the feature point belongs is used to measure the possibility that the object to which the feature point belongs is a large object.

[0068] In one embodiment, the single importance of the feature point in the current image is determined according to the product of the multi-dimension features of the feature point in the current image.

[0069] Step 202, for each feature point, determine the comparative importance of the feature point relative to the position movement in the previous image according to the change of the movement distance of the feature point and the rest of the feature points in the current image and the previous image.

[0070] Among them, the rest of the feature points refers to the feature points in the current image except the feature point being processed.

[0071] It can be understood that in the modeling of smart city, if the relative distance between the feature point and the feature points belonging to the same object based on the single importance between the current image and the previous image changes greatly, the object corresponding to the feature point is considered to be moving or variable, and therefore the importance of the feature point to the modeling will decrease. First, since the modeling of smart city usually relies on the spatial structure and features of static objects to construct the three-dimensional model of the city, if the object corresponding to a certain feature point shows obvious relative displacement between different images, it means that the object is not static, but a moving object (such as vehicles, pedestrians, etc.) or a variable object (such as flags, building curtain walls, etc.). The position of such objects changes greatly in different times and spaces, resulting in unstable geometric features, so they are not suitable for accurate modeling. Second, the feature points of moving objects or variable objects have low spatial consistency. The relative position in different images changes greatly, indicating that these feature points no longer represent a fixed geometric structure, but are affected by time, environment or other factors. Such changes will lead to a decrease in the accuracy of the three-dimensional model, and even errors or inconsistencies. Therefore, the comparative importance of the feature point relative to the position movement in the previous image is determined according to the change of the movement distance of the feature point and the rest of the feature points in the current image and the previous image.

[0072] Step 203, determine the target importance of the feature point according to the single importance and the comparative importance of the feature point.

[0073] In the above embodiments, the multi-dimension feature of each feature point in the current image is analyzed to determine the single importance of each feature point in the current image. For each feature point, the comparative importance of the feature point relative to the position movement in the previous image is determined according to the change of the movement distance of the feature point and the remaining feature points in the current image and the previous image. The target importance of the feature point is determined by combining the single importance and the comparative importance of the feature point.

[0074] In one embodiment, the multi-dimension feature includes the huge degree of the object to which the feature point belongs. The multi-dimension feature of each feature point in the current image is analyzed, including: for each feature point in the current image, comparing the gray value of the feature point with the gray value of the remaining pixel points in the image to determine the huge degree of the object to which the feature point belongs.

[0075] In the above embodiments, the multi-dimension feature of each feature point in the current image is analyzed to determine the single importance of each feature point in the current image. For each feature point, the comparative importance of the feature point relative to the position movement in the previous image is determined according to the change of the movement distance of the feature point and the remaining feature points in the current image and the previous image. The target importance of the feature point is determined by combining the single importance and the comparative importance of the feature point.

[0076] It can be understood that in the modeling of smart cities, each feature point in each image contributes differently to the overall modeling accuracy, especially when the object corresponding to the feature point is very large, its importance is more prominent. The reason is that these large objects usually occupy important spatial positions in the modeling process and have a great influence on the geometric structure and spatial distribution of the overall model. Since large objects (such as high-rise buildings, squares, and roads, etc.) usually play a "reference" or "support" role in the three-dimensional model of a smart city, they not only occupy a prominent position visually, but also help define the spatial pattern and structure of the city. For example, in the modeling process, large buildings can serve as reference points or alignment benchmarks to ensure that the position relationship of other small objects relative to these large objects is more accurate. If the feature points of these large objects are ignored, it may cause model distortion, especially in the modeling of large buildings or infrastructure, the lack of these important feature points will affect the overall accuracy and even cause the loss of key parts in the model. Therefore, the multi-dimension feature used to determine the single importance of the feature point can include the huge degree of the object to which the feature point belongs. In addition, since the larger the object, the wider the range of its gray value, the huge degree of the object to which the feature point belongs can be determined by comparing the gray value of the feature point with the gray value of the remaining pixel points in the image.

[0077] In the above embodiments, the huge degree of the object to which the feature point belongs can be accurately determined by comparing the gray value of the feature point with the gray value of the remaining pixel points in the image, and the single importance of the feature point can be accurately determined based on the huge degree of the object to which the feature point belongs.

[0078] In one embodiment, comparing the grayscale values ​​of a feature point with those of other pixels in the image to determine the giantness of the object to which the feature point belongs includes: determining the giantness contribution value corresponding to each of the other pixels based on the difference between the feature point and the grayscale values ​​of each of the other pixels in the image; the giantness contribution value and the difference between the grayscale values ​​are negatively correlated; and performing a weighted summation of the giantness contribution values ​​corresponding to each of the other pixels based on the distance between each of the other pixels and the feature point, and determining the giantness of the object to which the feature point belongs based on the weighted summation result.

[0079] The giantness contribution value measures the contribution of the difference between the gray values ​​of other pixels and the feature point to the giantness of the object to which the feature point belongs. Since the larger the object, the wider its gray value range, the smaller the difference between the gray values ​​of other pixels and the feature point, the larger the giantness contribution value.

[0080] In one embodiment, the weights of the remaining pixels are determined based on their distances to the feature point. Then, the gravitability contributions of the remaining pixels are weighted and summed, and the gravitability of the object to which the feature point belongs is determined based on the weighted sum. The weights of the remaining pixels are negatively correlated with their distances to the feature point; that is, the closer the remaining pixels are to the feature point, the greater their weights.

[0081] In one embodiment, the size of the object to which the feature point belongs can be determined according to the following formula:

[0082]

[0083] in, Indicates the current image Single feature point The size of the object to which it belongs. Indicates the current image Single feature point With the remaining pixels The distance between them. Indicates the current image Single feature point The grayscale value. Indicates the current image The remaining pixels The grayscale value. This represents an exponential function with base e. Represents the remaining pixels The corresponding huge contribution value. Represents the remaining pixels The corresponding weights.

[0084] In the above embodiments, since the greater the object, the wider the gray value extension range, according to the difference between the gray value of the feature point and each of the remaining pixel points in the image, the contribution value of each of the remaining pixel points to the largeness of the object can be accurately determined, and then the contribution value of each of the remaining pixel points to the largeness of the object is weighted and summed according to the distance between each of the remaining pixel points and the feature point, and the closer the remaining pixel point to the feature point, the greater the weight, so that the largeness of the object to which the feature point belongs can be accurately determined.

[0085] In one embodiment, the multi-dimension feature includes a possibility that the feature point belongs to the top of the object. The multi-dimension feature of each feature point in the current image is analyzed, including: determining the possibility that each feature point belongs to the top of the object according to the gradient of the pixel point corresponding to each feature point in the current image.

[0086] It can be understood that in the modeling of smart city, if the feature point corresponds to the top of the object on the basis of the corresponding large object, the value of this feature point is more prominent, because the top is usually the highest point or the most representative part of the object, which is of great significance to the spatial structure, visual effect and data accuracy of the whole city. First of all, the top is usually a prominent feature of the object, especially in high-rise buildings, towers, bridges and other buildings, the top often determines the height, appearance and function of the building. For three-dimensional modeling, the top of the building can be used as a key point for measurement and alignment to help the system determine the spatial coordinates and height information of the object. Therefore, the feature point of the top is crucial for the vertical structure modeling of the city, which can accurately reflect the three-dimensional sense of the city. Secondly, the top feature point has a strong calibration effect on the shape, spatial position and relationship with the surrounding environment of the object. In the modeling of smart city, accurately obtaining the geometric information of the top can optimize the relative position relationship and spatial layout between buildings, and avoid deviations or overlaps in the model. Therefore, the multi-dimension feature used to determine the importance of the feature point can include the possibility that the feature point belongs to the top of the object.

[0087] Since the gray value of the pixel point of the top of the object often changes sharply, the gradient is large, so the possibility that the feature point belongs to the top of the object is positively correlated with the gradient of the pixel point corresponding to the feature point.

[0088] In one embodiment, the gradient of the pixel point corresponding to each feature point in the current image is normalized, and the normalized result is used as the possibility that each feature point belongs to the top of the object. For example, the gradient of the feature point can be normalized by the maximum and minimum values of the gradients of the remaining pixel points around the feature point.

[0089] In one embodiment, the possibility that the feature point belongs to the top of the object can be determined according to the following formula:

[0090]

[0091] wherein, denotes a single feature point in the current image the likelihood that the feature point belongs to the top of the object. denotes a single feature point in the current image the gradient of the corresponding pixel point. denotes a maximum-minimum normalization function. In the above embodiment, since the likelihood that the feature point belongs to the top of the object is positively correlated with the gradient of the corresponding pixel point, the likelihood that each feature point belongs to the top of the object can be accurately determined according to the gradient of the pixel point corresponding to each feature point in the current image.

[0092] In one embodiment, the multi-dimension feature includes the likelihood that the feature point belongs to the edge of the image. Analyzing the multi-dimension feature of each feature point in the current image includes: for each feature point in the current image, respectively, determining the likelihood that the feature point belongs to the edge of the image according to the difference between the coordinates of the feature point and the coordinates of the center point of the image.

[0093] In one embodiment, the multi-dimension feature includes the likelihood that the feature point belongs to the edge of the image. Analyzing the multi-dimension feature of each feature point in the current image includes: for each feature point in the current image, respectively, determining the likelihood that the feature point belongs to the edge of the image according to the difference between the coordinates of the feature point and the coordinates of the center point of the image.

[0094] It can be understood that, on the basis of corresponding to a large object and corresponding to the top of the object, if the feature point is also located at the edge of the image, the importance of the feature point to modeling is further enhanced. Since the number of feature points in the center region of the image is large, but they can only well constrain the position of the camera, while the number of feature points in the edge region of the image is small, but they are crucial to accurately constrain the rotation attitude of the camera and the lens distortion parameter, therefore, the high-quality feature point located at the edge of the image can effectively prevent the entire reconstructed model from producing "bowing" or "dome" distortion, and has very high "cost performance" and importance. Therefore, the multi-dimension feature used to determine the single importance of the feature point can include the likelihood that the feature point belongs to the edge of the image.

[0095] Since the greater the difference between the coordinates of the feature point and the coordinates of the center point of the image, the farther the feature point is from the center point of the image, the greater the likelihood that the feature point belongs to the edge of the image. Therefore, the likelihood that the feature point belongs to the edge of the image is positively correlated with the difference between the coordinates of the feature point and the coordinates of the center point of the image.

[0096] In one embodiment, the difference between the horizontal coordinate value of the feature point and the horizontal coordinate value of the center point of the image, and the difference between the vertical coordinate value of the feature point and the vertical coordinate value of the center point of the image can be calculated respectively, and the likelihood that the feature point belongs to the edge of the image is determined according to the sum of the difference between the horizontal coordinate value and the difference between the vertical coordinate value.

[0097] ​In one embodiment, in order to facilitate calculation, the coordinates of the feature points can be normalized so that the coordinate values of each pixel in the image range from 0 to 1, and thus the coordinate of the center point of the image is (0.5, 0.5). The absolute values of the differences between the x-coordinate and y-coordinate of the normalized feature points and 0.5 are calculated, and then the possibility that the feature points belong to the image edge is determined according to the sum of the absolute values of the differences. The formula is as follows:

[0098]

[0099] wherein, represents the possibility that a single feature point in a current image belongs to an image edge. represents the x-axis coordinate value of a single feature point in a current image. represents the y-axis coordinate value of a single feature point in a current image. represents a maximum-minimum normalization function. In the above embodiment, according to the difference between the coordinates of the feature points and the coordinates of the center point of the image, the possibility that the feature points belong to the image edge can be accurately determined. In one embodiment, the single importance of a feature point in a current image is determined according to the product of the bulkiness of the object to which the feature point belongs, the possibility that the feature point belongs to the top of the object, and the possibility that the feature point belongs to the image edge. The formula is as follows:

[0100] wherein,

[0101] represents the single importance of a single feature point in a current image. represents the bulkiness of the object to which a single feature point in a current image belongs.

[0102] represents the possibility that a single feature point in a current image belongs to the top of the object. represents the possibility that a single feature point in a current image belongs to an image edge.

[0103]

[0104] ​​​​​​​​​​​​​​In another embodiment, the initial single importance of the feature point can be determined by first considering the size of the object to which it belongs. Then, a further single importance is determined by multiplying the initial single importance by the probability that the feature point belongs to the top of the object. Finally, the single importance of the feature point is determined by multiplying the further single importance by the probability that the feature point belongs to the edge of the image. The formula is as follows:

[0105]

[0106]

[0107]

[0108] in, Indicates the current image Single feature point The initial single importance. Indicates the current image Single feature point The further singular importance of. Indicates the current image Single feature point The singular importance of. Indicates the current image Single feature point With the remaining pixels The distance between them. Indicates the current image Single feature point The grayscale value. Indicates the current image The remaining pixels The grayscale value. This represents an exponential function with base e. Indicates the current image Single feature point The gradient of the corresponding pixel. This represents the maximum and minimum value normalization function. Indicates the current image Single feature point The x-axis coordinate value. Indicates the current image Single feature point The y-axis coordinate value.

[0109] In one embodiment, see Figure 3 Based on the changes in the movement distance of feature points and other feature points in the current image compared to the previous image, the comparative importance of the positional movement of feature points relative to the previous image is determined, including the following steps:

[0110] Step 301, determining the possibility that the feature point belongs to the same object as the rest of the feature points in the current image.

[0111] Step 302, determining the movement consistency of the feature point relative to the rest of the feature points according to the change of the movement distance of the feature point and the rest of the feature points in the current image relative to the previous image.

[0112] The movement consistency is used to measure the consistency degree of the movement distance of the feature point in the current image relative to the previous image and the movement distance of the rest of the feature points in the current image relative to the previous image.

[0113] Step 303, weighting and summing the movement consistency of the feature point relative to each of the rest of the feature points according to the possibility that the feature point belongs to the same object as each of the rest of the feature points, and determining the comparative importance of the position movement of the feature point relative to the previous image according to the weighted sum result.

[0114] In one embodiment, the sum of the possibilities that the feature point belongs to the same object as each of the rest of the feature points is calculated, for each of the rest of the feature points of the feature point, the weight corresponding to the rest of the feature point is determined according to the ratio between the possibility that the feature point belongs to the same object as the rest of the feature point and the sum of the possibilities, the movement consistency of the feature point relative to each of the rest of the feature points is weighted and summed according to the weights corresponding to each of the rest of the feature points, and the comparative importance of the position movement of the feature point relative to the previous image is determined according to the weighted sum result.

[0115] In one embodiment, the comparative importance of the position movement of the feature point relative to the previous image can be determined according to the following formula:

[0116]

[0117] wherein, represents the comparative importance of the position movement of a single feature point in the current image relative to the previous image. represents the possibility that a single feature point in the current image belongs to the same object as the rest of the feature points. represents the movement consistency of a single feature point in the current image relative to the rest of the feature points.

[0118] ​​​​​In the above embodiment, the moving consistency of the feature point relative to each of the remaining feature points is weighted and summed according to the possibility that the feature point and each of the remaining feature points belong to the same object, so that the comparative importance of the moving condition of the feature point relative to the position in the previous image can be accurately determined.

[0119] In one embodiment, the target importance of the feature point is determined according to the product of the single importance and the comparative importance of the feature point. The formula is as follows:

[0120]

[0121] wherein, represents the target importance of a single feature point in the current image. represents the comparative importance of a single feature point in the current image relative to the moving condition of the position in the previous image. represents the single importance of a single feature point in the current image. represents the possibility that a single feature point in the current image and the remaining feature points belong to the same object. represents the moving consistency of a single feature point in the current image relative to the remaining feature points.

[0122] In one embodiment, the possibility that the feature point and the remaining feature points in the current image belong to the same object is determined by determining the possibility according to the dispersion degree of the gray value distribution of the pixel points between the feature point and the remaining feature points in the current image.

[0123] ​​​​​​​​​​​​It can be understood that the gray scale change is an important feature in image processing, which reflects the intensity change of the light reflected by the surface of the object. When the light, texture or surface characteristics of a part of the image change, it usually corresponds to the boundary or contact surface of different objects. If there is no obvious gray scale change between two feature points, it means that the two feature points can be located on the same surface or the same structure of the object. And the gray scale change in the image is usually related to the edge, joint or transition area of the object and the background. If there is no gray scale change between the feature points, it indicates that they can be in the flat or continuous part of the object and are not affected by the change of the object form. Therefore, in this case, the two feature points are more likely to belong to the same object. Therefore, the possibility that the feature points and the remaining feature points in the current image belong to the same object can be determined according to the dispersion degree of the gray scale value distribution of the pixel points between the feature points and the remaining feature points in the current image.

[0124] The possibility that the remaining feature points in the current image belong to the same object is negatively related to the dispersion degree.

[0125] For example: the dispersion degree can be measured by variance or standard deviation.

[0126] For example: the dispersion degree of the gray scale value distribution of the pixel points on the nearest path between the feature points and the remaining feature points in the current image can be used to determine the possibility that the feature points and the remaining feature points in the current image belong to the same object.

[0127] In one embodiment, the possibility that the feature points and the remaining feature points in the current image belong to the same object can be determined according to the following formula:

[0128]

[0129] wherein, represents the possibility that a single feature point in the current image and the remaining feature points belong to the same object. represents the dispersion degree of the gray scale value distribution of the pixel points between a single feature point in the current image and the remaining feature points. represents the possibility that a single feature point in the current image and the remaining feature points belong to the same object. represents the dispersion degree of the gray scale value distribution of the pixel points between a single feature point in the current image and the remaining feature points. represents the exponential function with e as the base.

[0130] In the above embodiment, the dispersion degree of the gray scale value distribution of the pixel points between the feature points and the remaining feature points in the current image can be used to accurately determine the possibility that the feature points and the remaining feature points in the current image belong to the same object.

[0131] In one embodiment, refer to Figure 4Based on the changes in the movement distance of the feature point and other feature points in the current image compared to the previous image, the consistency of the movement of the feature point relative to other feature points is determined, including the following steps:

[0132] Step 401: Determine the distance the feature point moves in the current image relative to its position in the previous image, and the distance the other feature points move in the current image relative to their positions in the previous image.

[0133] Step 402: Determine the consistency of movement of the feature point relative to the other feature points based on the difference between the movement distance of the feature point and the movement distance of the other feature points, as well as the single importance of the other feature points.

[0134] It can be understood that the smaller the difference between the movement distance of a feature point and the movement distances of other feature points, the more consistent the movement of the feature point is with the other feature points. Therefore, the consistency of a feature point's movement relative to other feature points is negatively correlated with the difference in movement distance. The greater the individual importance of other feature points, the greater their contribution weight. Therefore, the consistency of a feature point's movement relative to other feature points is positively correlated with the individual importance of other feature points.

[0135] In one embodiment, the consistency of movement of a feature point relative to other feature points can be determined according to the following formula:

[0136]

[0137] in, Indicates the current image Single feature point Compared to other feature points Movement consistency. Indicates the current image Other feature points The singular importance of. Representing feature points In the current image The position in the image relative to the previous image The distance the position moves. Representing feature points In the current image The position in the image is relative to the previous image. The distance the position moves. This represents an exponential function with base e.

[0138] In the above embodiments, based on the difference between the movement distance of the feature point and the movement distance of the other feature points, and the single importance of the other feature points, the movement consistency of the feature point relative to the other feature points can be accurately determined.

[0139] In one embodiment, the actual overlap rate is compared with the preset overlap rate, and the flight speed of the UAV is adjusted according to the comparison result, including: if the actual overlap rate is less than the preset overlap rate, the flight speed of the UAV is reduced; if the actual overlap rate is greater than the preset overlap rate, the flight speed of the UAV is increased.

[0140] It can be understood that if the actual overlap rate is less than the preset overlap rate, it indicates that the image acquisition interval is too large, and there is a risk of failure in splicing, and the flight speed of the UAV needs to be reduced to shorten the image acquisition interval; if the actual overlap rate is greater than the preset overlap rate, it indicates that the image acquisition interval is too small, the image acquisition efficiency is low, and redundant image data is generated, and the flight speed of the UAV needs to be increased to improve the image acquisition efficiency.

[0141] In one embodiment, if the actual overlap rate is equal to the preset overlap rate, the flight speed of the UAV is not adjusted, that is, the flight speed of the UAV remains unchanged.

[0142] In the above embodiment, if the actual overlap rate is less than the preset overlap rate, the flight speed of the UAV is reduced; if the actual overlap rate is greater than the preset overlap rate, the flight speed of the UAV is increased. This fine-tuning mechanism effectively offsets the influence of external factors such as wind speed changes, ensures that the image overlap rate is stable around the preset overlap rate, and thus guarantees the uniformity of the data. For smart city modeling, uniform and high-quality data is crucial, and the model is prevented from being hollow or having redundant data due to insufficient image overlap, significantly improving the accuracy and efficiency of subsequent modeling. Ultimately, this closed-loop system realizes dynamic adaptive flight path planning, providing reliable protection for the generation of high-precision smart city base maps.

[0143] Referring to Figure 5The application provides an aerial photography unmanned aerial vehicle route planning method for smart city modeling, and the whole process schematic diagram comprises the following steps: firstly, inputting a target region, a ground sampling distance and a target overlap rate (a preset overlap rate), generating a preliminary route by using a path planning algorithm, then determining the preliminary single importance of a single feature point in each image shot by the unmanned aerial vehicle by analyzing the largeness of the object corresponding to the feature point (the largeness of the object to which the feature point belongs), determining the further single importance of the single feature point in each image shot by the unmanned aerial vehicle by analyzing the top of the object corresponding to the feature point (the possibility that the feature point belongs to the top of the object), and determining the final single importance of the single feature point in each image shot by the unmanned aerial vehicle by analyzing the edginess of the feature point (the possibility that the feature point belongs to the edge), then determining the possibility that the single feature point in each image shot by the unmanned aerial vehicle belongs to the same object compared with the rest feature points, evaluating the comparative importance of the single feature point in each image shot by the unmanned aerial vehicle compared with the feature point in the previous image by using the golden rule that the same object is more important when moving and deforming less, combining the coincidence rate of the feature point under the single importance and the comparative importance to calculate the actual overlap rate of the current image and the previous image in real time, and feeding back and adjusting the flight speed of the subsequent route according to the comparison result of the actual overlap rate and the target overlap rate.

[0144] It should be understood that, although each step in the flowchart involved in each of the above-described embodiments is shown in sequence according to the arrow, these steps are not necessarily executed in the order indicated by the arrow. Unless otherwise specified herein, the execution of these steps is not strictly limited in sequence, and these steps can be executed in other orders. Moreover, at least part of the steps in the flowchart involved in each of the above-described embodiments can comprise multiple steps or stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution sequence of these steps or stages is not necessarily sequential, but can be executed alternately or alternately with at least part of other steps or stages.

[0145] The technical features of the above embodiments can be combined in any manner. In order to make the description concise, all possible combinations of the technical features in the above embodiments are not described, but as long as the combinations of the technical features do not contradict, they should be considered within the scope of the present application.

[0146] The above-described embodiments only express several implementation manners of the present application, and the description is relatively specific and detailed, but it should not be understood as a limitation on the scope of the present application. It should be pointed out that, for those skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are within the protection scope of the present application.

[0147] It is to be noted that the sequential order of the above-described embodiments of the present application only for the purpose of description, but not the advantages and disadvantages of the embodiments. The processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multi-task processing and parallel processing are also possible or can be advantageous.

[0148] Each of the embodiments in the specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other. Each embodiment focuses on the difference from the other embodiments.

Claims

1. A method for flight path planning of aerial photography drones for smart city modeling, characterized in that, The method includes: During the process of drones performing smart city modeling and photography tasks along planned routes, feature points of the current image are identified; The target importance of each feature point is determined based on its individual importance in the current image and its comparative importance relative to its positional movement in the previous image. The actual overlap rate between the current image and the previous image is determined based on the ratio of the sum of the importance of matching points to the total importance. The sum of the importance of matching points is the sum of the target importance of feature points in the current image that match those in the previous image. The total importance is the sum of the target importance of all feature points in the current image. The actual overlap rate is compared with the preset overlap rate, and the flight speed of the UAV is adjusted according to the comparison result; Determining the target importance of each feature point based on its individual importance in the current image and its comparative importance relative to its positional movement in the previous image includes: Analyze the multi-dimensional features of each feature point in the current image to determine the single importance of each feature point in the current image; For each feature point, the comparative importance of the positional movement of the feature point relative to the previous image is determined based on the change in the movement distance of the feature point and other feature points in the current image and the previous image. The target importance of the feature point is determined based on its single importance and comparative importance.

2. The aerial photography UAV flight path planning method for smart city modeling according to claim 1, characterized in that, The multi-dimensional features include the size of the object to which the feature point belongs; The analysis of the multi-dimensional features of each feature point in the current image includes: For each feature point in the current image, the grayscale value of the feature point is compared with that of the other pixels in the image to determine the size of the object to which the feature point belongs.

3. The aerial photography UAV flight path planning method for smart city modeling according to claim 2, characterized in that, The step of comparing the grayscale value of the feature point with that of other pixels in the image to determine the size of the object to which the feature point belongs includes: Based on the difference between the feature point and the gray values ​​of all other pixels in the image, a gravitational contribution value is determined for each of the other pixels; the gravitational contribution value is negatively correlated with the difference between the gray values. Based on the distance between each of the remaining pixels and the feature point, the giantness contribution values ​​corresponding to each of the remaining pixels are weighted and summed, and the giantness of the object to which the feature point belongs is determined based on the weighted summation result.

4. The aerial photography UAV flight path planning method for smart city modeling according to claim 1, characterized in that, The multi-dimensional features include the probability that a feature point belongs to the top of an object; The analysis of the multi-dimensional features of each feature point in the current image includes: Based on the gradient of the pixel corresponding to each feature point in the current image, the probability that each feature point belongs to the top of the object is determined.

5. The aerial photography UAV flight path planning method for smart city modeling according to claim 1, characterized in that, The multi-dimensional features include the probability that a feature point belongs to an image edge; The analysis of the multi-dimensional features of each feature point in the current image includes: For each feature point in the current image, the probability that the feature point belongs to the image edge is determined based on the difference between the coordinates of the feature point and the coordinates of the image center point.

6. The aerial photography UAV flight path planning method for smart city modeling according to claim 1, characterized in that, The step of determining the comparative importance of the positional movement of a feature point relative to the previous image based on the changes in the movement distance of the feature point and other feature points in the current image compared to the previous image includes: Determine the probability that the feature point and other feature points in the current image belong to the same object; Based on the changes in the movement distance of the feature point and the other feature points in the current image and the previous image, the consistency of movement of the feature point relative to the other feature points is determined; Based on the probability that the feature point belongs to the same object as each of the other feature points, a weighted sum is performed on the movement consistency of the feature point relative to each of the other feature points, and the comparative importance of the position movement of the feature point relative to the previous image is determined based on the weighted sum result.

7. The aerial photography UAV flight path planning method for smart city modeling according to claim 6, characterized in that, Determining the probability that the feature point and other feature points in the current image belong to the same object includes: Based on the degree of dispersion of the gray value distribution of the pixels between the feature point and the other feature points in the current image, the probability that the feature point and the other feature points in the current image belong to the same object is determined.

8. The aerial photography UAV flight path planning method for smart city modeling according to claim 6, characterized in that, Determining the consistency of movement of the feature point relative to the other feature points based on the change in movement distance between the feature point and the other feature points in the current image and the previous image includes: Determine the distance the feature point moves in the current image relative to its position in the previous image, and the distance the remaining feature points move in the current image relative to their positions in the previous image; The consistency of movement of the feature point relative to the remaining feature points is determined based on the difference between the movement distance of the feature point and the movement distance of the remaining feature points, and the single importance of the remaining feature points.

9. The aerial photography UAV flight path planning method for smart city modeling according to any one of claims 1 to 8, characterized in that, The step of comparing the actual overlap rate with the preset overlap rate and adjusting the flight speed of the UAV based on the comparison result includes: If the actual overlap rate is less than the preset overlap rate, then reduce the flight speed of the UAV; If the actual overlap rate is greater than the preset overlap rate, the flight speed of the UAV is increased.

Images