Planning method and system for oblique photogrammetry of unmanned aerial vehicle, and electronic equipment

Abstract

The invention discloses a planning method and system for oblique photogrammetry of an unmanned aerial vehicle and electronic equipment, and relates to the technical field of oblique photogrammetry of the unmanned aerial vehicle, and the method comprises the steps: connecting an unmanned aerial vehicle control terminal, and obtaining a plurality of initial oblique photogrammetry paths; performing path similarity calculation on the plurality of initial oblique photogrammetry paths, and outputting a path similarity set; a building area where the unmanned aerial vehicle group performs oblique photogrammetry is recorded and identified, a building feature vector group is extracted, lateral overlapping degree adaptation calculation is performed, and an adaptive lateral overlapping degree is output; and re-planning and identifying the oblique photography angle of the unmanned aerial vehicle group, and outputting an optimized oblique photography measurement path. The technical problems that in the prior art, a multi-unmanned-aerial-vehicle oblique photogrammetry path lacks reasonable planning, the lateral overlapping degree cannot be adaptively adjusted according to building area characteristics, and consequently the accuracy and efficiency of measured data are insufficient are solved, and the technical effect of improving the unmanned-aerial-vehicle oblique photogrammetry efficiency and the accuracy of the measured data is achieved.

Description

Technical Field

[0001] This application relates to the technical field of drone oblique photogrammetry, and particularly to a planning method, system and electronic device for drone oblique photogrammetry. Background Art

[0002] With the rapid development of urban construction and the continuous improvement of the accuracy requirements for geographic information data, traditional surveying and mapping methods are difficult to meet the diverse and high-precision measurement needs. Drone oblique photogrammetry can truly reflect the actual situation of ground objects by collecting image data from multiple angles simultaneously with multiple sensors mounted on the same flight platform, providing a rich data source for urban modeling, geographic information system updating, etc. However, in practical applications, when using multiple drones for oblique photogrammetry simultaneously, the initial oblique photogrammetry path planning of different drones often lacks systematicness and coordination. If the paths of different drones are highly similar, it may lead to redundant shooting of some areas and insufficient coverage of other areas, not only wasting flight resources but also making it difficult to ensure the integrity and accuracy of measurement data. At the same time, due to the different building characteristics in different building areas, the traditional fixed side overlap degree setting cannot be adaptively adjusted according to the actual situation, making it difficult to achieve efficient and accurate oblique photogrammetry, thus affecting the overall measurement efficiency and quality.

[0003] Therefore, in the current related technologies, there are technical problems such as the lack of reasonable planning for the paths of multi-drone oblique photogrammetry and the inability to adaptively adjust the side overlap degree according to the characteristics of building areas, resulting in insufficient accuracy and efficiency of measurement data. Summary of the Invention

[0004] This application provides a planning method, system and electronic device for drone oblique photogrammetry, solving the technical problems in the prior art that the paths of multi-drone oblique photogrammetry lack reasonable planning and the side overlap degree cannot be adaptively adjusted according to the characteristics of building areas, resulting in insufficient accuracy and efficiency of measurement data, and achieving the technical effect of improving the efficiency of drone oblique photogrammetry and the accuracy of measurement data.

[0005] The present application provides a planning method for drone oblique photogrammetry. The method includes: respectively arranging an attached sensor group in a storage container within a storage space to obtain an array of arranged attached sensor groups, where the storage space is a semi-closed space; identifying a set of environmental monitoring points in the storage space, and arranging an environmental sensor group at each environmental monitoring point to obtain an array of arranged environmental sensor groups; collecting monitoring data of the attached sensor group array within a preset monitoring window to obtain an array of sequences of attached sensor monitoring data groups; traversing the array of sequences of attached sensor monitoring data groups for in-group fusion iterative analysis to determine an array of container state factors; within the preset monitoring window, performing array interaction hazard boundary identification on the environmental sensor group array to determine an environmental hazard area and an environmental hazard state factor; and performing storage environmental pollution early warning analysis based on the array of container state factors, the environmental hazard area, and the environmental hazard state factor to obtain a first storage environmental pollution early warning information.

[0006] The present application also provides a planning system for drone oblique photogrammetry, including: an initial oblique photogrammetry path acquisition module, configured to connect to a drone control terminal to obtain a plurality of initial oblique photogrammetry paths corresponding to a plurality of drones; a path similarity calculation module, configured to calculate the path similarity of the plurality of initial oblique photogrammetry paths and output a set of path similarities; a lateral overlap degree adaptation calculation module, configured to identify, according to the set of path similarities, an identified drone group with adjacent path similarities greater than a preset similarity threshold, record the building area where the identified drone group performs oblique photogrammetry, extract a building feature vector group of the building area, input the building feature vector group into a dynamic overlap degree calculation model for lateral overlap degree adaptation calculation, and output an adapted lateral overlap degree; and an optimized oblique photogrammetry path output module, configured to enable the drone control terminal to re-plan the oblique photography angle of the identified drone group according to the adapted lateral overlap degree and output an optimized oblique photogrammetry path.

[0007] The present application also provides an electronic device, including: a memory, configured to store executable instructions; and a processor, configured to implement a planning method for drone oblique photogrammetry when executing the executable instructions stored in the memory.

[0008] A planning method, system and electronic device for drone oblique photogrammetry proposed in this application are connected to the drone control terminal to obtain multiple initial oblique photogrammetry paths; calculate the path similarity of the multiple initial oblique photogrammetry paths and output a set of path similarities; record the building areas where the drone group conducts oblique photogrammetry, extract the building feature vector group, perform side overlap degree adaptation calculation, and output the adapted side overlap degree; re-plan the oblique photography angle of the identified drone group and output the optimized oblique photogrammetry path. This solves the technical problems in the prior art that the oblique photogrammetry paths of multiple drones lack reasonable planning and the side overlap degree cannot be adaptively adjusted according to the characteristics of the building area, resulting in insufficient accuracy and efficiency of the measurement data, and achieves the technical effect of improving the efficiency of drone oblique photogrammetry and the accuracy of the measurement data. Brief Description of the Drawings

[0009] To more clearly illustrate the technical solutions of the embodiments of the present disclosure, the accompanying drawings of the embodiments of the present disclosure will be briefly introduced below. Flowcharts are used in this application to illustrate the operations performed by the system according to the embodiments of the present application. It should be understood that the operations before or below do not necessarily need to be executed precisely in sequence. On the contrary, according to needs, various steps can be executed in reverse order or simultaneously. At the same time, other operations can also be added to these processes, or one or several operations can be removed from these processes.

[0010] Figure 1 Schematic flowchart of a planning method for drone oblique photogrammetry provided by an embodiment of the present application.

[0011] Figure 2 Schematic structural diagram of a planning system for drone oblique photogrammetry provided by an embodiment of the present application.

[0012] Figure 3 Schematic structural diagram of an electronic device provided by an embodiment of the present application.

[0013] Explanation of reference numerals: Initial oblique photogrammetry path acquisition module 10, path similarity calculation module 20, side overlap degree adaptation calculation module 30, optimized oblique photogrammetry path output module 40, input device 401, processor 402, memory 403, output device 404. Detailed Description of the Embodiments

[0014] The above description is only an overview of the technical solutions of this application. In order to be able to understand the technical means of this application more clearly, it can be implemented according to the content of the description. And in order to make the above and other purposes, features and advantages of this application more obvious and understandable, the following specifically illustrates the specific embodiments of this application.

[0015] To make the objectives, technical solutions, and advantages of this application clearer, the following will further describe this application in detail with reference to the accompanying drawings. The described embodiments should not be construed as limitations on this application. All other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the scope of protection of this application.

[0016] In the following description, reference is made to "some embodiments", which describe a subset of all possible embodiments. However, it can be understood that "some embodiments" can be the same subset or different subsets of all possible embodiments and can be combined with each other without conflict. The terms "first / second" involved are only used to distinguish similar objects and do not represent a specific order for the objects. The terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or server that includes a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or modules that are not clearly listed or are inherent to these processes, methods, products, or devices. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the technical field to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application.

[0017] An embodiment of this application provides a method for planning an unmanned aerial vehicle (UAV) oblique photogrammetry, as Figure 1 shown. The method includes: Step S100: Connect to the UAV control terminal and obtain multiple initial oblique photogrammetry paths corresponding to multiple UAVs.

[0018] Preferably, through a specific communication protocol and interface, using hardware devices (such as wireless communication modules, data lines, etc.) or software programs (such as specialized UAV control software), establish a connection between the device for planning and controlling the UAV flight and the control terminals of each UAV to achieve remote control and data interaction of the UAV, thereby obtaining the initial oblique photogrammetry paths. That is, send instructions to each UAV through the control terminal, requiring it to return the preset or current existing oblique photogrammetry path information. Usually, before the task starts, according to factors such as the scope of the measurement area, terrain and landform, building distribution, etc., as well as the accuracy requirements of photogrammetry and the performance parameters of the UAV, etc., it is specified by a professional path planning algorithm. Each UAV has its own independent initial path to ensure that the target area can be photographed from different angles when performing the oblique photogrammetry task and rich image data can be obtained.

[0019] Step S200: Calculate the path similarity of the multiple initial oblique photogrammetry paths and output a set of path similarities.

[0020] Preferably, path similarity calculation is performed on multiple initial oblique photogrammetry paths. Specifically, each initial oblique photogrammetry path is regarded as a trajectory composed of multiple coordinate points, which records different positions of the UAV during flight. Then, these path coordinate points are extracted, and information that can represent path features is further extracted, such as the length, direction, curvature change, coordinates of key turning points, etc. The extracted features can reflect the shape and trend of the path to a certain extent. Then, a similarity calculation method is selected, such as Euclidean distance, cosine similarity, dynamic time warping (DTW), etc. For example, the Euclidean distance calculates the sum of the distances between corresponding coordinate points on two paths. The smaller the distance, the more similar the two paths are. Cosine similarity focuses on measuring the consistency of the directions of two paths and judges the similarity by calculating the cosine value of the angle between path vectors. The closer the value is to 1, the more similar the path directions are. All the initial oblique photogrammetry paths are combined in pairs, and then the selected similarity calculation method is used to calculate the similarity between each pair of paths, obtaining a set containing the similarities of all path pairs, that is, the output similarity set, which comprehensively reflects the similarity relationship between multiple initial oblique photogrammetry paths.

[0021] Further, step S200 further includes step S210 of calibrating the coordinate systems of the multiple initial oblique photogrammetry paths, and performing path similarity feature extraction according to the calibrated coordinate systems, including horizontal speed, vertical climb rate, heading angle change rate, and path curvature; step S220, performing DTW similarity calculation on the multiple initial oblique photogrammetry paths according to the horizontal speed, vertical climb rate, heading angle change rate, and path curvature, and outputting a similarity matrix, where the similarity matrix includes each path and a path similarity set corresponding to each path.

[0022] Preferably, calibrate the coordinate systems of multiple initial oblique photogrammetry paths so as to be able to accurately compare, that is, unify these initial oblique photogrammetry paths into the same coordinate system. Through coordinate transformation, the coordinates of all points on the paths are transformed into this coordinate system, and then path similarity features are extracted according to the calibrated coordinate system, including extracting the horizontal speed, vertical climb rate, heading angle change rate, and path curvature. Among them, the horizontal speed refers to the speed at which the unmanned aerial vehicle moves in the horizontal direction, reflecting the speed of the unmanned aerial vehicle flying on the horizontal plane, and is used to judge the movement mode of the path in the horizontal direction. It is determined by calculating the ratio of the displacement of adjacent coordinate points in the horizontal direction (usually the x-y plane) to the time interval; the vertical climb rate represents the speed at which the unmanned aerial vehicle rises or falls in the vertical direction, reflecting the change of the path in the vertical height, and is used to analyze the flight mode of the unmanned aerial vehicle under different terrains or mission requirements. It is determined by calculating the ratio of the displacement of adjacent coordinate points in the vertical direction (usually the z-axis direction) to the time interval; the heading angle is the angle between the flight direction of the unmanned aerial vehicle and the reference direction (such as the due north direction), and the heading angle change rate reflects the speed of the change of the flight direction of the unmanned aerial vehicle, which can help judge the turning situation of the path and is used to identify the bending degree and direction change of the path. It is obtained by calculating the ratio of the difference between the heading angles at adjacent moments to the time interval; the path curvature describes the bending degree of the path. The greater the curvature, the more curved the path. If the curvature is zero, the path is a straight line. The path curvature is obtained by mathematical methods, such as calculating the change rate of the tangent direction at a certain point on the path.

[0023] Preferably, perform DTW similarity calculation on multiple initial oblique photogrammetry paths according to the horizontal speed, vertical climb rate, heading angle change rate, and path curvature. Among them, DTW (Dynamic Time Warping) is used to calculate the similarity between two time series, especially suitable for processing sequences with different lengths and inconsistent time scales. Specifically, in actual situations, the flight paths of different unmanned aerial vehicles may have different lengths of time series of the same features and the time points may not correspond exactly due to factors such as flight speed and attitude adjustment. Therefore, the horizontal speed, vertical climb rate, heading angle change rate, and path curvature of each path are regarded as a multi-dimensional time series. The DTW algorithm finds the optimal matching relationship between the two sequences by elastically stretching and compressing the time axis, so as to accurately calculate their similarity. That is, for any two paths, perform DTW calculations on their horizontal speed sequences, vertical climb rate sequences, heading angle change rate sequences, and path curvature sequences respectively, obtain four similarity values and perform weighted summation to obtain the comprehensive similarity of the two paths. Similarly, calculate the similarities of multiple paths; assume there are m initial paths, the similarity matrix is an m×m matrix S, and the element in the i-th row and j-th column of the matrix represents the comprehensive similarity between path i and j. Obviously = 1 (because a path is completely similar to itself), and = (The similarity is symmetric), thus intuitively showing the similarity degree between any two paths. The similarity matrix includes each path and the set of path similarities corresponding to each path. By analyzing the elements in the matrix, pairs of paths with higher similarity can be found. Suppose there are five paths as , , , , . An example of the similarity matrix obtained through DTW calculation is shown in Table 1: Table 1 Example data of the DTW similarity matrix for oblique photography paths <![CDATA[P1]]> <![CDATA[P2]]> <![CDATA[P3]]> <![CDATA[P4]]> <![CDATA[P5]]> <![CDATA[P1]]> 1.0 0.7 0.3 0.5 0.1 <![CDATA[P2]]> 0.7 1.0 0.4 0.6 0.2 <![CDATA[P3]]> 0.3 0.4 1.0 0.2 0.3 <![CDATA[P4]]> 0.5 0.6 0.2 1.0 0.3 <![CDATA[P5]]> 0.1 0.2 0.6 0.3 1.0 It can be seen from the matrix that and have a similarity of 0.7 and are relatively similar, while and have a similarity of 0.1 and have a large difference.

[0024] Step S300, identify the marked unmanned aerial vehicle (UAV) groups with adjacent path similarities greater than a preset similarity threshold according to the set of path similarities, record the building areas where the marked UAV groups perform oblique photography surveys, extract the building feature vector groups of the building areas, input the building feature vector groups into a dynamic overlap degree calculation model for cross-track overlap degree adaptation calculation, and output the adapted cross-track overlap degree.

[0025] Preferably, identify the marked UAV groups with adjacent path similarities greater than a preset similarity threshold according to the set of path similarities. The preset similarity threshold is a standard value preset based on historical experience for judging whether the similarity between two paths is high enough. For example, if it is desired to more strictly screen out similar paths, the threshold can be set higher; conversely, it can be set lower. Specifically, in the set of path similarities, find the paths with adjacent path similarities greater than the preset similarity threshold, and group the UAVs with these highly similar adjacent paths together to form marked UAV groups. There may be overlapping flight paths, repeated shooting areas, etc. When the UAV groups perform oblique photography survey tasks, they will all cover a certain building area, which are the target areas that need to be imaged. Obtain the specific scope and boundary information of these target areas, and determine the building areas through the flight trajectories of the UAVs, the geographical location information of the captured images, etc.

[0026] Preferably, various attributes and characteristics of the buildings are quantified to form feature vectors, so as to more accurately describe the characteristics of the buildings in the building area. The building characteristics may include the height, area, shape, orientation, roof type, etc. of the buildings. For example, for a rectangular building, its length, width, height, and orientation angle and other parameters can be used to form a feature vector. Furthermore, corresponding feature vectors are extracted for all the buildings in the building area, and these feature vectors are combined together to form a building feature vector group, which can comprehensively reflect the overall characteristics of the buildings in the building area. The lateral overlap degree adaptation calculation of the building feature vector group is calculated by using the dynamic overlap degree calculation model, and the adapted lateral overlap degree is output, that is, in the current building feature situation, it can not only ensure the acquisition of complete building image information, but also make the image stitching effect better.

[0027] Further, step S300 further includes step S310 of extracting the building feature items corresponding to the building area of each drone, where the building feature items include the building top mark height, building density, building layer type, and obstacle distribution; step S320 of performing quantization processing on the building feature items to output building feature quantization parameters; step S330 of configuring weight factors for the building top mark height, building density, building layer type, and obstacle distribution, and processing the building feature quantization parameters according to the weight factors to obtain building feature vectors.

[0028] Preferably, each drone has its specific flight path during oblique photogrammetry, and the covered area is the corresponding building area. For example, the drone flies according to the planned flight route, and the range where the buildings are located during its flight is the building area corresponding to the drone. Then, building feature items such as the building top mark height, building density, building layer type, and obstacle distribution are extracted. Among them, the building top mark height refers to the height of the building top relative to a certain reference plane (such as sea level or local ground), which is used to judge the scale of the building and the possible occlusion in photogrammetry. For example, high-rise buildings may affect the shooting of surrounding low-rise buildings. The building density reflects the density of the building distribution in the building area, which is measured by calculating the number of buildings per unit area, the ratio of the building floor area to the total area of the area, etc. Dense building areas may require higher shooting accuracy and appropriate overlap degrees during photogrammetry. The building layer type refers to the classification of the number of floors of the building, such as single-story, multi-story, high-rise, etc. Buildings of different layer types are different in structure and appearance, and the requirements for photogrammetry will also vary. For example, high-rise buildings need to be photographed from more angles to obtain complete information. The obstacle distribution includes the distribution of other obstacles in the building area except buildings, such as the distribution of trees, hills, large billboards, etc., which may affect the flight path and shooting angle of the drone.

[0029] Preferably, each extracted building feature item is quantified and converted into a numerical form. Specifically, the height value obtained by measurement (unit such as meters) is directly used to quantify the height of the building top mark; the building density is quantified by specific values obtained through calculation, such as the number of buildings or the proportion of building floor area per square kilometer; it is quantified by means of coding. For example, a single - layer building is coded as 1, a multi - layer building is coded as 2, a high - rise building is coded as 3, etc. to quantify the building type; the obstacle distribution is quantified into a numerical value according to factors such as the proportion and quantity of the coverage area of the obstacles. For example, if the proportion of the obstacle coverage area in the total area of the region is 20%, the quantified value can be 0.2. After quantization processing, building feature quantization parameters are obtained. According to actual requirements and experience, weight factors are configured for the building top mark height, building density, building layer type, and obstacle distribution respectively to reflect the importance of each feature item in comprehensively evaluating building features. The quantified building feature parameters are multiplied by the corresponding weight factors, and then the products are added together to obtain a comprehensive value, which is combined with the values obtained after the same processing of other feature items to form a building feature vector, comprehensively reflecting the comprehensive features of the buildings in the building area.

[0030] Further, step S300 further includes step S340 of collecting a sample group of building feature vectors for training; step S350 of setting constraint conditions, where the constraint conditions include the maximum and minimum values of the lateral overlap degree and a preset overlap degree difference, and the preset overlap degree difference is used to limit the excessive difference in the overlap degree of adjacent building areas; step S360 of constructing an AdamW optimizer, and under the constraint conditions, using the AdamW optimizer to train the building feature vector samples and the labels representing the lateral overlap degree to obtain a lateral overlap degree adaptation calculation model, and using the lateral overlap degree adaptation calculation model to calculate the building feature vectors.

[0031] Preferably, by conducting actual measurements, data collection, and processing in different building areas, a large number of building feature vector group samples are collected. Each sample corresponds to a specific building area and contains various feature information of the buildings in that area. After quantization processing, these feature information form building feature vectors, such as the vectors composed of the quantization of features such as the building top height, building density, building layer type, and obstacle distribution mentioned above. Then, constraint conditions are set, including the maximum and minimum values of the lateral overlap degree and a preset overlap degree difference. Among them, setting the maximum value is to avoid excessive data redundancy caused by too high an overlap degree, wasting storage resources and processing time; setting the minimum value is to ensure there is sufficient overlap so that accurate results can be obtained during subsequent image stitching and 3D modeling. For example, the maximum value of the lateral overlap degree may be set to 80% and the minimum value to 30%. The preset overlap degree difference is used to limit the situation where the overlap degree difference between adjacent building areas is too large. If the overlap degree difference between adjacent building areas is too large, it may affect the data consistency within the entire measurement area. The preset overlap degree difference is used to limit this situation from occurring and ensure that the overlap degrees of adjacent areas do not differ too much. For example, it is set that the overlap degree difference between adjacent building areas cannot exceed 10%, making the image acquisition in the entire measurement area more uniform and reasonable.

[0032] Preferably, an AdamW optimizer is constructed. Based on the constructed constraint conditions, the AdamW optimizer is used to train the building feature vector samples and the labels representing the lateral overlap degree to obtain a lateral overlap degree adaptation calculation model. Specifically, AdamW combines the ideas of the Adam optimization algorithm and weight decay (L2 regularization) to adjust the model parameters so that the prediction results of the model are as close as possible to the true values. When constructing the AdamW optimizer, some hyperparameters need to be set, such as the learning rate and the weight decay coefficient. The learning rate controls the step size of each parameter update, and the weight decay coefficient is used to prevent the model from overfitting. By constraining the model parameters, the parameter values will not be too large. For example, Table 2 shows example data of hyperparameter combinations when using the AdamW optimizer to train the model: Table 2 Table of Related Parameters of AdamW Optimizer Hyperparameters Hyperparameter Name Meaning Value Range Example Learning Rate Controls the step size of each parameter update and determines the learning speed of the model during training <![CDATA[Usually between 10 -1 and 10 -6 with common values such as 0.001, 0.0001]]> For example, setting it to 0.001 means that each time the parameter is updated, the change in the parameter is adjusted by multiplying the calculated gradient by 0.001 Weight Decay Coefficient Used to implement L2 regularization, constrain the parameters of the model, prevent the parameter values from being too large, and thus avoid overfitting Common value ranges from 0.0001 to 0.1, such as 0.001 When the weight decay coefficient is set to 0.001, a penalty term proportional to the sum of the squares of the parameters will be added to the loss function, and the penalty strength is 0.001 times β1 (Exponential Decay Rate of the First Moment Estimate) Used to calculate the first moment estimate of the gradient (i.e., momentum), and control the influence degree of the historical gradient on the current update Usually takes a value close to 1, such as 0.9 <![CDATA[When β1 is set to 0.9, it means that the current gradient update will consider 90% of the previous gradient information (in an exponentially decaying manner).]]> β2 (Exponential Decay Rate of the Second Moment Estimate) Used to calculate the second moment estimate of the gradient, and control the influence degree of the square of the historical gradient Usually takes a value close to 1, such as 0.999 <![CDATA[When β2 is set to 0.999, it means that the current update will consider the previous 99.9% of the gradient squared information (in an exponentially decaying manner). <!-- 6 -->]]> ε (Small Constant for Numerical Stability) Prevents division by zero in calculations and is a very small positive number <![CDATA[Common values such as 10 -8 > <![CDATA[For example, when ε takes the value of 10 -8 , when calculating operations involving the denominator (such as the calculation of the adaptive learning rate), this small constant is added to ensure the stability of numerical calculations]]> Preferably, the collected building feature vector samples are used as the input of the model, and the actual side overlap degree corresponding to each sample is used as the label (i.e., the correct result that the model should predict). Using the AdamW optimizer, under the previously set constraints (the maximum and minimum values of the side overlap degree and the preset overlap degree difference), the model is trained. During the training process, the model makes predictions based on the input building feature vector samples to obtain a predicted value of the side overlap degree. Then, the error between the predicted value and the label is calculated (usually measured using a loss function). The AdamW optimizer adjusts the parameters of the model according to this error to gradually reduce the error. At the same time, due to the existence of constraints, the model will be restricted during the process of adjusting parameters to ensure that the predicted side overlap degree is within a reasonable range. After multiple iterative trainings, when the prediction results of the model can meet certain performance requirements on both the training set and the validation set (such as the loss function converges to a small value and the prediction error on the validation set is within an acceptable range), the side overlap degree adaptation calculation model is obtained. The new building feature vector is input into the trained side overlap degree adaptation calculation model to calculate the side overlap degree value suitable for the building area, which is used to guide the setting of the shooting parameters when the drone conducts oblique photogrammetry in this area to obtain better measurement results and ensure the quality and accuracy of the measurement data.

[0033] Further, step S340 further includes step S341 of performing class balance identification on each building feature item in the building feature vector to obtain identified building feature items, where the identified building feature items are unbalanced small sample feature items; step S342 of performing SMOTE oversampling compensation on the identified building feature items and outputting the compensated building feature vector.

[0034] Preferably, class balance identification is performed on each building feature item in the building feature vector, that is, the small sample feature items with unbalanced classes are identified and determined as the identified building feature items, and then SMOTE oversampling compensation is performed on the identified building feature items. Among them, SMOTE is an oversampling method used to solve the problem of class imbalance in the dataset. That is, for the identified building feature items (i.e., unbalanced small sample feature items), the SMOTE algorithm generates new samples by interpolating in the feature space of these small sample feature items. Specifically, the nearest neighbor samples of the small sample feature items are found, and then new sample points are randomly generated between them to increase the number of samples of the small sample feature items and make the number of feature items in each class more balanced. After SMOTE oversampling compensation, the compensated building feature vector is output, which can improve the data distribution and avoid the poor learning effect of the model on small sample feature items due to data imbalance during the training process, and improve the accuracy and generalization ability of the model.

[0035] Further, step S360 further includes that the calculation formula of the lateral overlap degree adaptation calculation model is as follows: ; ; where, is the adapted lateral overlap degree after the UAV group adjustment, is the initial overlap degree of the UAV group, is the number of the building feature vector groups, is the contribution ratio of the d-th feature to the lateral overlap degree adjustment, is the standardized adjustment value of the d-th feature, is the actual feature value, is the feature mean value trained based on the building feature vector samples, is the standard deviation trained based on the building feature vector samples, is the scaling factor. Among them, high-weight features (such as building density) play a dominant role in the overlap degree adjustment; low-weight features (such as the number of surrounding obstacles) have less influence and may only be significant in specific scenarios.

[0036] Step S400, the UAV control terminal re-plans the oblique photography angles of the marked UAV group according to the adapted lateral overlap degree, and outputs an optimized oblique photogrammetry path.

[0037] Preferably, according to the requirements of the adapted lateral overlap degree, the UAV control terminal re-sets the oblique photography angles of each UAV in the marked UAV group. Specifically, the oblique photography angle determines the tilting direction and angle of the camera when the UAV takes pictures. Different oblique photography angles will affect the content and range of the captured images. By adjusting the oblique photography angle, the shooting perspective of the UAV can be changed, so that the captured images can better meet the requirements of the adapted lateral overlap degree, avoiding redundant shooting or missing important areas. For example, if the adapted lateral overlap degree requirement for a certain building area is high, it may be necessary to increase the oblique photography angle of the UAV to increase the overlapping part of the images. On the contrary, if the adapted lateral overlap degree requirement is low, the oblique photography angle can be appropriately reduced; after re-planning the oblique photography angles of each UAV, the UAV control terminal generates a new flight path for each UAV in the marked UAV group according to the new oblique photography angle and other relevant flight parameters (such as flight altitude, speed, etc.), that is, an optimized oblique photogrammetry path, which can enable the UAV to perform oblique photogrammetry on the building area more efficiently and accurately under the premise of meeting the adapted lateral overlap degree, improving the quality of the measurement data and the efficiency of subsequent processing.

[0038] Further, step S400 further includes step S410, where the UAV control terminal re-plans the oblique photography angles of the marked UAV group according to the adapted lateral overlap degree, where the marked UAV group consists of adjacent UAVs with a path similarity greater than a preset similarity threshold; step S420, decomposes the adapted lateral overlap degree according to the UAV parameters of the marked UAV group to obtain the oblique photography angles of the marked UAV group, including a first oblique photography angle and a second oblique photography angle.

[0039] Preferably, the UAV control terminal re-plans the oblique photography angles of the marked UAV group according to the adapted lateral overlap degree. The UAV control terminal needs to re-plan the oblique photography angles of the marked UAV group according to the adapted lateral overlap degree, including a first oblique photography angle and a second oblique photography angle. Specifically, the UAV parameters may include the UAV model, sensor characteristics, flight altitude, speed, etc. Different UAV parameters will affect its shooting ability and image overlap situation. For example, a UAV with a higher flight altitude may require a larger oblique photography angle to achieve the same lateral overlap degree. Different sensor resolutions may also have different requirements for the oblique photography angle. According to these UAV parameters, as well as geometric optics principles, photogrammetry principles, etc., the adapted lateral overlap degree is converted into specific requirements for the oblique photography angle, that is, considering the mutual relationship between various factors. For example, according to the flight altitude and camera focal length, calculate the angle range that the UAV needs to tilt under a certain lateral overlap degree requirement. Then, through further analysis and calculation, this angle range is refined into a first oblique photography angle and a second oblique photography angle. Among them, the first oblique photography angle and the second oblique photography angle may respectively correspond to different shooting directions or different shooting stages. For example, the first oblique photography angle may be the initial tilt angle of the UAV when approaching the building area, used to initially obtain the images of the building side; the second oblique photography angle may be the angle adjusted when the UAV flies above the building area or a specific position to better capture the building top and other details, while ensuring a suitable lateral overlap degree with the images taken by adjacent UAVs. Through two different oblique photography angles, the image information of the building area can be obtained more comprehensively and accurately, and the requirement of the adapted lateral overlap degree can be met, thereby improving the accuracy and quality of UAV photogrammetry.

[0040] In the above text, reference is made to Figure 1 A method for planning UAV oblique photogrammetry according to an embodiment of the present invention is described in detail. Next, a system for planning UAV oblique photogrammetry according to an embodiment of the present invention will be described with reference to Figure 2

[0041] ​A planning system for drone oblique photogrammetry according to an embodiment of the present invention is used to solve the technical problems existing in the prior art, such as the lack of reasonable planning for the paths of multi-drone oblique photogrammetry and the inability to adaptively adjust the side overlap degree according to the characteristics of the building area, resulting in insufficient accuracy and efficiency of the measurement data. It achieves the technical effect of improving the efficiency of drone oblique photogrammetry and the accuracy of the measurement data. As Figure 2 shown, a planning system for drone oblique photogrammetry includes: an initial oblique photogrammetry path acquisition module 10, a path similarity calculation module 20, a side overlap degree adaptation calculation module 30, and an optimized oblique photogrammetry path output module 40.

[0042] The initial oblique photogrammetry path acquisition module 10 is used to connect to the drone control terminal and obtain multiple initial oblique photogrammetry paths corresponding to multiple drones; the path similarity calculation module 20 is used to calculate the path similarity of the multiple initial oblique photogrammetry paths and output a path similarity set; the side overlap degree adaptation calculation module 30 is used to identify the marked drone groups with adjacent path similarities greater than a preset similarity threshold according to the path similarity set, record the building areas where the marked drone groups perform oblique photogrammetry, extract the building feature vector groups of the building areas, input the building feature vector groups into a dynamic overlap degree calculation model for side overlap degree adaptation calculation, and output an adapted side overlap degree; the optimized oblique photogrammetry path output module 40 is used to re-plan the oblique photography angles of the marked drone groups by the drone control terminal according to the adapted side overlap degree and output an optimized oblique photogrammetry path.

[0043] Next, the specific configuration of the side overlap degree adaptation calculation module 30 will be described in detail. The side overlap degree adaptation calculation module 30 further includes: extracting the building feature items corresponding to each drone's building area, where the building feature items include the building top mark height, building density, building layer type, and obstacle distribution; performing quantization processing on the building feature items and outputting building feature quantization parameters; configuring the weight factors of the building top mark height, building density, building layer type, and obstacle distribution, and processing the building feature quantization parameters according to the weight factors to obtain building feature vectors.

[0044] Next, the specific configuration of the lateral overlap degree adaptation calculation module 30 will be further described in detail. The lateral overlap degree adaptation calculation module 30 further includes: collecting a set of building feature vector samples for training; setting constraint conditions, where the constraint conditions include the maximum and minimum values of the lateral overlap degree, and a preset overlap degree difference, and the preset overlap degree difference is used to limit the excessive difference in the overlap degree of adjacent building areas; constructing an AdamW optimizer, and under the constraint conditions, using the AdamW optimizer to train the building feature vector samples and the labels representing the lateral overlap degree to obtain a lateral overlap degree adaptation calculation model, and using the lateral overlap degree adaptation calculation model to calculate the building feature vectors.

[0045] Next, the specific configuration of the lateral overlap degree adaptation calculation module 30 will be further described in detail. The lateral overlap degree adaptation calculation module 30 further includes: performing class balance recognition on each building feature item in the building feature vector to obtain an identified building feature item, where the identified building feature item is a small sample feature item with imbalance; performing SMOTE oversampling compensation on the identified building feature item, and outputting the compensated building feature vector.

[0046] Next, the specific configuration of the lateral overlap degree adaptation calculation module 30 will be further described in detail. The calculation formula of the lateral overlap degree adaptation calculation model of the lateral overlap degree adaptation calculation module 30 is as follows: ; ; where is the adapted lateral overlap degree after identifying the adjustment of the drone group, is the initial overlap degree of the identified drone group, is the number of the set of building feature vectors, is the contribution ratio of the d-th feature to the adjustment of the lateral overlap degree, is the standardized adjustment value of the d-th feature, is the actual value of the feature, is the feature mean value trained based on the building feature vector samples, is the standard deviation trained based on the building feature vector samples, is the scaling factor.

[0047] Next, the specific configuration of the path similarity calculation module 20 will be described in detail. The path similarity calculation module 20 further includes: calibrating the coordinate systems of the multiple initial oblique photogrammetry paths, and extracting path similarity features according to the calibrated coordinate systems, including horizontal speed, vertical climb rate, heading angle change rate, and path curvature; performing DTW similarity calculation on the multiple initial oblique photogrammetry paths according to the horizontal speed, vertical climb rate, heading angle change rate, and path curvature, and outputting a similarity matrix, where the similarity matrix includes each path and a set of path similarities corresponding to each path.

[0048] Next, the specific configuration of the optimized oblique photogrammetry path output module 40 will be described in detail. The optimized oblique photogrammetry path output module 40 further includes: the drone control terminal re-planning the oblique photography angles of the identified drone group according to the adapted lateral overlap degree, where the identified drone group consists of adjacent drones with path similarity greater than a preset similarity threshold; decomposing the adapted lateral overlap degree according to the drone parameters of the identified drone group to obtain the oblique photography angles of the identified drone group, including a first oblique photography angle and a second oblique photography angle.

[0049] Figure 3 FIG. is a schematic structural diagram of an electronic device provided by an embodiment of the present invention, showing a block diagram of an exemplary electronic device suitable for implementing the embodiments of the present invention. Figure 3 The shown electronic device is only an example and should not impose any limitation on the functions and usage scope of the embodiments of the present invention. The electronic device is presented in the form of a general computing device, and its components may include, but are not limited to, an input device 401, a processor 402, a memory 403, and an output device 404. Among them, the processor 402 may be one or more; the memory 403 may include a computer-readable medium and at least one program product, and the program product has a set (at least one) of program modules, and these program modules are configured to execute the functions of the embodiments of the present application.

[0050] The memory 403 shown in the embodiments of the present invention may adopt any combination of one or more computer-readable media; the computer-readable storage medium may be, but is not limited to, infrared rays, semiconductor systems, devices or components, or any combination of the above, for storing software programs, computer-executable programs, and modules, such as program instructions / modules corresponding to a method for planning drone oblique photogrammetry in the embodiments of the present invention. The processor 402 executes various functional applications and data processing of the computer device by running the software programs, instructions, and modules stored in the memory 403, that is, implementing the above-mentioned method for planning drone oblique photogrammetry.

[0051] The planning system for drone oblique photogrammetry provided by the embodiments of the present invention can execute the planning method for drone oblique photogrammetry provided by any embodiment of the present invention, and has the corresponding functional modules and beneficial effects for executing the method.

[0052] Although the present application makes various references to certain modules in the system according to the embodiments of the present application, however, any number of different modules can be used and run on the user terminal and / or the server. The included individual units and modules are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of mutual distinction and do not limit the protection scope of the present invention.

[0053] The above specific embodiments do not constitute a limitation on the protection scope of the present application. Those skilled in the art should understand that various modifications, combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principle of the present application shall be included within the protection scope of the present application.

Claims

1. A planning method for drone oblique photogrammetry, characterized in that The method includes: Connect to the UAV control terminal to obtain multiple initial oblique photogrammetry paths corresponding to multiple UAVs; Calculate the path similarity of the multiple initial oblique photogrammetry paths and output a path similarity set; Identify the marked UAV group with adjacent path similarity greater than the preset similarity threshold according to the path similarity set, record the building area where the marked UAV group conducts oblique photogrammetry, extract the building feature vector group of the building area, input the building feature vector group into the dynamic overlap degree calculation model for side overlap degree adaptation calculation, and output the adapted side overlap degree; The UAV control terminal re-plans the oblique photography angle of the marked UAV group according to the adapted side overlap degree and outputs an optimized oblique photogrammetry path.

2. The planning method for drone oblique photogrammetry according to claim 1, characterized in that, Extract the building feature vectors of the building area corresponding to each UAV. The method includes: Extract the building feature items of the building area corresponding to each UAV. The building feature items include the building top height, building density, building layer type, and obstacle distribution; Perform quantization processing on the building feature items and output building feature quantization parameters; Configure the weight factors of the building top height, building density, building layer type, and obstacle distribution, and process the building feature quantization parameters according to the weight factors to obtain building feature vectors.

3. The planning method for drone oblique photogrammetry according to claim 2, characterized in that, Input the building feature vector group into the dynamic overlap degree calculation model for side overlap degree adaptation calculation and output the adapted side overlap degree. The method includes: Collect the building feature vector group samples for training; Set the constraint conditions, where the constraint conditions include the maximum and minimum values of the side overlap degree, and the preset overlap degree difference, and the preset overlap degree difference is used to limit the excessive overlap degree difference between adjacent building areas; Construct an AdamW optimizer. Under the constraint conditions, use the AdamW optimizer to train the building feature vector samples and the labels representing the side overlap degree to obtain a side overlap degree adaptation calculation model, and use the side overlap degree adaptation calculation model to calculate the building feature vectors.

4. The planning method for drone oblique photogrammetry according to claim 3, wherein Before inputting the building feature vectors into the dynamic overlap degree calculation model for side overlap degree adaptation calculation, the method further includes: Perform class balance identification on each building feature item in the building feature vectors to obtain the marked building feature items, and the marked building feature items are unbalanced small sample feature items; Perform SMOTE oversampling compensation on the marked building feature items and output the compensated building feature vectors.

5. The planning method for drone oblique photogrammetry according to claim 3, wherein The calculation formula of the side overlap degree adaptation calculation model is as follows: ； ； Among them, is used to identify the adjusted adapted lateral overlap degree of the UAV group, is used to identify the initial overlap degree of the UAV group, is the number of the building feature vector groups, is the contribution ratio of the d-th feature to the adjustment of the lateral overlap degree, is the standardized adjustment value of the d-th feature, is the actual value of the feature, is the feature mean value trained based on the building feature vector samples, is the standard deviation trained based on the building feature vector samples, is the scaling factor.

6. The planning method for oblique photogrammetry of an unmanned aerial vehicle according to claim 1, wherein, Calculate the path similarity of the multiple initial oblique photogrammetry paths and output a path similarity set. The method includes: Calibrate the coordinate systems of the multiple initial oblique photogrammetry paths, and extract path similarity features according to the calibrated coordinate systems, including horizontal speed, vertical climb rate, heading angle change rate, and path curvature; Calculate the DTW similarity of the multiple initial oblique photogrammetry paths according to the horizontal speed, vertical climb rate, heading angle change rate, and path curvature, and output a similarity matrix, where the similarity matrix includes each path and a set of path similarities corresponding to each path.

7. The planning method for drone oblique photogrammetry according to claim 1, characterized in that, The UAV control terminal re-plans the oblique photography angles of the identified UAV group according to the adapted side overlap degree, where the identified UAV group consists of adjacent UAVs with path similarities greater than a preset similarity threshold; Decompose the adapted side overlap degree according to the UAV parameters of the identified UAV group to obtain the oblique photography angles of the identified UAV group, including the first oblique photography angle and the second oblique photography angle.

8. A planning system for drone oblique photogrammetry, characterized in that, The system is used to implement a planning method for UAV oblique photogrammetry according to any one of claims 1 to 7, and the system includes: An initial oblique photogrammetry path acquisition module, which is used to connect to the UAV control terminal and acquire multiple initial oblique photogrammetry paths corresponding to multiple UAVs; A path similarity calculation module, which is used to calculate the path similarity of the multiple initial oblique photogrammetry paths and output a set of path similarities; A side overlap degree adaptation calculation module, which is used to identify an identified UAV group with adjacent path similarities greater than a preset similarity threshold according to the set of path similarities, record the building area where the identified UAV group conducts oblique photogrammetry, extract the building feature vector group of the building area, input the building feature vector group into a dynamic overlap degree calculation model for side overlap degree adaptation calculation, and output the adapted side overlap degree; An optimized oblique photogrammetry path output module, which is used for the UAV control terminal to re-plan the oblique photography angles of the identified UAV group according to the adapted side overlap degree and output an optimized oblique photogrammetry path.

9. An electronic device, characterized in that, The electronic device includes: A memory, which is used to store executable instructions; A processor, which is used to implement a planning method for UAV oblique photogrammetry according to any one of claims 1 to 7 when executing the executable instructions stored in the memory.

Images