A method, system and electronic device for planning unmanned aerial vehicle oblique photogrammetry

Abstract

This invention discloses a planning method, system, and electronic device for UAV oblique photogrammetry, relating to the field of UAV oblique photogrammetry technology. The method includes: connecting to a UAV control terminal to acquire multiple initial oblique photogrammetry paths; calculating the path similarity of the multiple initial oblique photogrammetry paths and outputting a path similarity set; recording and identifying the building areas to be obliquely photogramrated by the UAV group, extracting building feature vector groups, performing lateral overlap adaptation calculations, and outputting the adapted lateral overlap; and replanning the oblique photogrammetry angles of the identified UAV group, outputting an optimized oblique photogrammetry path. This invention solves the technical problems in existing technologies where multiple UAV oblique photogrammetry paths lack reasonable planning and lateral overlap cannot be adaptively adjusted according to building area characteristics, leading to insufficient accuracy and efficiency of measurement data. It achieves the technical effect of improving the efficiency and accuracy of UAV oblique photogrammetry.

Description

Technical Field

[0001] This application relates to the technical field of UAV oblique photogrammetry, specifically to a planning method, system, and electronic equipment for UAV oblique photogrammetry. Background Technology

[0002] With the rapid development of urban construction and the ever-increasing demands for the accuracy of geographic information data, traditional surveying methods are no longer sufficient to meet the diverse and high-precision measurement needs. Unmanned aerial vehicle (UAV) oblique photogrammetry, by carrying multiple sensors on the same flight platform and simultaneously acquiring image data from multiple angles, can accurately reflect the actual situation of ground features, providing a rich data source for urban modeling and geographic information system (GIS) updates. However, in practical applications, when multiple UAVs are used simultaneously for oblique photogrammetry, the initial oblique photogrammetry path planning of different UAVs often lacks systematicity and coordination. If the paths of different UAVs are highly similar, it may lead to redundant shooting in some areas and insufficient coverage in others, wasting flight resources and making it difficult to guarantee the integrity and accuracy of the measurement data. Furthermore, due to the varying building characteristics in different building areas, traditional fixed lateral overlap settings cannot be adaptively adjusted according to the actual situation, making it difficult to achieve efficient and accurate oblique photogrammetry; thus affecting the overall measurement efficiency and quality.

[0003] Therefore, current technologies suffer from technical problems such as a lack of reasonable planning for multi-UAV oblique photogrammetry paths and the inability to adaptively adjust lateral overlap based on the characteristics of the building area, resulting in insufficient accuracy and efficiency of measurement data. Summary of the Invention

[0004] This application provides a planning method, system, and electronic device for UAV oblique photogrammetry, which solves the technical problems in the prior art where the lack of reasonable planning of multi-UAV oblique photogrammetry paths and the inability to adaptively adjust the lateral overlap according to the characteristics of the building area lead to insufficient accuracy and efficiency of measurement data. It achieves the technical effect of improving the efficiency and accuracy of UAV oblique photogrammetry.

[0005] This application provides a planning method for UAV oblique photogrammetry. The method includes: deploying attached sensor groups in storage containers within a storage space to obtain a deployed attached sensor array, wherein the storage space is a semi-enclosed space; identifying a set of environmental monitoring points in the storage space, and deploying an environmental sensor group at each environmental monitoring point to obtain a deployed environmental sensor array; collecting monitoring data from the attached sensor array within a preset monitoring window to obtain an attached sensor monitoring data group sequence array; traversing the attached sensor monitoring data group sequence array to perform intra-group fusion iterative analysis to determine a container state factor array; within the preset monitoring window, performing array interaction hazard boundary identification on the environmental sensor array to determine environmental hazard areas and environmental hazard state factors; and performing storage environmental pollution early warning analysis based on the container state factor array, environmental hazard areas, and environmental hazard state factors to obtain first storage environmental pollution early warning information.

[0006] This application also provides a planning system for UAV oblique photogrammetry, comprising: an initial oblique photogrammetry path acquisition module, used to connect to a UAV control terminal and acquire multiple initial oblique photogrammetry paths corresponding to multiple UAVs; a path similarity calculation module, used to calculate the path similarity of the multiple initial oblique photogrammetry paths and output a path similarity set; a lateral overlap adaptation calculation module, used to identify a group of UAVs whose adjacent path similarity is greater than a preset similarity threshold based on the path similarity set, record the building areas where the identified UAVs perform oblique photogrammetry, extract the building feature vector group of the building areas, input the building feature vector group into a dynamic overlap calculation model for lateral overlap adaptation calculation, and output the adapted lateral overlap; and an optimized oblique photogrammetry path output module, used by the UAV control terminal to replan the oblique photogrammetry angle of the identified UAVs according to the adapted lateral overlap and output the optimized oblique photogrammetry path.

[0007] This application also provides an electronic device, including: a memory for storing executable instructions; and a processor for implementing a planning method for UAV oblique photogrammetry when executing the executable instructions stored in the memory.

[0008] This application proposes a planning method, system, and electronic equipment for UAV oblique photogrammetry. The system connects to a UAV control terminal to acquire multiple initial oblique photogrammetry paths. Path similarity calculations are performed on these paths, outputting a path similarity set. The system records and identifies the building areas where the UAV group will perform oblique photogrammetry, extracts building feature vectors, performs lateral overlap adaptation calculations, and outputs the adapted lateral overlap. Finally, the system re-plans the oblique photogrammetry angles of the identified UAV group, outputting an optimized oblique photogrammetry path. This addresses the technical problems in existing technologies where multiple UAV oblique photogrammetry paths lack reasonable planning and lateral overlap cannot be adaptively adjusted according to building area characteristics, leading to insufficient accuracy and efficiency of measurement data. This achieves the technical effect of improving the efficiency and accuracy of UAV oblique photogrammetry. Attached Figure Description

[0009] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the accompanying drawings of the embodiments of this disclosure will be briefly described below. Flowcharts are used in this application to illustrate the operations performed by the system according to the embodiments of this application. It should be understood that the preceding or following operations are not necessarily performed precisely in sequence. Instead, various steps can be processed in reverse order or simultaneously as needed. Furthermore, other operations can be added to these processes, or one or more steps can be removed from these processes.

[0010] Figure 1 This is a schematic flowchart of a planning method for UAV oblique photogrammetry provided in an embodiment of this application.

[0011] Figure 2 This is a schematic diagram of a planning system for UAV oblique photogrammetry provided in an embodiment of this application.

[0012] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0013] Explanation of reference numerals in the attached figures: Initial oblique photogrammetry path acquisition module 10, path similarity calculation module 20, lateral overlap adaptation calculation module 30, optimized oblique photogrammetry path output module 40, input device 401, processor 402, memory 403, output device 404. Detailed Implementation

[0014] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, specific embodiments of this application are given below.

[0015] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description of this application will be provided in conjunction with the accompanying drawings. The described embodiments should not be considered as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0016] In the following description, references to "some embodiments" describe a subset of all possible embodiments. However, it is understood that "some embodiments" can be the same or different subsets of all possible embodiments and can be combined with each other without conflict. The terms "first" and "second" are used merely to distinguish similar objects and do not represent a specific ordering of objects. The terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or server that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or modules not explicitly listed or inherent to these processes, methods, products, or devices. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only.

[0017] This application provides a planning method for UAV oblique photogrammetry, such as... Figure 1 As shown, the method includes:

[0018] Step S100: Connect the UAV control terminal and obtain multiple initial oblique photogrammetry paths corresponding to multiple UAVs.

[0019] Preferably, through specific communication protocols and interfaces, hardware devices (such as wireless communication modules, data cables, etc.) or software programs (such as dedicated UAV control software) are used to establish a connection between the equipment used to plan and control the flight of the UAVs and the control terminals of each UAV, so as to realize remote control and data interaction of the UAVs and obtain the initial oblique photogrammetry path. That is, the control terminal sends instructions to each UAV, requesting it to return its own preset or existing oblique photogrammetry path information. Usually, before the mission begins, based on factors such as the scope of the measurement area, topography, building distribution, as well as the accuracy requirements of photogrammetry and the performance parameters of the UAVs, a professional path planning algorithm is used to specify the path. Each UAV has its own independent initial path to ensure that the target area can be photographed from different angles when performing oblique photogrammetry tasks and obtain rich image data.

[0020] Step S200: Calculate the path similarity of the multiple initial oblique photogrammetry paths and output a path similarity set.

[0021] Preferably, path similarity calculation is performed on multiple initial oblique photogrammetry paths. Specifically, each initial oblique photogrammetry path is considered as a trajectory composed of multiple coordinate points, recording the different positions of the UAV during flight. These path coordinate points are then extracted, and information representing path characteristics, such as path length, direction, curvature changes, and coordinates of key turning points, is further extracted. The extracted features can reflect the shape and trend of the path to a certain extent. Then, a similarity calculation method is selected, such as Euclidean distance, cosine similarity, and dynamic time warping (DTW). For example, Euclidean distance calculates the sum of the distances between corresponding coordinate points on two paths; the smaller the distance, the more similar the two paths are. Cosine similarity focuses on measuring the consistency of the directions of two paths, judging similarity by calculating the cosine value of the angle between the path vectors; the closer the value is to 1, the more similar the path directions are. All initial oblique photogrammetry paths are paired, and then the selected similarity calculation method is used to calculate the similarity between each pair of paths, resulting in a set containing the similarity of all path pairs, i.e., the output similarity set, which comprehensively reflects the similarity relationship between multiple initial oblique photogrammetry paths.

[0022] Furthermore, step S200 also includes step S210, calibrating the coordinate system of the plurality of initial oblique photogrammetry paths, and extracting path similarity features based on the calibrated coordinate system, including horizontal velocity, vertical climb rate, heading angle change rate, and path curvature; step S220, performing DTW similarity calculation on the plurality of initial oblique photogrammetry paths based on the horizontal velocity, vertical climb rate, heading angle change rate, and path curvature, and outputting a similarity matrix, wherein the similarity matrix includes each path and the path similarity set corresponding to each path.

[0023] Preferably, the coordinate systems of multiple initial oblique photogrammetry paths are calibrated to enable accurate comparison. This involves unifying these initial oblique photogrammetry paths to the same coordinate system. Through coordinate transformation, the coordinates of all points on the path are converted to this coordinate system. Then, path similarity features are extracted based on the calibrated coordinate system, including horizontal velocity, vertical climb rate, heading angle change rate, and path curvature. Horizontal velocity refers to the speed at which the UAV moves horizontally, reflecting its speed on the horizontal plane. It is used to determine the horizontal motion pattern of the path and is determined by calculating the ratio of the displacement of adjacent coordinate points in the horizontal direction (usually the xy plane) to the time interval. The vertical climb rate represents the rate of ascent or descent of the UAV in the vertical direction. Speed ​​reflects the change in vertical altitude of the path and is used to analyze the flight patterns of UAVs under different terrains or mission requirements. It is determined by calculating the ratio of the displacement of adjacent coordinate points in the vertical direction (usually the z-axis) to the time interval. Heading angle is the angle between the UAV's flight direction and a reference direction (such as true north). The rate of change of heading angle reflects how quickly the UAV's flight direction changes and can help determine the turning of the path. It is used to identify the curvature and direction change of the path and is obtained by calculating the ratio of the difference in heading angle between adjacent moments to the time interval. Path curvature describes the curvature of the path. The greater the curvature, the more curved the path. If the curvature is zero, the path is a straight line. Path curvature is obtained through mathematical methods, such as calculating the rate of change of the tangent direction at a point on the path.

[0024] Preferably, DTW (Dynamic Time Warping) similarity calculation is performed on multiple initial oblique photogrammetry paths based on horizontal velocity, vertical climb rate, yaw rate of change, and path curvature. DTW is used to calculate the similarity between two time series, and is particularly suitable for handling sequences of different lengths and inconsistent time scales. Specifically, in practice, the flight paths of different UAVs may have different lengths of time series with the same characteristics and the time points may not correspond completely due to factors such as flight speed and attitude adjustments. Therefore, the horizontal velocity, vertical climb rate, yaw rate of change, and path curvature of each path are considered as a multidimensional... For time series analysis, the DTW algorithm finds the optimal matching relationship between two sequences by elastically stretching and compressing the time axis, thereby accurately calculating their similarity. Specifically, for any two paths, DTW calculations are performed on their horizontal velocity sequence, vertical climb rate sequence, heading angle change rate sequence, and path curvature sequence, respectively, obtaining four similarity values. These values ​​are then weighted and summed to obtain the comprehensive similarity between the two paths. Similarly, multiple path similarities are calculated. Assuming there are m initial paths, the similarity matrix is ​​an m×m matrix S, where the element in the i-th row and j-th column represents the comprehensive similarity between paths i and j. =1 (because a path is completely similar to itself), and = (Similarity exhibits symmetry), thus visually demonstrating the degree of similarity between any two paths. The similarity matrix includes each path and its corresponding set of path similarities. By analyzing the elements in the matrix, highly similar path pairs can be identified. Assume there are five paths... , , , , The similarity matrix obtained by DTW is shown in Table 1:

[0025] Table 1. Example data of DTW similarity matrix for oblique photogrammetry paths

[0026] <![CDATA[P1]]> <![CDATA[P2]]> <![CDATA[P3]]> <![CDATA[P4]]> <![CDATA[P5]]> <![CDATA[P1]]> 1.0 0.7 0.3 0.5 0.1 <![CDATA[P2]]> 0.7 1.0 0.4 0.6 0.2 <![CDATA[P3]]> 0.3 0.4 1.0 0.2 0.3 <![CDATA[P4]]> 0.5 0.6 0.2 1.0 0.3 <![CDATA[P5]]> 0.1 0.2 0.6 0.3 1.0

[0027] From the matrix, we can see that and The similarity score is 0.7, indicating they are quite similar. and The similarity is 0.1, indicating a significant difference.

[0028] Step S300: Identify the marked UAV groups whose adjacent path similarity is greater than a preset similarity threshold according to the path similarity set, record the building areas of the marked UAV groups that are measured by oblique photogrammetry, extract the building feature vector group of the building area, input the building feature vector group into the dynamic overlap calculation model to perform lateral overlap adaptation calculation, and output the adapted lateral overlap.

[0029] Preferably, the method involves identifying drone groups whose adjacent paths have a similarity greater than a preset similarity threshold based on a path similarity set. The preset similarity threshold is a standard value pre-set based on historical experience to determine whether the similarity between two paths is sufficiently high. For example, if a more stringent screening of similar paths is desired, the threshold can be set higher; conversely, it can be set lower. Specifically, within the path similarity set, paths whose adjacent paths have a similarity greater than the preset similarity threshold are identified. Drones with these highly similar adjacent paths are grouped together to form an identified drone group. This may involve overlapping flight paths and repeated shooting areas. When performing oblique photogrammetry tasks, the drone group covers certain building areas. These areas are the target areas for image acquisition. The method obtains the specific range and boundary information of these target areas and determines the building areas through the drone's flight trajectory and the geographical location information of the captured images.

[0030] Preferably, the various attributes and characteristics of buildings are quantified to form feature vectors, which more accurately describe the characteristics of buildings within the building area. Building features may include the building's height, area, shape, orientation, roof type, etc. For example, for a rectangular building, its length, width, height, and orientation angle can be used to construct a feature vector. Then, the corresponding feature vectors are extracted for all buildings within the building area, and these feature vectors are combined to form a building feature vector group, which can comprehensively reflect the overall characteristics of buildings within the building area. The lateral overlap adaptation calculation of the building feature vector group is calculated using a dynamic overlap calculation model, and the adapted lateral overlap is output. That is, under the current building characteristics, it can ensure the acquisition of complete building image information and achieve a good image stitching effect.

[0031] Furthermore, step S300 also includes step S310, extracting building feature items for the building area corresponding to each UAV, the building feature items including building top height, building density, building layer type and obstacle distribution; step S320, quantizing the building feature items and outputting building feature quantization parameters; step S330, configuring weight factors for the building top height, building density, building layer type and obstacle distribution, and processing the building feature quantization parameters according to the weight factors to obtain a building feature vector.

[0032] Preferably, each UAV has a specific flight path when performing oblique photogrammetry, and the area it covers corresponds to the building area. For example, the UAV flies along a planned route, and the area of ​​buildings it photographs during its flight is the building area corresponding to that UAV. Then, building features such as building top elevation, building density, building type, and obstacle distribution are extracted. Among them, building top elevation refers to the height of the building's top relative to a certain reference plane (such as sea level or local ground level), which is used to determine the size of the building and potential occlusion during photogrammetry. For example, high-rise buildings may affect the photography of surrounding low-rise buildings; building density reflects the building's height. The density of buildings within a region is measured by calculating the number of buildings per unit area and the ratio of building footprint to the total area. Densely built areas may require higher shooting accuracy and appropriate overlap during photogrammetry. Building type refers to the classification of the number of floors in a building, such as single-story, multi-story, and high-rise. Different types of buildings differ in structure and appearance, and their photogrammetry requirements will also differ. For example, high-rise buildings require shooting from more angles to obtain complete information. Obstacle distribution includes the distribution of other obstacles within the building area besides buildings, such as trees, hills, and large billboards, which may affect the drone's flight path and shooting angle.

[0033] Preferably, each extracted building feature is quantified and converted into a numerical form. Specifically, the building's top elevation is quantified directly using measured height values ​​(e.g., meters); building density is quantified using calculated values, such as the number of buildings per square kilometer or the proportion of building area; building type is quantified using coding methods, such as single-story buildings being coded as 1, multi-story buildings as 2, and high-rise buildings as 3; obstacle distribution is quantified into numerical values ​​based on factors such as the coverage area ratio and quantity of obstacles. For example, if the obstacle coverage area accounts for 20% of the total area, the quantified value can be 0.2. After quantification, the building feature quantification parameters are obtained. Based on actual needs and experience, weighting factors are assigned to building top elevation, building density, building type, and obstacle distribution to reflect the importance of each feature in the comprehensive evaluation of building features. The quantified building feature parameters are multiplied by the corresponding weighting factors, and then the products are added together to obtain a comprehensive value. This comprehensive value is then combined with the values ​​obtained from the same processing of other feature items to form a building feature vector, which comprehensively reflects the comprehensive characteristics of buildings within the building area.

[0034] Furthermore, step S300 also includes step S340, collecting building feature vector group samples for training; step S350, setting constraints, including the maximum and minimum values ​​of lateral overlap, and a preset overlap difference, the preset overlap difference being used to limit the overlap difference between adjacent building areas to be too large; step S360, constructing an AdamW optimizer, under the constraints, using the AdamW optimizer to train the building feature vector samples and labels representing lateral overlap to obtain a lateral overlap adaptation calculation model, and using the lateral overlap adaptation calculation model to calculate the building feature vectors.

[0035] Preferably, a large number of building feature vector samples are collected by conducting actual measurements, data collection and processing in different building areas. Each sample corresponds to a specific building area and contains various feature information of the buildings in that area. After quantization, these feature information are formed into building feature vectors, such as the vectors composed of quantized features like building top height, building density, building floor type and obstacle distribution mentioned above. Then, constraints are set, including the maximum and minimum values ​​of lateral overlap and the preset overlap difference. The maximum value is set to avoid excessive overlap leading to data redundancy, wasting storage resources and processing time. Setting a minimum value ensures sufficient overlap for accurate results during subsequent image stitching and 3D modeling. For example, the maximum lateral overlap might be set to 80%, and the minimum to 30%. Preset overlap difference limits excessive overlap between adjacent building areas. Excessive overlap can affect data consistency across the entire measurement area. Preset overlap difference limits this, ensuring that the overlap between adjacent areas doesn't differ too drastically. For instance, setting the overlap difference between adjacent building areas to no more than 10% ensures more uniform and reasonable image acquisition across the entire measurement area.

[0036] Preferably, an AdamW optimizer is constructed. Based on the constructed constraints, the AdamW optimizer is used to train the building feature vector samples and labels representing lateral overlap to obtain a lateral overlap adaptation calculation model. Specifically, AdamW combines the Adam optimization algorithm with the idea of ​​weight decay (L2 regularization) to adjust the model parameters so that the model's prediction results are as close as possible to the true values. When constructing the AdamW optimizer, some hyperparameters need to be set, such as the learning rate and weight decay coefficient. The learning rate controls the step size of each parameter update, while the weight decay coefficient is used to prevent the model from overfitting. By constraining the model parameters, the parameter values ​​are prevented from becoming too large. For example, Table 2 shows example data of hyperparameter combinations when training the model using the AdamW optimizer:

[0037] Table 2. Hyperparameters of the AdamW optimizer

[0038] Hyperparameter name meaning Range of values Example Learning rate The size of the step size for each parameter update determines the learning speed of the model during training. <![CDATA[Usually between 10 -1 and 10 -6 Common values are such as 0.001, 0.0001]]> For example, setting it to 0.001 means that each time the parameters are updated, the amount of change in the parameters is adjusted by multiplying the calculated gradient by 0.001. Weight decay coefficient Used to implement L2 regularization, it constrains the model's parameters to prevent them from becoming too large, thereby avoiding overfitting. Common values ​​range from 0.0001 to 0.1, such as 0.001. When the weight decay coefficient is set to 0.001, a penalty term proportional to the sum of squared parameters will be added to the loss function, with a penalty strength of 0.001 times. β1 (the exponential decay rate estimated by the first moment) The first-order moment estimate (i.e., momentum) used to calculate the gradient controls the degree of influence of historical gradients on the current update. It typically takes a value close to 1, such as 0.9. <![CDATA[When β1 is set to 0.9, it means that the current gradient update will consider the previous 90% of gradient information (in an exponentially decaying manner). <!-- 6 -->]]> β2 (the exponential decay rate estimated by the second moment) Used to calculate the second-order moment estimate of the gradient, controlling the influence of the historical gradient squares. It typically takes a value close to 1, such as 0.999. <![CDATA[When β2 is set to 0.999, it means that the current update will consider the gradient square information of the previous 99.9% (in an exponentially decaying manner).]]> ε (a small constant used for numerical stability) To prevent division by zero in calculations, it is a very small positive number. <![CDATA[Common values such as 10 -8 > <![CDATA[For example, ε is set to 10 -8 , when calculating operations involving denominators (such as calculating the adaptive learning rate), this small constant is added to ensure the stability of numerical calculations]]>

[0039] Preferably, the collected building feature vector samples are used as input to the model, and the actual lateral overlap corresponding to each sample is used as the label (i.e., the correct result the model should predict). Using the AdamW optimizer, the model is trained under the previously set constraints (maximum and minimum values ​​of lateral overlap and a preset overlap difference). During training, the model predicts based on the input building feature vector samples, obtaining a predicted value for lateral overlap. Then, the error between the predicted value and the label is calculated (usually measured using a loss function). The AdamW optimizer adjusts the model parameters based on this error, gradually reducing the error. Simultaneously, due to the constraints... The existence of this constraint restricts the model's parameter adjustment process to ensure that the predicted lateral overlap is within a reasonable range. After multiple iterations of training, when the model's prediction results meet certain performance requirements on both the training and validation sets (e.g., the loss function converges to a small value, and the prediction error on the validation set is within an acceptable range), the lateral overlap adaptation calculation model is obtained. The new building feature vector is input into the trained lateral overlap adaptation calculation model to calculate the lateral overlap value suitable for the building area. This value is used to guide the shooting parameter settings when the UAV performs oblique photogrammetry in the area, so as to obtain better measurement results and ensure the quality and accuracy of the measurement data.

[0040] Furthermore, step S340 also includes step S341, performing class balance recognition on each building feature item in the building feature vector to obtain an identifying building feature item, wherein the identifying building feature item is an imbalanced small sample feature item; step S342, performing SMOTE oversampling compensation on the identifying building feature item, and outputting the compensated building feature vector.

[0041] Preferably, class balance identification is performed on each building feature item in the building feature vector, that is, identifying the imbalanced small sample feature items and determining them as the identifying building feature items. Then, SMOTE oversampling compensation is applied to the identifying building feature items. SMOTE is an oversampling method used to solve the problem of class imbalance in the dataset. For the identified building feature items (i.e. imbalanced small sample feature items), the SMOTE algorithm generates new samples by interpolating in the feature space of these small sample feature items. Specifically, it finds the nearest neighbor samples of the small sample feature items and then randomly generates new sample points between them, thereby increasing the number of samples of the small sample feature items and making the number of feature items of each class more balanced. After SMOTE oversampling compensation, the compensated building feature vector is output, which can improve the distribution of the data, avoid the poor learning effect of the model on small sample feature items due to data imbalance during the training process, and improve the accuracy and generalization ability of the model.

[0042] Furthermore, step S360 also includes the following formula for calculating the lateral overlap adaptation calculation model:

[0043] ;

[0044] ;

[0045] in, To identify the adjusted adaptive lateral overlap of the drone group, To identify the initial overlap of the drone group, The number of feature vector groups for buildings. Let d be the contribution ratio of the d-th feature to the lateral overlap adjustment. The standardized adjustment value of the d-th feature. For the actual value of the characteristic, The mean value of features trained based on the feature vector samples of the building. The standard deviation is the result of training based on the feature vector samples of the building. , which is the scaling factor. Among them, high-weight features (such as building density) play a dominant role in the overlap adjustment; low-weight features (such as the number of surrounding obstacles) have a smaller impact and may only be significant in specific scenarios.

[0046] In step S400, the UAV control terminal replans the oblique photography angle of the identified UAV group according to the adapted lateral overlap and outputs an optimized oblique photography measurement path.

[0047] Preferably, based on the requirements for adapting lateral overlap, the UAV control terminal resets the oblique photography angle of each UAV in the identification UAV group. Specifically, the oblique photography angle determines the tilt direction and angle of the camera when the UAV takes pictures. Different oblique photography angles will affect the content and range of the captured images. By adjusting the oblique photography angle, the viewing angle of the UAV can be changed, so that the captured images can better meet the requirements for adapting lateral overlap, avoiding redundant shooting or missing important areas. For example, if the requirements for adapting lateral overlap of a certain building area are high, it may be necessary to increase the oblique photography angle of the UAV to increase the overlapping part of the image. Conversely, if the requirements for adapting lateral overlap are low, the oblique photography angle can be appropriately reduced. After replanning the oblique photography angle of each UAV, the UAV control terminal generates a new flight path for each UAV in the identification UAV group based on the new oblique photography angle and other relevant flight parameters (such as flight altitude, speed, etc.), that is, optimizes the oblique photogrammetry path. This enables the UAV to perform oblique photogrammetry of the building area more efficiently and accurately while meeting the requirements for adapting lateral overlap, improving the quality of the measurement data and the efficiency of subsequent processing.

[0048] Furthermore, step S400 also includes step S410, whereby the UAV control terminal re-plans the oblique camera angle of the identified UAV group according to the adaptive lateral overlap, wherein the identified UAV group consists of adjacent UAVs whose path similarity is greater than a preset similarity threshold; step S420, whereby the adaptive lateral overlap is decomposed according to the UAV parameters of the identified UAV group to obtain the oblique camera angle of the identified UAV group, including a first oblique camera angle and a second oblique camera angle.

[0049] Preferably, the UAV control terminal recalculates the oblique camera angle of the identified UAV group based on the adaptive lateral overlap. This recalculation includes a first oblique camera angle and a second oblique camera angle. Specifically, UAV parameters may include the UAV model, sensor characteristics, flight altitude, speed, etc. Different UAV parameters affect their shooting capabilities and image overlap. For example, UAVs flying at higher altitudes may require a larger oblique camera angle to achieve the same lateral overlap. Different sensor resolutions may also have different requirements for the oblique camera angle. Based on these UAV parameters, as well as principles of geometric optics and photogrammetry, the adaptive lateral overlap is translated into specific requirements for the oblique camera angle. This involves considering the interrelationships between various factors. For example, based on flight altitude and camera focal length, the required tilt angle range for the drone under certain lateral overlap requirements is calculated. Then, through further analysis and calculation, this angle range is refined into a first tilt angle and a second tilt angle. The first and second tilt angles may correspond to different shooting directions or different shooting stages. For instance, the first tilt angle might be the initial tilt angle when the drone approaches a building area, used to initially acquire images of the building's side. The second tilt angle might be the angle adjusted when the drone flies above the building area or to a specific position to better capture the building's top and other details, while ensuring appropriate lateral overlap with images captured by adjacent drones. By using two different tilt angles, more comprehensive and accurate image information of the building area can be acquired, and the requirements for adaptable lateral overlap can be met, thereby improving the accuracy and quality of drone photogrammetry.

[0050] In the above text, refer to Figure 1 A planning method for UAV oblique photogrammetry according to an embodiment of the present invention is described in detail. Next, reference will be made to... Figure 2 A planning system for unmanned aerial vehicle (UAV) oblique photogrammetry is described according to an embodiment of the present invention.

[0051] According to an embodiment of the present invention, a planning system for UAV oblique photogrammetry is provided to address the technical problems in the prior art, such as the lack of reasonable planning for multi-UAV oblique photogrammetry paths and the inability to adaptively adjust lateral overlap according to the characteristics of building areas, resulting in insufficient accuracy and efficiency of measurement data. This system achieves the technical effect of improving the efficiency and accuracy of UAV oblique photogrammetry. Figure 2 As shown, a planning system for UAV oblique photogrammetry includes: an initial oblique photogrammetry path acquisition module 10, a path similarity calculation module 20, a lateral overlap adaptation calculation module 30, and an optimized oblique photogrammetry path output module 40.

[0052] The initial oblique photogrammetry path acquisition module 10 is used to connect to the UAV control terminal and acquire multiple initial oblique photogrammetry paths corresponding to multiple UAVs; the path similarity calculation module 20 is used to calculate the path similarity of the multiple initial oblique photogrammetry paths and output a path similarity set; the lateral overlap adaptation calculation module 30 is used to identify the marked UAV group whose adjacent path similarity is greater than a preset similarity threshold according to the path similarity set, record the building area of ​​the marked UAV group for oblique photogrammetry, extract the building feature vector group of the building area, input the building feature vector group into the dynamic overlap calculation model for lateral overlap adaptation calculation, and output the adapted lateral overlap; the optimized oblique photogrammetry path output module 40 is used by the UAV control terminal to replan the oblique photogrammetry angle of the marked UAV group according to the adapted lateral overlap and output the optimized oblique photogrammetry path.

[0053] The following will describe in detail the specific configuration of the lateral overlap adaptation calculation module 30. The lateral overlap adaptation calculation module 30 further includes: extracting building feature items for each UAV's corresponding building area, the building feature items including building top height, building density, building layer type, and obstacle distribution; quantizing the building feature items and outputting building feature quantization parameters; configuring weight factors for the building top height, building density, building layer type, and obstacle distribution, and processing the building feature quantization parameters according to the weight factors to obtain a building feature vector.

[0054] The following will describe in detail the specific configuration of the lateral overlap adaptation calculation module 30. The lateral overlap adaptation calculation module 30 further includes: collecting building feature vector group samples for training; setting constraints, including a maximum and minimum value of lateral overlap, and a preset overlap difference, the preset overlap difference being used to limit excessive overlap differences between adjacent building areas; constructing an AdamW optimizer, and under the constraints, training the building feature vector samples and labels representing lateral overlap using the AdamW optimizer to obtain a lateral overlap adaptation calculation model, and using the lateral overlap adaptation calculation model to calculate the building feature vectors.

[0055] The following will describe in detail the specific configuration of the lateral overlap adaptation calculation module 30. The lateral overlap adaptation calculation module 30 further includes: performing class balance recognition on each building feature item in the building feature vector to obtain an identifying building feature item, wherein the identifying building feature item is an unbalanced small sample feature item; performing SMOTE oversampling compensation on the identifying building feature item, and outputting the compensated building feature vector.

[0056] The specific configuration of the lateral overlap adaptation calculation module 30 will be described in detail below. The lateral overlap adaptation calculation module 30 further includes the following formula for the lateral overlap adaptation calculation model:

[0057] ;

[0058] ;

[0059] in, To identify the adjusted adaptive lateral overlap of the drone group, To identify the initial overlap of the drone group, The number of feature vector groups for buildings. Let d be the contribution ratio of the d-th feature to the lateral overlap adjustment. The standardized adjustment value of the d-th feature. For the actual value of the characteristic, The mean value of features trained based on the feature vector samples of the building. The standard deviation is the result of training based on the feature vector samples of the building. This is the scaling factor.

[0060] The specific configuration of the path similarity calculation module 20 will be described in detail below. The path similarity calculation module 20 further includes: calibrating the coordinate system of the plurality of initial oblique photogrammetry paths; extracting path similarity features based on the calibrated coordinate system, including horizontal velocity, vertical climb rate, heading angle change rate, and path curvature; performing DTW similarity calculation on the plurality of initial oblique photogrammetry paths based on the horizontal velocity, vertical climb rate, heading angle change rate, and path curvature; and outputting a similarity matrix, wherein the similarity matrix includes each path and the path similarity set corresponding to each path.

[0061] The specific configuration of the optimized oblique photogrammetry path output module 40 will be described in detail below. The optimized oblique photogrammetry path output module 40 further includes: the UAV control terminal replanning the oblique photogrammetry angle of the identified UAV group according to the adaptive lateral overlap, wherein the identified UAV group consists of adjacent UAVs with path similarity greater than a preset similarity threshold; and decomposing the adaptive lateral overlap according to the UAV parameters of the identified UAV group to obtain the oblique photogrammetry angle of the identified UAV group, including a first oblique photogrammetry angle and a second oblique photogrammetry angle.

[0062] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention, showing a block diagram of an exemplary electronic device suitable for implementing the embodiments of the present invention. Figure 3 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments of the present invention. This electronic device is in the form of a general-purpose computing device, and its components may include, but are not limited to, an input device 401, a processor 402, a memory 403, and an output device 404. The processor 402 may be one or more; the memory 403 may include a computer-readable medium and at least one program product having a set (at least one) of program modules configured to perform the functions of the embodiments of this application.

[0063] The memory 403 shown in this embodiment of the invention can be any combination of one or more computer-readable media. The computer-readable storage media can be, but is not limited to, infrared, semiconductor systems, devices or components, or any combination thereof, used to store software programs, computer-executable programs and modules, such as the program instructions / modules corresponding to a planning method for UAV oblique photogrammetry in this embodiment of the invention. The processor 402 executes various functional applications and data processing of the computer device by running the software programs, instructions and modules stored in the memory 403, thereby realizing the above-mentioned planning method for UAV oblique photogrammetry.

[0064] The planning system for UAV oblique photogrammetry provided in this embodiment of the invention can execute the planning method for UAV oblique photogrammetry provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0065] Although this application makes various references to certain modules in the system according to the embodiments of this application, any number of different modules can be used and run on user terminals and / or servers. The various units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of each functional unit are only for easy distinction between each other and are not used to limit the scope of protection of this invention.

[0066] The specific embodiments described above do not constitute a limitation on the scope of protection of this application. Those skilled in the art should understand that various modifications, combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A planning method for UAV oblique photogrammetry, characterized in that, The method includes: Connect to the UAV control terminal to obtain multiple initial oblique photogrammetry paths for multiple UAVs; Calculate the path similarity for the multiple initial oblique photogrammetry paths and output a path similarity set; Identify UAV groups whose adjacent path similarity is greater than a preset similarity threshold based on the path similarity set, record the building areas measured by the oblique photogrammetry of the UAV groups, extract the building feature vector groups of the building areas, input the building feature vector groups into the dynamic overlap calculation model to perform lateral overlap adaptation calculation, and output the adapted lateral overlap. The UAV control terminal replans the oblique photography angle of the identified UAV group according to the adaptive lateral overlap, and outputs an optimized oblique photography measurement path. The method involves inputting the building feature vector group into a dynamic overlap calculation model for lateral overlap adaptation calculation and outputting the adapted lateral overlap. Collect sample building feature vector sets for training; Set constraints, including the maximum and minimum values ​​of lateral overlap, and a preset overlap difference, which is used to limit the overlap difference between adjacent building areas to be too large. An AdamW optimizer is constructed. Under the constraints, the AdamW optimizer is used to train the building feature vector samples and the labels representing the lateral overlap to obtain the lateral overlap adaptation calculation model. The lateral overlap adaptation calculation model is then used to calculate the building feature vectors.

2. The planning method for UAV oblique photogrammetry as described in claim 1, characterized in that, The method for extracting the building feature vectors for each drone's corresponding building area includes: Extract building features for each UAV's corresponding building area. These features include building top height, building density, building layer type, and obstacle distribution. The building features are quantized, and the building feature quantization parameters are output. Configure weighting factors for the building's top elevation, building density, building floor type, and obstacle distribution, and process the building feature quantization parameters according to the weighting factors to obtain the building feature vector.

3. The planning method for UAV oblique photogrammetry as described in claim 1, characterized in that, Before inputting the building feature vector into the dynamic overlap calculation model for lateral overlap adaptation calculation, the method further includes: The class balance identification is performed on each building feature item in the building feature vector to obtain the identifying building feature item, which is an imbalanced small sample feature item; SMOTE oversampling compensation is applied to the identified building feature items, and the compensated building feature vector is output.

4. The planning method for UAV oblique photogrammetry as described in claim 1, characterized in that, The calculation formula for the lateral overlap adaptation calculation model is as follows: ； ； in, To identify the adjusted adaptive lateral overlap of the drone group, To identify the initial overlap of the drone group, The number of feature vector groups for buildings. Let d be the contribution ratio of the d-th feature to the lateral overlap adjustment. The standardized adjustment value of the d-th feature. For the actual value of the characteristic, The mean value of features trained based on the feature vector samples of the building. The standard deviation is the result of training based on the feature vector samples of the building. This is the scaling factor.

5. The planning method for UAV oblique photogrammetry as described in claim 1, characterized in that, The method includes calculating path similarity for the multiple initial oblique photogrammetry paths and outputting a path similarity set. The coordinate systems of the multiple initial oblique photogrammetry paths are calibrated, and path similarity features are extracted based on the calibrated coordinate systems, including horizontal velocity, vertical climb rate, heading angle change rate, and path curvature. Based on the horizontal velocity, vertical climb rate, heading angle change rate, and path curvature, DTW similarity calculation is performed on the multiple initial oblique photogrammetry paths, and a similarity matrix is ​​output. The similarity matrix includes each path and the path similarity set corresponding to each path.

6. The planning method for UAV oblique photogrammetry as described in claim 1, characterized in that, The UAV control terminal replans the tilt camera angle of the identified UAV group according to the adaptive lateral overlap, wherein the identified UAV group consists of adjacent UAVs with path similarity greater than a preset similarity threshold; The adaptive lateral overlap is decomposed according to the drone parameters of the identified drone group to obtain the oblique imaging angle of the identified drone group, including a first oblique imaging angle and a second oblique imaging angle.

7. A planning system for unmanned aerial vehicle (UAV) oblique photogrammetry, characterized in that, The system is used to implement the planning method for UAV oblique photogrammetry according to any one of claims 1 to 6, and the system comprises: The initial oblique photogrammetry path acquisition module is used to connect to the UAV control terminal and acquire multiple initial oblique photogrammetry paths corresponding to multiple UAVs. The path similarity calculation module is used to calculate the path similarity of the multiple initial oblique photogrammetry paths and output a path similarity set. The lateral overlap adaptation calculation module is used to identify the marked UAV group whose adjacent path similarity is greater than a preset similarity threshold according to the path similarity set, record the building area of ​​the marked UAV group by oblique photogrammetry, extract the building feature vector group of the building area, input the building feature vector group into the dynamic overlap calculation model to perform lateral overlap adaptation calculation, and output the adapted lateral overlap. An optimized oblique photogrammetry path output module is used by the UAV control terminal to replan the oblique photogrammetry angle of the identified UAV group according to the adapted lateral overlap, and output an optimized oblique photogrammetry path.

8. An electronic device, characterized in that, The electronic device includes: Memory, used to store executable instructions; The processor, when executing executable instructions stored in the memory, implements the planning method for UAV oblique photogrammetry as described in any one of claims 1 to 6.

Images