Three-dimensional map surveying and mapping method and system based on unmanned aerial vehicle

Abstract

The invention discloses a three-dimensional map surveying and mapping method and system based on an unmanned aerial vehicle, and relates to the technical field of map surveying and mapping. Based on a depth estimation result Dep formed by an image sequence Img and pose data Pos, a ground object normal field Nor for describing a real surface orientation can be accurately constructed in an area with a complex ground object surface structure and multiple overlapped slopes; and the situation of insufficient stereo parallax caused by incapability of identifying the orientation change of the ground object in a traditional fixed track mode is avoided. And then, a solid angle evaluation result Ang is obtained by calculating a space included angle between the observation direction vector Dir and a ground object normal field Nor, a solid angle bad area set Bad is determined according to an area lower than a preset solid angle threshold value, and the space position with insufficient solid angle caused by too high building elevation, abrupt terrain slope, gully shielding and the like can be identified in actual flight. And the three-dimensional reconstruction missing rate of areas such as inclined facades, abrupt slope sections and road canyons can be obviously reduced.

Description

Technical Field

[0001] This invention relates to the field of map surveying technology, specifically to a method and system for three-dimensional map surveying based on unmanned aerial vehicles (UAVs). Background Technology

[0002] Driven by the rapid digitalization of geographic information, drones have become an important data source for regional 3D reality reconstruction, and various stereo mapping technologies based on aerial imagery are maturing and becoming more widespread. Within this technological system, how to organize multi-angle, multi-viewpoint mapping images to meet the conditions for stereo geometric calculation is one of the key factors affecting the quality of 3D maps. Therefore, it is necessary to focus on the drone's flight path planning methods, viewpoint change patterns, and the stability of image geometric relationships to ensure that the drone can continuously provide basic imagery data suitable for high-precision stereo reconstruction during flight.

[0003] In existing UAV mapping workflows, flight paths are typically planned based on fixed directions and overlap rates, and do not automatically adjust attitude or yaw direction according to changes in ground features during flight. However, actual ground structures exhibit significant spatial diversity. For example, building facades may be arranged in rows along a certain direction, and slope angles may have abrupt changes. This means that when UAVs fly along fixed paths, the imagery's imaging perspective on these structures often fails to maintain a suitable solid angle relationship. Insufficient solid angle can lead to problems such as sparse features, unstable depth, or insufficient parallax in subsequent stereo matching processes, resulting in the inability to form reliable 3D point clouds in certain areas. Therefore, a technical solution is needed that can identify areas with unfavorable solid angles during flight and automatically perform minor yaw adjustments to improve the stereo geometric quality of the imagery and avoid the risk of reconstruction failure. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides a method and system for stereo mapping based on unmanned aerial vehicles (UAVs), which solves the problems mentioned in the background section.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a method for stereoscopic map mapping based on unmanned aerial vehicles (UAVs), comprising the following steps: S1. Control the UAV to fly along the preset trajectory, collect aerial images, and form flight trajectory data Tra, multi-view image sequence Img, and image shooting pose data Pos. S2. Based on the image sequence Img and pose data Pos, feature points are extracted to form a feature point set Fea, and a depth estimation result Dep is generated through multi-view depth estimation. Based on the depth estimation result Dep, a ground normal field Nor describing the orientation of the ground surface is constructed. S3. Based on flight track data Tra, pose data Pos, and ground object normal field Nor, calculate the solid angle evaluation quantity of the local area to form the solid angle evaluation result Ang, and obtain the set of solid angle bad areas Bad by judging the result against the preset threshold. S4. Generate an automatic yaw command Cmd to adjust the UAV's heading based on the angle deviation in the Bad set of solid angle defective areas, update the flight track data Tra to form updated track data NewTra, and send it to the UAV for application.

[0006] Preferably, S1 includes S11; S11. During the flight of the UAV according to the preset trajectory, the flight status information is collected in real time through the flight control and navigation module of the UAV. The flight status information includes longitude, latitude and altitude obtained by the global navigation satellite system receiving module, as well as pitch angle, roll angle and yaw angle obtained by the inertial measurement unit. By using a unified time reference of the UAV's flight control system, longitude, latitude, altitude, pitch angle, roll angle, and yaw angle are synchronously recorded in chronological order, and continuous data processing is performed according to a fixed sampling period to compensate for attitude jumps caused by sampling period differences, thereby generating flight track data Tra to describe the UAV's flight path.

[0007] Preferably, S1 further includes S12; S12. While generating flight track data Tra, based on the heading overlap and lateral overlap of the mapping task, the UAV imaging device is controlled to collect aerial images at a fixed shooting frequency to form a multi-view image sequence covering the mapping area, resulting in image sequence Img. For each image in the image sequence Img, based on the unified time reference provided by the flight control system, the shooting time of each image in the image sequence Img is matched with the longitude, latitude, altitude, pitch angle, roll angle and yaw angle of the corresponding time in the flight track data Tra, so that the shooting position and shooting attitude of each image are obtained. In this method, longitude, latitude, and altitude are used as the image capture position, and pitch angle, roll angle, and yaw angle are used as the image capture attitude, forming pose data Pos used to describe the image imaging extrinsic parameters.

[0008] Preferably, S2 includes S21; S21. Based on the acquired image sequence Img and pose data Pos, feature point detection processing is performed on each image. Image features with texture changes are identified through feature extraction algorithm, and corresponding feature points are determined among multiple images using feature matching algorithm to form a feature point set Fea for depth estimation. After obtaining the feature point set Fea, depth estimation is performed based on the spatial projection relationship of the feature points in the multi-view image, combined with the image extrinsic parameters provided by the pose data Pos, through multi-view geometric constraints. The depth estimation uses a multi-view depth calculation method based on the principle of triangulation. By solving the projection difference of feature points under different views, the spatial depth value of the feature points is obtained, forming a depth estimation result covering the surveyed area, which is marked as the depth estimation result Dep, providing a spatial geometric basis for subsequent calculation of the surface orientation of ground features.

[0009] Preferably, S2 further includes S22; S22. Based on the obtained depth estimation result Dep, for each spatial depth value in the depth estimation result Dep, determine several spatial depth values ​​that are spatially adjacent to the spatial depth value as a depth neighborhood set. By comparing the change in height difference and the trend of depth gradient between each spatial depth value and the corresponding spatial depth value in the depth neighborhood set, deduce the surface orientation of the ground feature at the location corresponding to the spatial depth value, and obtain the direction vector used to represent the surface orientation at the corresponding location. After generating all direction vectors from the depth estimation result Dep, the direction vectors corresponding to adjacent spatial depth values ​​are weighted and fused to form the ground feature normal field Nor.

[0010] Preferably, S3 includes S31; S31. Based on the obtained flight track data Tra and pose data Pos, as well as the ground normal field Nor, the survey area is spatially divided, and each local area after spatial division is matched to determine the corresponding shooting position and shooting posture in each local area. Based on the pitch, roll, and yaw angles recorded in the shooting posture, the observation direction of each local area is calculated to form an observation direction vector Dir representing the observation direction of each local area. Based on the surface orientation of the ground object recorded in the ground object normal field Nor, the spatial angle between the observation direction vector Dir and the surface orientation of the ground object is calculated, and the spatial angle is used as the solid angle evaluation quantity of the local area. By summarizing the solid angle evaluation values ​​of all local regions, an evaluation result of the solid angle quality distribution is formed and marked as the solid angle evaluation result Ang.

[0011] Preferably, S3 further includes S32; S32. Based on the obtained solid angle evaluation result Ang, perform threshold determination on the solid angle evaluation quantity corresponding to each local region in the solid angle evaluation result Ang. The threshold is determined as follows: When the solid angle evaluation value is less than the preset threshold, the local area is marked as a solid angle insufficient area; When the solid angle evaluation value is greater than or equal to the preset threshold, the local area is marked as the solid angle satisfaction area; The solid angle evaluation values ​​of all local areas are evaluated one by one to determine the threshold. The spatial locations of all areas marked as having insufficient solid angle are summarized to form a set of unfavorable solid angle areas, resulting in the set of bad solid angle areas.

[0012] Preferably, S4 includes S41; S41. Based on the obtained set of poor solid angle regions Bad and the solid angle evaluation result Ang, combined with the observation direction vector Dir and the ground object normal field Nor, calculate the solid angle deviation between the actual observation direction and the ground object surface direction for each local area in the set of poor solid angle regions Bad. Based on the difference between the solid angle deviation and the preset solid angle target range, determine the yaw rotation angle required to restore the solid angle to the target range, and form the yaw adjustment amount Yaw corresponding to each local area; After determining the yaw adjustment amount (Yaw) for each local area, the yaw adjustment amount (Yaw) is associated with the corresponding flight position and flight time in the flight track data (Tra) so that the track segment of the solid angle poor area set Bad meets the yaw adjustment requirements. By arranging and organizing the yaw adjustment values ​​(Yaw) for all track segments, an automatic yaw command (Cmd) is formed to control the UAV to adjust its heading during flight, providing a clear basis for yaw control for subsequent track updates.

[0013] Preferably, S4 further includes S42; S42. Based on the generated automatic yaw command Cmd, calculate the heading adjustment for the track segment in the flight track data Tra that corresponds to the automatic yaw command Cmd. For each track segment that needs to be yawed, adjust the heading angle based on the original flight direction according to the yaw adjustment amount Yaw corresponding to the track segment to obtain a new flight direction. Then, based on the new flight direction and the UAV's predetermined flight speed, the flight path of the track segment is recalculated, causing the position sequence of the track segment in space to change; After performing heading adjustment calculations on all track segments that need adjustment, updated track data NewTra, which includes the spatial trajectory after heading correction, is generated. After obtaining the updated track data NewTra, the UAV is re-controlled to fly along the updated track to re-observe the area corresponding to the Bad solid angle defect set and acquire image data from the new observation direction.

[0014] A UAV-based stereo mapping system includes a UAV aerial photography data acquisition module, a data feature extraction and calculation module, a stereo defect area determination module, and a UAV flight path update module. The UAV aerial photography data acquisition module controls the UAV to fly along a preset trajectory, collect aerial images, and form flight trajectory data Tra, multi-view image sequence Img, and image shooting pose data Pos. The data feature extraction and estimation module extracts feature points based on the image sequence Img and pose data Pos to form a feature point set Fea, and generates a depth estimation result Dep through multi-view depth estimation. Based on the depth estimation result Dep, a ground normal field Nor describing the orientation of the ground surface is constructed. The solid angle defect determination module calculates the solid angle evaluation quantity of the local area based on the flight track data Tra, pose data Pos, and ground object normal field Nor to form a solid angle evaluation result Ang, and obtains the solid angle defect area set Bad by judging the result against the preset threshold. The UAV route update module generates an automatic yaw command Cmd to adjust the UAV's heading based on the angle deviation in the Bad set of solid angle defects, updates the flight track data Tra to form updated track data NewTra, and sends it to the UAV for application.

[0015] This invention provides a method and system for stereo mapping based on unmanned aerial vehicles (UAVs), which has the following advantages: (1) The depth estimation result Dep, formed by the image sequence Img and pose data Pos, can accurately construct the ground normal field Nor, which describes the true surface orientation, in areas with complex surface structures and overlapping slopes. This avoids the situation where the traditional fixed-track method cannot identify changes in ground orientation, resulting in insufficient stereo parallax. Subsequently, the stereo angle evaluation result Ang is obtained by calculating the spatial angle between the observation direction vector Dir and the ground normal field Nor. Based on the areas below the preset stereo angle threshold, a set of bad stereo angle areas is established. This can identify the spatial locations where the stereo angle is insufficient due to reasons such as excessively high building facades, steep terrain slopes, and valley obstruction during actual flight. It can significantly reduce the 3D reconstruction loss rate in areas such as inclined facades, steep slopes, and road canyons. It can also improve the stereo parallax quality of images without additional hardware, thus significantly improving the stereo mapping accuracy and completeness of UAVs in complex terrain environments.

[0016] (2) By jointly processing the image sequence Img and pose data Pos, this step can continuously acquire spatially consistent feature point matching results under complex terrain structure conditions, thereby forming a stable and reliable feature point set Fea. On this basis, by using the projection differences of multi-view images to calculate the depth estimation result Dep, the depth estimation accuracy can be effectively improved in areas with sparse texture or repetitive texture, so that the depth data can still maintain continuity under long-distance parallax or lateral view. The resulting terrain normal field Nor can accurately describe the orientation differences of these slope changes, providing a continuous and reliable geometric structure basis for subsequent solid angle quality determination, thereby avoiding stereo reconstruction errors caused by surface orientation inconsistencies in areas with drastic terrain undulations.

[0017] (3) By calculating the solid angle evaluation result Ang, a quantifiable spatial diagnosis of the solid geometry of the entire survey area can be performed before the UAV enters the 3D reconstruction stage, so that the observation direction quality of each local area can be clearly expressed numerically. By calculating the spatial angle formed by the observation direction vector Dir and the ground normal field Nor, this step can identify spatial areas that are difficult to form stable parallax due to improper observation direction during the flight of the UAV, and summarize them into the solid angle poor area set Bad. In actual aerial survey scenarios, the location of insufficient solid angle can be located in real time during the image shooting stage, so that these areas are accurately classified into the solid angle poor area set Bad. This predictive ability enables the UAV to avoid reconstruction risks in advance in subsequent stages, avoiding large-area structural voids, facade collapse or depth error accumulation caused by small observation angles of facades or slopes in traditional methods, providing a clear target area for subsequent automatic yaw adjustment, and significantly improving the success rate of 3D reconstruction and the integrity of the overall real scene model. Attached Figure Description

[0018] Figure 1 This is a schematic diagram illustrating the steps of a UAV-based stereo mapping method according to the present invention. Figure 2 This is a schematic diagram of a UAV-based stereo mapping system according to the present invention. Detailed Implementation

[0019] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0020] Example 1: This invention provides a method for stereo mapping based on unmanned aerial vehicles (UAVs). Please refer to [link / reference]. Figure 1 This includes the following steps: S1. Control the UAV to fly along the preset trajectory, collect aerial images, and form flight trajectory data Tra, multi-view image sequence Img, and image shooting pose data Pos. S2. Based on the image sequence Img and pose data Pos, feature points are extracted to form a feature point set Fea, and a depth estimation result Dep is generated through multi-view depth estimation. Based on the depth estimation result Dep, a ground normal field Nor describing the orientation of the ground surface is constructed. S3. Based on flight track data Tra, pose data Pos, and ground object normal field Nor, calculate the solid angle evaluation quantity of the local area to form the solid angle evaluation result Ang, and obtain the set of solid angle bad areas Bad by judging the result against the preset threshold. S4. Generate an automatic yaw command Cmd to adjust the UAV's heading based on the angle deviation in the Bad set of solid angle defective areas, update the flight track data Tra to form updated track data NewTra, and send it to the UAV for application.

[0021] In this embodiment, the depth estimation result Dep, formed based on the image sequence Img and pose data Pos, can accurately construct the ground normal field Nor, which describes the true surface orientation, in areas with complex surface structures and overlapping slopes. This avoids the problem of insufficient stereo parallax caused by the inability to identify changes in ground orientation in traditional fixed-track methods. Subsequently, the stereo angle evaluation result Ang is obtained by calculating the spatial angle between the observation direction vector Dir and the ground normal field Nor. Based on areas with stereo angle defects below a preset threshold, a set of bad stereo angle defects is established. This method can identify spatial locations where insufficient stereo angle is caused by factors such as excessively tall building facades, steep terrain slopes, and valley obstructions during actual flight. For example, in urban blocks, stereo reconstruction often fails when UAVs fly along fixed routes because the orientation of tall building facades is almost parallel to the observation direction. This method can automatically calculate the corresponding yaw adjustment amount Yaw and generate an automatic yaw command Cmd after the area is identified as a bad stereo angle defect set, further forming updated track data NewTra that can improve stereo angle conditions. By automatically correcting the heading of defective areas, the UAV can proactively change its observation angle at necessary locations, thereby obtaining imagery that better meets the requirements of stereo reconstruction. Overall, this method not only significantly reduces the missing rate of 3D reconstruction in areas such as sloping facades, steep slopes, and roads and canyons, but also improves the stereo parallax quality of the imagery without additional hardware, thus significantly improving the accuracy and completeness of stereo mapping results for UAVs in complex terrain environments.

[0022] Example 2: Specifically: S1 includes S11; S11. During the flight of the UAV according to the preset trajectory, the flight status information is collected in real time through the flight control and navigation module of the UAV. The flight status information includes longitude, latitude and altitude obtained by the global navigation satellite system receiving module, as well as pitch angle, roll angle and yaw angle obtained by the inertial measurement unit. By using a unified time reference of the UAV's flight control system, longitude, latitude, altitude, pitch angle, roll angle, and yaw angle are synchronously recorded in chronological order, and continuous data processing is performed according to a fixed sampling period to compensate for attitude jumps caused by sampling period differences, thereby generating flight track data Tra to describe the UAV's flight path. During the generation of the flight track data Tra, interpolation correction is performed on longitude, latitude, and altitude. The interpolation correction uses linear interpolation or piecewise spline interpolation to form a continuous spatial position sequence of longitude, latitude, and altitude between sampling points. At the same time, smoothing is performed on pitch, roll, and yaw angles using moving average or first-order low-pass filtering to reduce abrupt changes in attitude data caused by measurement noise, so that pitch, roll, and yaw angles form a stable and usable attitude sequence. This ensures that the flight track data Tra can accurately reflect the spatial position and attitude of the UAV at each image capture time.

[0023] S1 further includes S12; S12. While generating flight track data Tra, based on the heading overlap and lateral overlap of the mapping task, the UAV imaging device is controlled to collect aerial images at a fixed shooting frequency to form a multi-view image sequence covering the mapping area, resulting in image sequence Img. For each image in the image sequence Img, based on the unified time reference provided by the flight control system, the shooting time of each image in the image sequence Img is matched with the longitude, latitude, altitude, pitch angle, roll angle and yaw angle of the corresponding time in the flight track data Tra, so that the shooting position and shooting attitude of each image are obtained. Among them, longitude, latitude and altitude are used as the shooting position of the image, and pitch angle, roll angle and yaw angle are used as the shooting attitude of the image, forming pose data Pos used to describe the external parameters of image imaging; After generating pose data Pos, attitude angle geometric correction is performed on pitch, roll, and yaw angles to improve the correspondence between the image imaging direction and the actual flight attitude, thereby improving the geometric consistency of subsequent depth estimation.

[0024] In this embodiment, the flight track data Tra avoids the pose jumps and abrupt attitude changes common in traditional methods during UAV flight, thus providing a continuous and stable spatial position and attitude sequence at the time of image capture. Combining this continuous track information, this step further generates multi-view images by fixing the shooting frequency and matching the shooting time with the flight track data Tra one by one, ensuring that each aerial image obtains an accurate shooting position and attitude, ultimately forming pose data Pos that accurately reflects the imaging extrinsic parameters. In actual mapping scenarios, such as when a UAV flies over mountainous terrain, airflow disturbances can cause slight fluctuations in altitude, pitch angle, or yaw drift. Conventional UAV track recording methods would record these attitude changes as discontinuous jumps, thus affecting the accuracy of the temporal correlation between images and extrinsic parameters. The interpolation correction and smoothing methods used in this step can convert the discrete records caused by these disturbances into a continuous and reliable pose sequence, enabling the images to achieve higher geometric consistency when forming pose data Pos. This provides a reliable external parameter basis for subsequent depth estimation, feature matching, and 3D modeling, and significantly improves the stability of image registration and reconstruction quality, especially in complex terrain or narrow urban flight paths.

[0025] Example 3: Specifically: S2 includes S21; S21. Based on the acquired image sequence Img and pose data Pos, feature point detection processing is performed on each image. Image features with texture changes are identified through feature extraction algorithm, and corresponding feature points are determined among multiple images using feature matching algorithm to form a feature point set Fea for depth estimation. After obtaining the feature point set Fea, depth estimation is performed based on the spatial projection relationship of the feature points in the multi-view image, combined with the image extrinsic parameters provided by the pose data Pos, through multi-view geometric constraints. The depth estimation uses a multi-view depth calculation method based on the principle of triangulation. By solving the projection difference of feature points under different views, the spatial depth value of the feature points is obtained, forming a depth estimation result covering the surveyed area, which is marked as the depth estimation result Dep, providing a spatial geometric basis for subsequent calculation of the surface orientation of ground features. It should be noted that the feature point set Fea is a set of feature points generated by identifying locations with obvious texture features after analyzing the texture changes in the image. Cross-image correspondence is established by comparing texture description values ​​in different images. Since the feature points have corresponding relationships in multiple images, the changes in the projection positions of the feature points in different images form calculable geometric constraints, which provide a stable spatial correlation basis for subsequent depth solutions. The depth estimation result Dep is a set of depth values ​​obtained after obtaining the correspondence between feature points across different viewpoints. It is obtained by using the positional differences of feature points in each image and combining the shooting position and shooting posture of each image to perform geometric solution. The depth solution includes positional difference analysis and triangulation calculation, so that each feature point can obtain accurate spatial depth information, thereby providing usable spatial geometric input for subsequent inference of the surface orientation of ground features.

[0026] S2 further includes S22; S22. Based on the obtained depth estimation result Dep, for each spatial depth value in the depth estimation result Dep, determine several spatial depth values ​​that are spatially adjacent to the spatial depth value as a depth neighborhood set. By comparing the change in height difference and the trend of depth gradient between each spatial depth value and the corresponding spatial depth value in the depth neighborhood set, deduce the surface orientation of the ground feature at the location corresponding to the spatial depth value, and obtain the direction vector used to represent the surface orientation at the corresponding location. After generating all direction vectors from the depth estimation result Dep, the direction vectors corresponding to adjacent spatial depth values ​​are weighted and fused to ensure that the direction derivation results remain continuous and stable in space, forming the ground normal field Nor used to express the orientation structure of the ground surface. It should be noted that: The surface normal field (Nor) is a set of surface direction vectors formed after continuous local geometric derivation of the spatial depth values ​​in the depth estimation result (Dep). The derivation process of the surface direction of the ground object first determines a depth neighborhood set among several spatial depth values ​​that are adjacent to a certain spatial depth value. By comparing the change in height difference between the central spatial depth value and each spatial depth value in the depth neighborhood set, a height difference sequence reflecting the rate of height change is formed. Subsequently, based on the differences in the height difference sequence in different spatial orientations, the local height increase direction and the local height decrease direction are derived. By calculating the proportion of the height change trend in the horizontal and vertical directions, the spatial distribution of the directions that best describe the upward and downward trends of the ground object surface is obtained. Based on the directional distribution, it can be determined which direction the ground surface has the most significant height change at the spatial depth value location. Then, based on the minimum and maximum trend directions of the height change, the direction vector of the ground surface can be calculated, so that the direction vector can describe the tilt direction and orientation characteristics of the ground surface at that location. After forming the initial direction vector, the direction vectors corresponding to the depth neighborhood set adjacent to the spatial depth value are weighted and fused to ensure that the direction information remains continuous and consistent within the neighborhood. The weighted fusion process assigns a weight to each direction vector, and the weight is determined based on the change in height difference between spatial depth values ​​and the spatial distance. Direction vectors with gentler height changes and closer spatial positions receive a larger weight. By superimposing and normalizing multiple direction vectors according to their weights, the fused direction vector retains the original local surface morphology features and has spatial continuity. This allows the final ground feature normal field Nor to accurately represent the overall orientation structure of the ground feature surface, providing a reliable geometric direction basis for subsequent solid angle quality determination and track yaw adjustment. The depth neighborhood set is a set of spatial depth values ​​that are relatively close in spatial distance, selected from the depth estimation result Dep based on the actual positional relationship of the spatial depth values ​​in three-dimensional space. The depth neighborhood set provides the necessary height change sequence for local geometric derivation, so that the surface orientation derivation does not depend on a single spatial depth value, but is calculated based on the continuous change trend of multiple spatial depth values, making the acquisition of the surface orientation of ground objects more stable. By providing local height trends, the depth neighborhood set ensures the spatial continuity and accuracy of the derivation of the surface orientation of ground objects, and is an important input source for the entire orientation vector derivation.

[0027] In this embodiment, by jointly processing the image sequence Img and pose data Pos, this step can continuously acquire spatially consistent feature point matching results under complex terrain structures, thereby forming a stable and reliable feature point set Fea. Based on this, by utilizing the projection differences of multi-view images to calculate the depth estimation result Dep, the accuracy of depth estimation can be effectively improved in areas with sparse texture or repetitive textures, ensuring the continuity of depth data even under long-distance parallax or lateral perspectives. For example, when a drone conducts aerial photography of areas with weak textures such as large areas of concrete walls, metal factory roofs, or homogeneous forest areas, traditional methods often result in large gaps in depth estimation due to insufficient feature points. However, this step, by capturing the feature point set Fea formed by cross-view texture changes, enables the depth estimation result Dep to recover relatively stable depth change information in such areas. Furthermore, based on the continuous geometric derivation of each spatial depth value in the depth estimation result Dep, this step can generate a surface orientation that truly reflects the local height change trend of the terrain. By weighted fusion of orientations among depth neighborhoods, a terrain normal field Nor is formed, enabling a smooth and continuous representation of the surface orientation of tilted surfaces, tortuous terrain, and areas of slope variation. For example, in hilly terrain, slopes change frequently over short distances. Traditional depth fields often exhibit discrete jumps, while the terrain normal field Nor generated in this step can accurately describe the orientation differences of these slope changes, providing a continuous and reliable geometric foundation for subsequent solid angle quality determination. This avoids stereo reconstruction errors caused by inconsistent surface orientations in areas of dramatic terrain undulation.

[0028] Example 4: Specifically: S3 includes S31; S31. Based on the obtained flight track data Tra and pose data Pos, as well as the ground normal field Nor, the survey area is spatially divided, and each local area after spatial division is matched to determine the corresponding shooting position and shooting posture in each local area. Based on the pitch, roll, and yaw angles recorded in the shooting posture, the observation direction of each local area is calculated to form an observation direction vector Dir representing the observation direction of each local area. Based on the surface orientation of the ground object recorded in the ground object normal field Nor, the spatial angle between the observation direction vector Dir and the surface orientation of the ground object is calculated, and the spatial angle is used as the solid angle evaluation quantity of the local area. By summarizing the solid angle evaluation values ​​of all local regions, an evaluation result of the solid angle quality distribution is formed and marked as the solid angle evaluation result Ang; It should be noted that: The solid angle evaluation result Ang is a set of evaluation results formed by calculating the spatial angle between the observation direction vector Dir and the corresponding ground surface direction in the ground surface normal field Nor after analyzing the shooting position in the flight track data Tra and the shooting attitude in the pose data Pos. The calculation process of solid angle evaluation first uses the pitch angle, roll angle and yaw angle recorded in the shooting attitude to determine the observation direction, so that the observation direction vector Dir can accurately describe the actual observation direction of the image to the ground object. Then, the angle between the two is calculated based on the ground surface direction to obtain the solid angle value of each local area. The closer the solid angle value is to the expected range, the more conducive it is to the formation of stable stereo parallax in that area. When the solid angle value deviates from the expected range, the area is more likely to have problems such as unstable matching, increased depth error or structural loss in subsequent 3D reconstruction. Therefore, the solid angle evaluation result Ang is used to represent the stereo geometric conditions of the entire mapping area under the current flight track and current shooting attitude, providing a clear numerical basis for subsequent identification of solid angle poor areas.

[0029] S3 further includes S32; S32. Based on the obtained solid angle evaluation result Ang, perform threshold determination on the solid angle evaluation quantity corresponding to each local region in the solid angle evaluation result Ang. The threshold is determined as follows: When the solid angle evaluation value is less than the preset threshold, the local area is marked as a solid angle insufficient area; When the solid angle evaluation value is greater than or equal to the preset threshold, the local area is marked as the solid angle satisfaction area; The solid angle evaluation of all local areas is judged one by one to complete the threshold determination. The spatial locations of all areas marked as solid angle insufficient are summarized to form a set of solid angle unfavorable areas, and thus the set of solid angle bad areas is obtained. It should be noted that: The "Bad" set of solid angle deficiencies is a set of regions formed by applying a threshold judgment operation to each solid angle evaluation value, based on the solid angle evaluation value of each local region given by the solid angle evaluation result Ang. When the solid angle evaluation value of a local region is less than the preset solid angle threshold, the local region is identified as a region with insufficient solid angle. It is believed that under the current observation conditions, it is difficult to obtain stable stereo parallax information in this region, which is prone to matching failure, amplified depth error, or missing local models during the 3D reconstruction process. By uniformly classifying all local regions with solid angle evaluation values ​​less than the preset solid angle threshold into the "Bad" set of solid angle deficiencies, the flight direction and shooting attitude are adjusted in the subsequent automatic yaw step according to the position indicated by the "Bad" set of solid angle deficiencies, thereby improving the stereo geometry conditions of these regions, reducing reconstruction risks, and improving the integrity and accuracy of the overall stereo map.

[0030] In this embodiment, the calculated solid angle evaluation result Ang enables a quantifiable spatial diagnosis of the solid geometry conditions of the entire surveying area before the UAV enters the 3D reconstruction stage, allowing the observation direction quality of each local area to be clearly expressed numerically. By calculating the spatial angle formed by the observation direction vector Dir and the ground normal field Nor, this step can identify spatial regions that are difficult to form stable parallax due to improper observation direction during UAV flight, and summarize them into a set of poor solid angle regions, Bad. In actual aerial surveying scenarios, such as when the UAV flies along roads, canyons, or narrow alleys, cliffs or building facades on one side of the road may be at very unfavorable observation angles, making the observation direction vector Dir almost parallel to the ground surface, thus causing stereo matching failure. Traditional methods often only discover these problems after 3D reconstruction fails, but with the solid angle evaluation result Ang generated in this step, the system can locate the locations of insufficient solid angle in real time during the image capture stage, accurately classifying these areas into the set of poor solid angle regions, Bad. This predictive capability enables drones to avoid reconstruction risks in advance during subsequent stages, preventing large-area structural voids, facade collapses, or depth error accumulation caused by small observation angles of facades or slopes in traditional methods. It provides a clear target area for subsequent automatic yaw adjustments, significantly improving the success rate of 3D reconstruction and the integrity of the overall real-world model.

[0031] Example 5: Specifically: S4 includes S41; S41. Based on the obtained set of poor solid angle regions Bad and the solid angle evaluation result Ang, combined with the observation direction vector Dir and the ground object normal field Nor, calculate the solid angle deviation between the actual observation direction and the ground object surface direction for each local area in the set of poor solid angle regions Bad. Based on the difference between the solid angle deviation and the preset solid angle target range, determine the yaw rotation angle required to restore the solid angle to the target range, and form the yaw adjustment amount Yaw corresponding to each local area; After determining the yaw adjustment amount (Yaw) for each local area, the yaw adjustment amount (Yaw) is associated with the corresponding flight position and flight time in the flight track data (Tra) so that the track segment of the solid angle poor area set Bad meets the yaw adjustment requirements. By arranging and organizing the yaw adjustment values ​​(Yaw) of all track segments, an automatic yaw command (Cmd) is formed to control the UAV to adjust its heading during flight, providing a clear basis for yaw control for subsequent track updates. It should be noted that: The automatic yaw command Cmd is a set of control commands formed after identifying the spatial location and solid angle deviation corresponding to the set of poor solid angle regions Bad. It establishes a correspondence between the yaw adjustment amount (Yaw) of each local region and the specific flight time and position in the flight track data Tra. The formation process determines the required increase or decrease in solid angle for each poor solid angle region based on the solid angle evaluation value in the solid angle evaluation result Ang. This adjustment requirement is then converted into an angle value representing the heading change, i.e., the yaw adjustment amount (Yaw). These yaw adjustment requirements are then sorted and combined according to the flight sequence. The automatic yaw command Cmd formed in this way can perform directional heading adjustments on a specified flight track segment during actual UAV flight. When the UAV passes through the region corresponding to the set of poor solid angle regions Bad, it changes its original observation direction, making the solid angle between the observation direction and the ground surface direction closer to the suitable range for stereo reconstruction, thus creating more favorable geometric conditions for subsequent 3D reconstruction.

[0032] S4 also includes S42; S42. Based on the generated automatic yaw command Cmd, calculate the heading adjustment for the track segment in the flight track data Tra that corresponds to the automatic yaw command Cmd. For each track segment that needs to be yawed, adjust the heading angle based on the original flight direction according to the yaw adjustment amount Yaw corresponding to the track segment to obtain a new flight direction. Then, based on the new flight direction and the UAV's predetermined flight speed, the flight path of the track segment is recalculated, causing the position sequence of the track segment in space to change; After performing heading adjustment calculations on all track segments that need adjustment, updated track data NewTra, which includes the spatial trajectory after heading correction, is generated. After obtaining the updated trajectory data NewTra, the UAV is re-controlled to fly along the updated trajectory to re-observe the area corresponding to the Bad solid angle region set, and to obtain image data under the new observation direction, so as to replace the original image observation results under the condition of insufficient solid angle in the subsequent reconstruction. It should be noted that: NewTra is a set of flight paths recalculated based on flight path data Tra. It is generated by applying a yaw adjustment (Yaw) to the corresponding path segment after addressing the Bad solid angle region set. During the process of creating NewTra, the spatial position sequence and heading information from the original flight path data Tra are used as a foundation. Angle corrections are made to the heading of specific path segments, causing them to extend in space along new directions, thus changing the observation direction of the UAV during flight. After heading correction, NewTra maintains the coverage of the original mapping area and provides more favorable observation geometry for stereo reconstruction when passing through the Bad solid angle region set. This improves the reliability and accuracy of subsequent images and stereo reconstruction results acquired based on NewTra in areas with insufficient solid angle.

[0033] In this embodiment, by generating an automatic yaw command Cmd and updating the trajectory data NewTra based on the command, the UAV can make precise and directional heading corrections for the observation direction at specific locations during flight, enabling areas that were previously at a disadvantage in observation to obtain new stereoscopic observation conditions. Unlike traditional fixed-route mapping methods, this step can not only identify the specific change in observation angle required for each local area in the set of bad solid angle areas, but also quantify the solid angle deviation into an executable heading adjustment angle through the yaw adjustment amount Yaw, allowing the UAV to complete real-time trajectory correction before entering bad areas. For example, when the UAV flies along a valley to photograph a steep slope, a fixed route often prevents the observation direction vector Dir from forming a sufficient solid angle with the slope surface for a long time, resulting in slope collapse or blurred edges in subsequent 3D reconstruction. However, the automatic yaw command Cmd generated in this step will adjust the heading in advance when the UAV approaches the area, so that the new flight direction can more effectively face the ground surface. The updated track data (NewTra) generated after heading correction not only maintains the coverage continuity of the original survey area but also allows the UAV to obtain a more suitable observation angle when passing through areas corresponding to the "Bad" region (area with poor solid angles). This results in clear stereo parallax information in subsequently acquired new imagery. Overall, this step enables the UAV to actively compensate for areas with insufficient solid angles, significantly reducing the risk of missing 3D structures caused by fixed tracks, and effectively improving the geometric stability and detail reproduction of stereo reconstruction in complex terrain or densely built-up areas.

[0034] Example 6: A UAV-based stereo mapping system, please refer to... Figure 2 Specifically, it includes a drone aerial photography data acquisition module, a data feature extraction and calculation module, a three-dimensional defect area determination module, and a drone flight path update module; The UAV aerial photography data acquisition module controls the UAV to fly along a preset trajectory, collect aerial images, and form flight trajectory data Tra, multi-view image sequence Img, and image shooting pose data Pos. The data feature extraction and estimation module extracts feature points based on the image sequence Img and pose data Pos to form a feature point set Fea, and generates a depth estimation result Dep through multi-view depth estimation. Based on the depth estimation result Dep, a ground normal field Nor describing the orientation of the ground surface is constructed. The solid angle defect determination module calculates the solid angle evaluation quantity of the local area based on the flight track data Tra, pose data Pos, and ground object normal field Nor to form a solid angle evaluation result Ang, and obtains the solid angle defect area set Bad by judging the result against the preset threshold. The UAV route update module generates an automatic yaw command Cmd to adjust the UAV's heading based on the angle deviation in the Bad set of solid angle defects, updates the flight track data Tra to form updated track data NewTra, and sends it to the UAV for application.

[0035] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for stereo mapping based on unmanned aerial vehicles (UAVs), characterized in that: Includes the following steps: S1. Control the UAV to fly along the preset trajectory, collect aerial images, and form flight trajectory data Tra, multi-view image sequence Img, and image shooting pose data Pos. S2. Based on the image sequence Img and pose data Pos, feature points are extracted to form a feature point set Fea, and a depth estimation result Dep is generated through multi-view depth estimation. Based on the depth estimation result Dep, a ground normal field Nor describing the orientation of the ground surface is constructed. S3. Based on flight track data Tra, pose data Pos, and ground object normal field Nor, calculate the solid angle evaluation quantity of the local area to form the solid angle evaluation result Ang, and obtain the set of solid angle bad areas Bad by judging the result against the preset threshold. S4. Generate an automatic yaw command Cmd to adjust the UAV's heading based on the angle deviation in the Bad set of solid angle defective areas, update the flight track data Tra to form updated track data NewTra, and send it to the UAV for application.

2. The method for stereo mapping based on unmanned aerial vehicles (UAVs) according to claim 1, characterized in that: S1 includes S11; S11. During the flight of the UAV according to the preset trajectory, the flight status information is collected in real time through the flight control and navigation module of the UAV. The flight status information includes longitude, latitude and altitude obtained by the global navigation satellite system receiving module, as well as pitch angle, roll angle and yaw angle obtained by the inertial measurement unit. By using a unified time reference of the UAV's flight control system, longitude, latitude, altitude, pitch angle, roll angle, and yaw angle are recorded synchronously in chronological order, and continuous data processing is performed according to a fixed sampling period to compensate for attitude jumps caused by sampling period differences, thereby generating flight track data Tra used to describe the UAV's flight path.

3. The method for stereo mapping based on unmanned aerial vehicles (UAVs) according to claim 2, characterized in that: S1 further includes S12; S12. While generating flight track data Tra, based on the heading overlap and lateral overlap of the mapping task, the UAV imaging device is controlled to collect aerial images at a fixed shooting frequency to form a multi-view image sequence covering the mapping area, resulting in image sequence Img. For each image in the image sequence Img, based on the unified time reference provided by the flight control system, the shooting time of each image in the image sequence Img is matched with the longitude, latitude, altitude, pitch angle, roll angle and yaw angle of the corresponding time in the flight track data Tra, so that each image obtains the shooting position and shooting attitude. In this method, longitude, latitude, and altitude are used as the image capture position, and pitch angle, roll angle, and yaw angle are used as the image capture attitude, forming pose data Pos used to describe the image imaging extrinsic parameters.

4. The method for stereo mapping based on unmanned aerial vehicles (UAVs) according to claim 3, characterized in that: S2 includes S21; S21. Based on the acquired image sequence Img and pose data Pos, feature point detection processing is performed on each image. Image features with texture changes are identified through feature extraction algorithm, and corresponding feature points are determined among multiple images using feature matching algorithm to form a feature point set Fea for depth estimation. After obtaining the feature point set Fea, depth estimation is performed based on the spatial projection relationship of the feature points in the multi-view image, combined with the image extrinsic parameters provided by the pose data Pos, through multi-view geometric constraints. The depth estimation uses a multi-view depth calculation method based on the principle of triangulation. By solving the projection difference of feature points under different views, the spatial depth value of the feature points is obtained, forming a depth estimation result covering the surveyed area, which is marked as the depth estimation result Dep, providing a spatial geometric basis for subsequent calculation of the surface orientation of ground features.

5. The method for stereo mapping based on unmanned aerial vehicles (UAVs) according to claim 4, characterized in that: S2 further includes S22; S22. Based on the obtained depth estimation result Dep, for each spatial depth value in the depth estimation result Dep, determine several spatial depth values ​​that are spatially adjacent to the spatial depth value as a depth neighborhood set. By comparing the change in height difference and the trend of depth gradient between each spatial depth value and the corresponding spatial depth value in the depth neighborhood set, deduce the surface orientation of the ground feature at the location corresponding to the spatial depth value, and obtain the direction vector used to represent the surface orientation at the corresponding location. After generating all direction vectors from the depth estimation result Dep, the direction vectors corresponding to adjacent spatial depth values ​​are weighted and fused to form the ground feature normal field Nor.

6. The method for stereo mapping based on unmanned aerial vehicles (UAVs) according to claim 5, characterized in that: S3 includes S31; S31. Based on the obtained flight track data Tra and pose data Pos, as well as the ground normal field Nor, the survey area is spatially divided, and each local area after spatial division is matched to determine the corresponding shooting position and shooting posture in each local area. Based on the pitch, roll, and yaw angles recorded in the shooting posture, the observation direction of each local area is calculated to form an observation direction vector Dir representing the observation direction of each local area. Based on the surface orientation of the ground object recorded in the ground object normal field Nor, the spatial angle between the observation direction vector Dir and the surface orientation of the ground object is calculated, and the spatial angle is used as the solid angle evaluation quantity of the local area. By summarizing the solid angle evaluation values ​​of all local regions, an evaluation result of the solid angle quality distribution is formed and marked as the solid angle evaluation result Ang.

7. The method for stereo mapping based on unmanned aerial vehicles (UAVs) according to claim 6, characterized in that: S3 further includes S32; S32. Based on the obtained solid angle evaluation result Ang, perform threshold determination on the solid angle evaluation quantity corresponding to each local region in the solid angle evaluation result Ang. The threshold is determined as follows: When the solid angle evaluation value is less than the preset threshold, the local area is marked as a solid angle insufficient area; When the solid angle evaluation value is greater than or equal to the preset threshold, the local area is marked as the solid angle satisfaction area; The solid angle evaluation values ​​of all local areas are evaluated one by one to determine the threshold. The spatial locations of all areas marked as having insufficient solid angle are summarized to form a set of unfavorable solid angle areas, resulting in the set of bad solid angle areas.

8. The method for stereo mapping based on unmanned aerial vehicles (UAVs) according to claim 7, characterized in that: S4 includes S41; S41. Based on the obtained set of poor solid angle regions Bad and the solid angle evaluation result Ang, combined with the observation direction vector Dir and the ground object normal field Nor, calculate the solid angle deviation between the actual observation direction and the ground object surface direction for each local area in the set of poor solid angle regions Bad. Based on the difference between the solid angle deviation and the preset solid angle target range, determine the yaw rotation angle required to restore the solid angle to the target range, and form the yaw adjustment amount Yaw corresponding to each local area; After determining the yaw adjustment amount (Yaw) for each local area, the yaw adjustment amount (Yaw) is associated with the corresponding flight position and flight time in the flight track data (Tra) so that the track segment of the solid angle poor area set Bad meets the yaw adjustment requirements. By arranging and organizing the yaw adjustment values ​​(Yaw) for all track segments, an automatic yaw command (Cmd) is formed to control the UAV to adjust its heading during flight, providing a clear basis for yaw control for subsequent track updates.

9. A method for stereo mapping based on unmanned aerial vehicles (UAVs) according to claim 8, characterized in that: S4 also includes S42; S42. Based on the generated automatic yaw command Cmd, calculate the heading adjustment for the track segment in the flight track data Tra that corresponds to the automatic yaw command Cmd. For each track segment that needs to be yawed, adjust the heading angle based on the original flight direction according to the yaw adjustment amount Yaw corresponding to the track segment to obtain a new flight direction. Then, based on the new flight direction and the UAV's predetermined flight speed, the flight path of the track segment is recalculated, causing the position sequence of the track segment in space to change; After performing heading adjustment calculations on all track segments that need adjustment, updated track data NewTra, which includes the spatial trajectory after heading correction, is generated. After obtaining the updated track data NewTra, the UAV is re-controlled to fly along the updated track to re-observe the area corresponding to the Bad solid angle defect set and acquire image data from the new observation direction.

10. A UAV-based stereo mapping system, applied to the UAV-based stereo mapping method described in any one of claims 1 to 9, characterized in that: It includes a drone aerial photography data acquisition module, a data feature extraction and calculation module, a three-dimensional defect area determination module, and a drone flight path update module; The UAV aerial photography data acquisition module controls the UAV to fly along a preset trajectory, collect aerial images, and form flight trajectory data Tra, multi-view image sequence Img, and image shooting pose data Pos. The data feature extraction and estimation module extracts feature points based on the image sequence Img and pose data Pos to form a feature point set Fea, and generates a depth estimation result Dep through multi-view depth estimation. Based on the depth estimation result Dep, a ground normal field Nor describing the orientation of the ground surface is constructed. The solid angle defect determination module calculates the solid angle evaluation quantity of the local area based on the flight track data Tra, pose data Pos, and ground object normal field Nor to form a solid angle evaluation result Ang, and obtains the solid angle defect area set Bad by judging the result against the preset threshold. The UAV route update module generates an automatic yaw command Cmd to adjust the UAV's heading based on the angle deviation in the Bad set of solid angle defects, updates the flight track data Tra to form updated track data NewTra, and sends it to the UAV for application.

Images