Oblique photography three-dimensional modeling method and system based on unmanned aerial vehicle

Abstract

The invention discloses an oblique photography three-dimensional modeling method and system based on an unmanned aerial vehicle, and particularly relates to the technical field of three-dimensional modeling. The method comprises the following steps: collecting multi-view inclined image data of a building through an unmanned aerial vehicle, and extracting feature point information in an adjacent image overlapping region; identifying real surface feature points of direct incident ray imaging and false reflection feature points formed by multi-mirror dynamic mutual reflection interference; eliminating the false reflection feature points by analyzing the spatial propagation trajectory features of the false reflection feature points and the topological constraint and texture continuity features of the real surface feature points to obtain an effective feature point set; and based on the effective feature point set, reconstructing and generating a building three-dimensional grid model meeting the surveying and mapping precision requirement. According to the method, the precision and reality of oblique photography three-dimensional modeling can be remarkably improved, and the method is suitable for high-precision three-dimensional modeling of urban building groups.

Description

Technical Field

[0001] This invention relates to the field of 3D modeling technology, and more specifically, to a method and system for 3D modeling based on oblique photogrammetry from unmanned aerial vehicles (UAVs). Background Technology

[0002] UAV oblique photogrammetry 3D modeling technology is widely used in the digital reconstruction of urban building complexes. In complex urban scenes, buildings often use glass curtain walls as their facade decoration, creating a multi-mirror reflection environment. The oblique image data acquired by UAVs in this multi-mirror reflection environment contains false secondary image features caused by mutual reflections between the glass curtain walls.

[0003] Existing image feature extraction and matching technologies cannot effectively distinguish between secondary reflection ghost images and real building surfaces, resulting in discrepancies between key indicators such as the shape, position, and size of local models in 3D modeling results and the actual scene, directly affecting the surveying accuracy and application reliability of 3D models. Summary of the Invention

[0004] In order to overcome the above-mentioned defects of the prior art, embodiments of the present invention provide a method and system for three-dimensional modeling based on oblique photogrammetry of unmanned aerial vehicles (UAVs) to solve the problems mentioned in the background art.

[0005] To achieve the above objectives, the present invention provides the following technical solution: The UAV-based oblique photogrammetry 3D modeling method includes the following steps: S1: Acquire the raw image sequence data obtained by the UAV during oblique photography, extract the feature point information of the overlapping area of ​​adjacent images in the raw image sequence data, and output the initial image feature point set; S2: Based on the initial image feature point set, extract the real surface feature points of direct incident light imaging and the false reflection feature points caused by dynamic mutual reflection of multiple mirrors to generate a composite feature set; S3: For false reflection feature points in the composite feature set, analyze the spatial propagation trajectory characteristics by utilizing the difference between the incident angle of light and the reflection propagation path, and generate reflection phantom trajectory data; S4: For real surface feature points in the composite feature set, generate real feature point matching constraint data based on spatial topological constraints and texture continuity feature analysis; S5: Based on the reflection phantom trajectory data and the matching constraint data of real feature points, identify and eliminate false reflection feature points in the composite feature set to obtain the effective feature point set; S6: Reconstruct 3D point clouds based on the effective feature point set to generate a 3D mesh model of a building that meets the requirements of surveying accuracy.

[0006] In a preferred embodiment, S1 specifically refers to: By controlling the drone to sequentially collect multi-view building images within a preset flight path and altitude range, the original image sequence data is obtained; Spatial alignment is performed on the original image sequence data to identify the spatial overlap region between each group of adjacent images; Image feature points are extracted within spatially overlapping regions. The pixel coordinates, orientation, and feature description information of each feature point in the original image are obtained, and an initial image feature point set containing feature point information of all spatially overlapping regions is constructed.

[0007] In a preferred embodiment, S2 specifically comprises: Based on the initial set of image feature points, and combined with the geometric relationship between the spatial position of the UAV camera, the lens viewpoint, and the normal direction of the building surface, the light propagation path of the image feature points is determined; Image feature points that meet the condition of directly reaching the drone camera lens from the building surface are determined to be real surface feature points; Image feature points that meet the condition that light reaches the drone camera lens after being reflected twice or multiple times by the glass curtain wall of a building are identified as false reflection feature points. A composite feature set is formed based on real surface feature points and false reflection feature points.

[0008] In a preferred embodiment, S3 specifically comprises: Based on the false reflection feature points in the composite feature set, and combined with the spatial position coordinates and lens view of the UAV camera, the incident angle and reflection angle of the incident and reflected light rays on the glass curtain wall surface of the building corresponding to each false reflection feature point are determined. Based on the incident angle and reflection angle of the light, the spatial trajectory path of the light propagation at each false reflection feature point is calculated and tracked to obtain the spatial trajectory path data of all false reflection feature points. By combining spatial trajectory path data, the similarity characteristics of false reflection feature points on the spatial propagation path are determined, and reflection ghost trajectory data is generated.

[0009] In a preferred embodiment, S4 specifically comprises: Based on the real surface feature points in the composite feature set, a spatial adjacency relationship is constructed, and the three-dimensional geometric distance and two-dimensional pixel distance between each real surface feature point and its nearest neighbor real surface feature point are calculated to form a spatial adjacency matrix. For the corresponding pairs of real surface feature points in the spatial adjacency matrix, extract the texture blocks from their respective original images, calculate the texture similarity, and generate a texture similarity matrix. The spatial adjacency matrix and texture similarity matrix are jointly thresholded and filtered, and a true feature point matching map is constructed. Output the true feature point matching constraint data based on the true feature point matching map.

[0010] In a preferred embodiment, S5 specifically comprises: Based on the reflection phantom trajectory data and the matching constraint data of real feature points, the difference in the spatial propagation trajectory between false reflection feature points and real surface feature points is determined. Based on the differences in spatial propagation trajectories, feature type classification is performed on all image feature points in the composite feature set. Remove false reflection feature points whose spatial propagation trajectory differences exceed a predetermined difference threshold; Preserve real surface feature points whose spatial propagation trajectory differences are within a predetermined difference threshold range; Based on the retained real surface feature points, a set of effective feature points is formed.

[0011] In a preferred embodiment, S6 specifically comprises: Based on the two-dimensional pixel coordinates of effective feature points in the original image sequence data and the spatial position of the UAV camera, the spatial three-dimensional coordinate data of each effective feature point is calculated to generate dense three-dimensional point cloud data containing the three-dimensional coordinate data of all effective feature points. Based on dense 3D point cloud data, a spatial triangular mesh partitioning method is used to construct triangular mesh topological connection relationships between adjacent effective feature points; Based on the 3D coordinate data corresponding to the effective feature points in the dense 3D point cloud data, the spatial position and geometry of each triangular mesh are determined, and a 3D mesh model of the building that meets the surveying accuracy requirements is generated.

[0012] On the other hand, the present invention provides an oblique photogrammetry 3D modeling system based on unmanned aerial vehicles (UAVs), comprising: Feature point extraction module: acquires the original image sequence data obtained by UAV oblique photography, extracts the feature point information of the overlapping area of ​​adjacent images in the original image sequence data, and outputs the initial image feature point set; Feature classification module: Based on the initial image feature point set, extract the real surface feature points of direct incident light imaging and the false reflection feature points caused by dynamic mutual reflection of multiple mirrors to generate a composite feature set; The ghost image analysis module analyzes the spatial propagation trajectory characteristics of false reflection feature points in the composite feature set by utilizing the difference between the incident angle of light and the reflection propagation path, and generates reflection ghost image trajectory data. Feature matching module: Based on spatial topological constraints and texture continuity feature analysis, it generates real feature point matching constraint data for real surface feature points in the composite feature set. The ghost image removal module identifies and removes false reflection feature points from the composite feature set based on the matching constraint data between the reflection ghost image trajectory data and the real feature points, thereby obtaining a set of effective feature points. 3D Reconstruction Module: Reconstructs 3D point clouds based on a set of effective feature points to generate 3D mesh models of buildings that meet the accuracy requirements of surveying and mapping.

[0013] The technical effects and advantages of the UAV-based oblique photogrammetry 3D modeling method and system are as follows: By extracting feature point information from overlapping areas of adjacent oblique photogrammetric images, the integrity and multi-view coverage of the initial image feature point set are ensured. A composite feature set is formed by distinguishing between real surface feature points imaged by direct incident light and false reflection feature points generated by dynamic mutual reflection from multiple mirrors, providing a basis for classification. Spatial trajectory feature analysis of false reflection feature points is performed using the differences in incident angle and reflection propagation path, accurately generating reflection ghost trajectory data and achieving high-precision positioning of false features. Depth matching constraint analysis is performed on real surface feature points based on spatial topological constraints and texture continuity, ensuring spatial consistency and texture similarity between real matching points and achieving high reliability of matching data. Combining reflection ghost trajectory data and real feature point matching constraint data, false reflection feature points are accurately identified and eliminated, constructing an effective feature point set and reducing the false matching rate. Based on the effective feature point set, 3D point cloud reconstruction is completed, outputting a 3D mesh model of a building that meets the accuracy requirements of surveying and mapping. This invention significantly improves the accuracy, stability, and applicability of oblique photogrammetric 3D modeling. Attached Figure Description

[0014] Figure 1 This is a schematic diagram of the UAV-based oblique photogrammetry 3D modeling method of the present invention; Figure 2 This is a schematic diagram of the structure of the UAV-based oblique photogrammetry 3D modeling system of the present invention. Detailed Implementation

[0015] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0016] Example 1 Figure 1 The present invention provides a UAV-based oblique photogrammetry 3D modeling method, which includes the following steps: S1: Acquire the raw image sequence data obtained by the UAV during oblique photography, extract the feature point information of the overlapping area of ​​adjacent images in the raw image sequence data, and output the initial image feature point set; S2: Based on the initial image feature point set, extract the real surface feature points of direct incident light imaging and the false reflection feature points caused by dynamic mutual reflection of multiple mirrors to generate a composite feature set; S3: For false reflection feature points in the composite feature set, analyze the spatial propagation trajectory characteristics by utilizing the difference between the incident angle of light and the reflection propagation path, and generate reflection phantom trajectory data; S4: For real surface feature points in the composite feature set, generate real feature point matching constraint data based on spatial topological constraints and texture continuity feature analysis; S5: Based on the reflection phantom trajectory data and the matching constraint data of real feature points, identify and eliminate false reflection feature points in the composite feature set to obtain the effective feature point set; S6: Reconstruct 3D point clouds based on the effective feature point set to generate a 3D mesh model of a building that meets the requirements of surveying accuracy.

[0017] S1 involves acquiring the original image sequence data obtained by the UAV during oblique photography, extracting feature point information from the overlapping areas of adjacent images in the original image sequence data, and outputting an initial set of image feature points, including: By controlling the drone to sequentially collect multi-view building images within a preset flight path and altitude range, the original image sequence data is obtained; The drone, equipped with an oblique photography camera, collects comprehensive, multi-angle image data of the building to be measured, based on a pre-set flight path and altitude range. The flight path is set as follows: based on the actual spatial coordinates of the building, several parallel or intersecting flight paths are established, with fixed horizontal intervals between them, such as 15 meters, to ensure overlap between adjacent images captured by the drone. The flight altitude range is determined by, for example, by measuring or estimating the building's height on-site, setting the flight altitude range above the building's top. For instance, if the building's highest point is 50 meters, the drone's flight altitude range is set to 60 to 80 meters to ensure the drone camera can comprehensively acquire image information from the building's top and all sides. The drone executes its flight mission according to the pre-set flight path and altitude range. For example, the drone first takes off from the mission's starting point and flies to the predetermined altitude range, then flies along the first flight path from the starting point to the end point while simultaneously taking pictures. After reaching the end of the first flight path, it turns to the second flight path and repeats the picture-taking process, completing the picture-taking task along all flight paths. The raw image sequence data collected by the drone fully covers all sides and top of the building, meeting the requirements for comprehensive 3D modeling. Each image is captured by the drone at an angle during flight. The angle, such as the angle between the drone camera and the normal to the building surface, is typically set between 15° and 45° to ensure spatial overlap between adjacent images. The drone automatically captures an angled image at regular intervals or along each flight path, such as every 10 meters or every 3 seconds, to obtain sufficient image data to cover the target building surface.

[0018] Spatial alignment is performed on the original image sequence data to identify the spatial overlap region between each group of adjacent images; By utilizing the spatial position and shooting attitude data recorded in real time by the drone during the shooting process, such as the spatial three-dimensional coordinate information and flight attitude information recorded by the onboard GPS and inertial measurement unit, the spatial position and shooting attitude information corresponding to each original image are obtained. Preliminary spatial alignment is performed on each image data in the original image sequence data according to its spatial position and shooting attitude information. For example, the image data is sorted using spatial position coordinates, and the ground coverage area of ​​each image is calculated based on the drone camera's field of view and shooting height, thus determining the spatial overlap area between each group of adjacent images. The spatial overlap area between each group of adjacent images is a local area on the side or top of a building simultaneously contained in both adjacent images. The size of the spatial overlap area is usually controlled by an overlap index; for example, the overlap of adjacent images in front is set to 80%, and the overlap of adjacent images on the side is set to 60%, to ensure spatial alignment between images.

[0019] Extract image feature points within spatially overlapping regions, obtain the pixel coordinates, orientation, and feature description information of each feature point in the original image, and construct an initial image feature point set containing feature point information of all spatially overlapping regions. In spatially overlapping regions, image feature extraction methods such as scale-invariant feature transformation (SIN) or accelerated robust feature extraction algorithms are used to detect a large number of spatial feature points from each image data. Each image feature point refers to a stable and reproducibly identifiable geometric point in the image, such as a pixel location with obvious texture changes, edge features, or corner features. For each feature point, the two-dimensional pixel coordinates of the feature point are calculated, specifically the horizontal and vertical pixel positions of the feature point in the original image coordinate system. Simultaneously, the orientation information of the feature point is calculated, for example, using feature extraction algorithms to calculate the principal direction of the local texture gradient around each feature point. After completing the calculation of the two-dimensional pixel coordinates and orientation information of each image feature point, feature description information is calculated for each image feature point, specifically a digital feature representation of the local image region around the feature point, such as a 128-dimensional or 64-dimensional vector digital description feature, used for image matching and feature classification. All feature points obtained through the above methods, along with their corresponding two-dimensional pixel coordinates, orientations, and feature description information, collectively construct an initial image feature point set containing feature point information from all spatially overlapping regions.

[0020] S2, based on the initial image feature point set, extracts the real surface feature points of direct incident light imaging and the false reflection feature points caused by dynamic mutual reflection of multiple mirrors, generating a composite feature set, including: Based on the initial set of image feature points, and combined with the geometric relationship between the spatial position of the UAV camera, the lens viewpoint, and the normal direction of the building surface, the light propagation path of the image feature points is determined; The initial image feature point set is the information of all image feature points obtained after extracting image feature points from spatially overlapping areas, including the two-dimensional pixel coordinates, orientation, and feature description information of each feature point. The spatial position of the UAV camera is the three-dimensional spatial coordinate position of the UAV recorded by the Global Positioning System (GPS) when capturing each image, represented as (X1, Y1, Z1). For example, when the UAV photographs a building, the spatial position coordinates are (100 meters, 200 meters, 70 meters). The lens angle of view refers to the pitch angle of the UAV camera lens relative to the horizontal ground and the azimuth angle relative to true north, for example, a pitch angle of 30 degrees and an azimuth angle of 45 degrees. The normal direction of the building surface is represented as a unit vector perpendicular to the building surface; for example, the normal direction vector of the side of the building can be represented as (Nx, Ny, Nz), such as (0, 1, 0). The ray propagation path corresponding to each image feature point represents the spatial propagation trajectory of light from the building surface to the UAV camera lens. Based on the above information, the light propagation path is determined by calculating the line vector connecting the optical center of the UAV camera and the feature point. Specifically, assuming the three-dimensional coordinates of the feature point are (X2, Y2, Z2), the line vector is obtained by subtracting the coordinates. The vector is calculated by subtracting the corresponding component of the three-dimensional spatial coordinates of the image feature point from each corresponding component of the UAV camera's spatial position coordinates to obtain the line vector L of the light propagation path. The angle between the line vector L and the normal direction vector N of the building surface is calculated through the inner product operation of the vectors, which is the incident angle of the light on the building surface. Specifically, the three components of the line vector are multiplied by the corresponding components of the normal direction vector and summed to obtain the inner product value of the line vector and the normal direction vector. Then, the magnitudes of the line vector and the normal direction vector are calculated separately by squaring the three components of each vector, adding the square roots, and dividing the inner product value by the product of the two magnitudes. The result is then calculated using the inverse cosine function to obtain the angle value, which is the angle between the line vector and the normal direction vector, and is the incident angle of the light, representing the geometric information of the light propagation path. The above method is used to determine the light propagation path of all image feature points in the initial image feature point set.

[0021] Image feature points that meet the condition of directly reaching the drone camera lens from the building surface are determined to be real surface feature points; Image feature points that directly reach the drone camera lens from the building surface refer to light received by the drone camera lens that, after reflection or diffuse reflection from the building surface, does not undergo any other reflection or refraction path. In other words, the light undergoes only a single reflection or diffuse reflection on the building surface and directly reaches the drone camera lens. This type of image feature point truly reflects the spatial location and texture characteristics of the building surface. The determination method is as follows: based on the light propagation path, determine the incident angle of the light at the normal direction of the building surface. When the angle between the incident angle and the normal meets a certain angular range, such as being less than or equal to the critical angle threshold for reflection or diffuse reflection of the building surface (e.g., within 45 degrees), it can be identified as a true surface feature point. For example, when analyzing the side of a building, if the light received by the drone camera lens is formed by direct diffuse reflection from the brick wall on the side of the building, without any other surface reflection path, and the incident angle is 30 degrees, which is less than the critical angle threshold for reflection or diffuse reflection of the building surface, this image feature point is identified as a true surface feature point. By determining the incident angle of the initial set of image feature points, all true surface feature points can be filtered out from the initial set for accurate 3D reconstruction.

[0022] Image feature points that meet the condition that light reaches the drone camera lens after being reflected twice or multiple times by the glass curtain wall of a building are identified as false reflection feature points. False reflection feature points refer to light rays received by a drone camera lens that are not directly reflected from the building surface in a single step, but undergo secondary or multiple reflections on the building's glass curtain wall before reaching the drone camera lens. This type of light propagation path causes the image feature points to not accurately reflect the spatial position of the building surface, resulting in significant spatial position errors and interfering with the reconstruction of the building's true 3D model. The method for identifying false reflection feature points is as follows: based on the light propagation path information, determine whether the light path has undergone two or more reflections. That is, after the first reflection occurs on the building surface, the light undergoes a second or more reflections with a smooth surface with strong reflective properties, such as a glass curtain wall, before finally reaching the drone camera lens. Multiple reflection paths are identified by the incident angle and reflection angle. For example, if the incident angle and reflection angle are the same during the first reflection on the building's glass curtain wall surface, and the light then encounters another glass curtain wall surface for a second reflection before finally reaching the drone camera lens. For example, a drone camera receives light that is first reflected off the east glass curtain wall of a tall building, then reflected a second time off another glass curtain wall on the adjacent south side before reaching the drone's lens. Based on the incident and reflection angles of each reflection, if the number of reflections exceeds one, the corresponding image feature point is identified as a false reflection feature point. By performing reflection count and path analysis on the initial set of image feature points, all false reflection feature points can be identified from the set.

[0023] A composite feature set is constructed based on real surface feature points and false reflection feature points; A composite feature set refers to a set of feature points containing two different types of features, formed after classifying all feature points in an initial set of image feature points as either real or fake. The composite feature set includes both real surface feature points and fake reflection feature points, and each feature point is labeled with its category information to ensure targeted classification processing. For example, when constructing the composite feature set, each real surface feature point is labeled as "real category," and each fake reflection feature point is labeled as "fake category." Each feature point in the composite feature set has a classification identifier to ensure the completeness and accuracy of the composite feature set.

[0024] S3, for false reflection feature points in the composite feature set, utilizes the difference between the incident angle of light and the reflection propagation path to analyze the spatial propagation trajectory characteristics and generate reflection phantom trajectory data, including: Based on the false reflection feature points in the composite feature set, and combined with the spatial position coordinates and lens view of the UAV camera, the incident angle and reflection angle of the incident and reflected light rays on the glass curtain wall surface of the building corresponding to each false reflection feature point are determined. The composite feature set consists of real surface feature points and false reflection feature points. False reflection feature points are those formed when light travels through the building's glass curtain wall, undergoing secondary or multiple reflections, before reaching the drone camera lens. The spatial coordinates of the drone camera refer to the three-dimensional spatial coordinates recorded by the drone when capturing each image. The lens angle represents the pitch angle of the drone camera lens relative to the horizontal ground and its azimuth angle relative to true north when capturing an image. For each false reflection feature point in the composite feature set, the spatial direction and coordinate position of the line vector connecting the light source from the drone camera lens to the false reflection feature point are determined using the drone camera's spatial coordinates and lens angle information. Based on the spatial position and surface normal direction of the building's glass curtain wall surface, the incident angle when the light reaches the glass curtain wall surface and the reflection angle after reflection from the glass curtain wall surface are calculated. Specifically: first, the spatial normal direction vector of the glass curtain wall surface is determined; for example, the surface normal vector of the building's glass curtain wall is represented as (Nx, Ny, Nz), specifically (0, 1, 0). The angle between the spatial line vector and the surface normal direction vector of the glass curtain wall is then used to calculate... The incident angle is calculated using the following formula: Each coordinate component of the connecting vector is multiplied by the corresponding component of the normal direction vector, and then summed to obtain the inner product of the connecting vector and the normal direction vector. Next, the magnitudes of the connecting vector and the normal direction vector are calculated by squaring each of their three components, adding them together, and taking the square root. The inner product is then divided by the product of the two magnitudes. The result is then calculated using the inverse cosine function to obtain the incident angle. The reflection angle is the same as the incident angle. Using the above method, the incident and reflection angles are calculated for each reflection path corresponding to the false reflection feature point, thus determining the incident and reflection angles of the light propagation paths for all false reflection feature points.

[0025] Based on the incident angle and reflection angle of the light, the spatial trajectory path of the light propagation at each false reflection feature point is calculated and tracked to obtain the spatial trajectory path data of all false reflection feature points. For each false reflection feature point in the composite feature set, the spatial coordinates of the light rays from each false reflection feature point on the surface of the building's glass curtain wall are calculated for each reflection process. This determines the change in the direction of light propagation after each reflection process, thus obtaining the complete spatial trajectory path data of the light propagation for each false reflection feature point. First, based on the spatial coordinates of the UAV camera and the lens angle, the spatial position where the light rays first propagate to the surface of the building's glass curtain wall and undergo the first reflection is determined. For example, if the UAV's spatial coordinates are (120 meters, 250 meters, 80 meters), the coordinates of the first reflection position are (130 meters, 260 meters, 60 meters). Based on the coordinates of the first reflection position and the direction of propagation after the first reflection, the position coordinates of the second propagation path of the light rays are calculated, and the coordinates of the second reflection position are determined. For example, the coordinates of the second reflection position are (135 meters, 255 meters, 55 meters). The spatial coordinates of each reflection of the light rays are determined in sequence using the above method. Connecting all the position coordinates sequentially forms a complete spatial trajectory path. The light propagation path is calculated for each false reflection feature point in the composite feature set to obtain the spatial trajectory path data corresponding to all false reflection feature points.

[0026] By combining spatial trajectory path data, the similarity characteristics of false reflection feature points on the spatial propagation path are determined, and reflection ghost trajectory data is generated; Spatial trajectory path data consists of the coordinates of each reflection position during the propagation of light from false reflection feature points. First, the difference in positional coordinate distance between the spatial trajectory paths of each false reflection feature point is calculated. For example, for the spatial trajectory paths of two different false reflection feature points, the coordinate difference corresponding to each reflection position is calculated separately: the difference in X-axis coordinates, Y-axis coordinates, and Z-axis coordinates of each reflection position is calculated separately, and then the spatial distance difference between the two trajectory paths is obtained by taking the square root of the sum of the squares of the three-axis coordinate differences. By setting a fixed spatial trajectory path difference threshold, such as 5 meters, the calculated difference values ​​of the trajectory paths of each false reflection feature point are compared with the spatial trajectory path difference threshold. If the difference value is less than the spatial trajectory path difference threshold, the corresponding two trajectory paths are considered to have similar spatial propagation path characteristics. All spatial propagation trajectory paths with similarity are determined using the above method and collectively labeled as similarity categories. Then, using all the spatial positional coordinate data in the similarity categories, reflection phantom trajectory data describing the similarity of the spatial trajectory paths of light propagation from all false reflection feature points is generated.

[0027] S4, based on spatial topological constraints and texture continuity feature analysis, generates real feature point matching constraint data for the real surface feature points in the composite feature set, including: Based on the real surface feature points in the composite feature set, a spatial adjacency relationship is constructed, and the three-dimensional geometric distance and two-dimensional pixel distance between each real surface feature point and its nearest neighbor real surface feature point are calculated to form a spatial adjacency matrix. True surface feature points refer to feature points formed by light directly reflected or diffusely reflected from the building surface that reach the drone camera lens without undergoing multiple reflection paths. First, the adjacency relationships between true surface feature points in three-dimensional space are determined through spatial adjacency. For example, a true surface feature point is selected as a reference point, and the differences in three-dimensional spatial coordinates of other true surface feature points relative to the reference point are calculated. This determines the three-dimensional distance relationships between all feature points. The method for determining spatial adjacency is as follows: for a reference true surface feature point, calculate the Euclidean distance between the spatial coordinates of other feature points and the coordinates of the reference feature point. This is done by subtracting the corresponding components of the three-dimensional coordinates of two true surface feature points along the X, Y, and Z axes, squared the coordinate differences along each axis, and then calculating... Then, perform a square root calculation to obtain the three-dimensional geometric distance between feature points. For example, if the three-dimensional coordinates of two real surface feature points are (100 meters, 200 meters, 50 meters) and (110 meters, 210 meters, 55 meters), the three-dimensional geometric distance between the two points can be obtained by the above calculation method: square the difference of 10 on the X-axis to get 100, square the difference of 10 on the Y-axis to get 100, and square the difference of 5 on the Z-axis to get 25. Add them together to get 225, and then take the square root of 225 to obtain a three-dimensional geometric distance of 15 meters. Calculate the distance between all real surface feature points and their nearest neighbor real surface feature points using the same method. The three-dimensional distance is calculated; the two-dimensional pixel distance between each real surface feature point and its nearest neighbor is calculated using the two-dimensional pixel coordinates of the real surface feature point in the original image. Specifically, the difference between the horizontal and vertical coordinates of the two feature points in the original image is squared, summed, and then the square root is taken to obtain the two-dimensional pixel distance between the two feature points in the original image. For example, if the two-dimensional coordinates of a real surface feature point in the original image are (300 pixels, 500 pixels), and the two-dimensional coordinates of its nearest neighbor real surface feature point are (310 pixels, 515 pixels), then the two feature points... The two-dimensional pixel distance is calculated as follows: the horizontal coordinate difference of 10 pixels squared equals 100, the vertical coordinate difference of 15 pixels squared equals 225, and the sum of these two values ​​is 325. Taking the square root of 325 gives a two-dimensional pixel distance of approximately 18.03 pixels. The above calculations of three-dimensional geometric distance and two-dimensional pixel distance are performed on each real surface feature point in the composite feature set. These three-dimensional geometric distances and two-dimensional pixel distances are then used to construct a spatial adjacency matrix. The spatial adjacency matrix uses real surface feature points as its rows and columns, with each matrix element corresponding to the calculated three-dimensional geometric distance and two-dimensional pixel distance between the real surface feature points.

[0028] For the corresponding pairs of real surface feature points in the spatial adjacency matrix, extract the texture blocks from their respective original images, calculate the texture similarity, and generate a texture similarity matrix. In a spatial adjacency matrix, a true surface feature point pair refers to a combination of points formed by each true surface feature point and its nearest neighbor true surface feature point determined in the spatial adjacency relationship. For each pair of true surface feature points, a local image region of a certain size is cropped from the original image, centered on the two-dimensional pixel coordinates of each feature point, as a texture block. For example, a square image region of size 31 pixels × 31 pixels is cropped with the center pixel as the coordinates of the true surface feature point (300 pixels, 500 pixels), forming the corresponding texture block. For the other true surface feature point in the point pair combination, a square texture block of the same size 31 pixels × 31 pixels is also cropped, centered on the two-dimensional pixel coordinates of this point. The texture similarity between the two texture blocks is calculated. Texture similarity is a numerical value reflecting the degree of similarity of image pixel grayscale values ​​or color features between texture blocks. For example, texture similarity is calculated using a normalized cross-correlation calculation method, specifically: calculating... The mean and standard deviation of all pixel grayscale values ​​within two texture blocks are calculated. Then, the pixel grayscale values ​​at corresponding positions in the two texture blocks are multiplied one by one and summed. The sum is divided by the product of the standard deviation of each texture block and the number of pixels to obtain the texture similarity. The texture similarity is between 0 and 1. The closer it is to 1, the more similar the two texture blocks are. For example, if the texture similarity of two texture blocks calculated by the above method is 0.95, it means that the two texture blocks have highly similar image features. The texture similarity is calculated by performing texture similarity calculation on all pairs of real surface feature points in the spatial adjacency matrix. All calculation results are used to form a texture similarity matrix. The texture similarity matrix uses real surface feature points as the rows and columns of the matrix, and each matrix element corresponds to the texture similarity of the feature point pair.

[0029] The spatial adjacency matrix and texture similarity matrix are jointly thresholded and filtered, and a true feature point matching map is constructed. Joint thresholding is a comprehensive screening method that utilizes the 3D geometric distance and 2D pixel distance values ​​in the spatial adjacency matrix, as well as the texture similarity in the texture similarity matrix. Joint threshold conditions are set for screening, such as a 3D geometric distance threshold of 20 meters, a 2D pixel distance threshold of 25 pixels, and a texture similarity threshold of 0.9. Only when the 3D geometric distance between real surface feature points is less than 20 meters, the 2D pixel distance is less than 25 pixels, and the texture similarity is greater than 0.9, is the feature point pair considered to meet the real feature point matching condition and retained for constructing the real feature point matching graph. The real feature point matching graph is a network structure with real surface feature points as nodes and the connections between feature point pairs that meet the joint thresholding condition as edges. For example, if the 3D geometric distance between two real surface feature points is 15 meters, the 2D pixel distance is 18 pixels, and the texture similarity is 0.95, then the two feature points are connected in the real feature point matching graph as a node-edge relationship. By performing joint thresholding on all real surface feature points using the above method, the real feature point matching graph is constructed.

[0030] Output the true feature point matching constraint data based on the true feature point matching map; The real feature point matching constraint data is a set of data that is based on the output of the real feature point matching map and is used to constrain and limit the matching relationship between the spatial location of real surface feature points and image texture features. It includes the three-dimensional geometric distance, two-dimensional pixel distance, texture similarity and corresponding feature point identification information between feature points, and is used to eliminate the interference of false reflection features and generate three-dimensional models of buildings.

[0031] In step S5, based on the matching constraint data between the reflected phantom trajectory data and the real feature points, false reflected feature points in the composite feature set are identified and eliminated to obtain a set of effective feature points, including: Based on the reflection phantom trajectory data and the matching constraint data of real feature points, the difference in the spatial propagation trajectory between false reflection feature points and real surface feature points is determined. By using the reflected virtual trajectory data and the matching constraint data of real feature points, the light propagation trajectory path characteristics of false reflection feature points and real surface feature points are determined, and then a spatial propagation trajectory difference analysis is performed. The spatial propagation trajectory difference analysis involves comparing the position coordinates of each reflection point in the spatial trajectory path data of the false reflection feature point with the spatial position coordinates of the real surface feature points in the matching constraint data. For example, taking a false reflection feature point as an example, if the reflection position coordinates in the spatial propagation trajectory path of the false reflection feature point are (135 meters, 255 meters, 55 meters), then the distance between these reflection position coordinates and the position coordinates of all real surface feature points in the matching constraint data is calculated one by one to determine the difference between the reflection position of the false reflection feature point and the real surface feature point. Specifically, the components of the reflection position coordinates of the false reflection feature point and the coordinates of the real surface feature point on the X, Y, and Z axes are subtracted respectively to obtain the differences in the three directions. The differences in each direction are squared and summed, and the square root of the sum is taken to obtain the spatial propagation trajectory difference index between the false reflection feature point and the real surface feature point. For example, if the spatial coordinates of a real surface feature point are (130 meters, 250 meters, 60 meters), then the spatial propagation trajectory difference is calculated as follows: the difference in X-axis coordinates squared by 5 equals 25, the difference in Y-axis coordinates squared by 5 equals 25, the difference in Z-axis coordinates squared by 5 equals 25, the sum is 75, and the square root of 75 gives a spatial propagation trajectory difference index of approximately 8.66 meters; the spatial propagation trajectory difference index between false reflection feature points and real surface feature points is calculated one by one using the above method.

[0032] Based on the differences in spatial propagation trajectories, feature type classification is performed on all image feature points in the composite feature set. Based on the calculated spatial propagation trajectory difference index, each image feature point in the composite feature set is classified. The judgment criterion is a pre-set difference threshold for spatial propagation trajectory differences, for example, a difference threshold of 10 meters. For each image feature point, the spatial propagation trajectory difference index is compared with the difference threshold. If the spatial propagation trajectory difference index of the image feature point exceeds the difference threshold, the image feature point is classified as a false reflection feature point; if the spatial propagation trajectory difference index of the feature point is lower than or equal to the difference threshold, it is classified as a true surface feature point. For example, if the spatial propagation trajectory difference index of the image feature point is 12 meters, which is greater than the difference threshold of 10 meters, the image feature point is classified as a false reflection feature point. By using the above method to classify and judge all feature points in the composite feature set, the classification results are obtained, and the corresponding classification information is marked for each feature point to determine the feature type of false reflection feature points and true surface feature points.

[0033] Remove false reflection feature points whose spatial propagation trajectory differences exceed a predetermined difference threshold; For all image feature points identified as false reflection feature points in the classification results, if the spatial propagation trajectory difference index is greater than the difference threshold, the false reflection feature points are deleted from the composite feature set and no longer participate in 3D modeling; all false reflection feature points with spatial propagation trajectory difference indices exceeding the difference threshold are removed to ensure that the remaining feature points have real and stable spatial positional relationships.

[0034] Preserve real surface feature points whose spatial propagation trajectory differences are within a predetermined difference threshold range; For all image feature points identified as true surface feature points in the classification results, if the spatial propagation trajectory difference index is lower than or equal to the difference threshold, these true surface feature points will be retained and used for 3D model reconstruction. For example, a true surface feature point with a spatial propagation trajectory difference index of 8 meters is less than the difference threshold of 10 meters, so this true surface feature point is retained; similarly, the above method is used to retain true surface feature points with a spatial propagation trajectory difference index less than or equal to the difference threshold.

[0035] Based on the retained real surface feature points, a set of effective feature points is formed; The effective feature point set refers to the new set formed after classification and judgment, eliminating false reflection feature points and retaining only true surface feature points. The effective feature point set contains only true surface feature points confirmed through spatial propagation trajectory difference analysis. Each effective feature point contains complete feature data, including its spatial three-dimensional coordinates, two-dimensional pixel coordinates, orientation, and feature description information. The effective feature point set is used for 3D model reconstruction of buildings to achieve accurate 3D modeling results unaffected by false reflections.

[0036] S6, based on the effective feature point set, performs 3D point cloud reconstruction to generate a 3D mesh model of the building that meets the surveying accuracy requirements, including: Based on the two-dimensional pixel coordinates of effective feature points in the original image sequence data and the spatial position of the UAV camera, the spatial three-dimensional coordinate data of each effective feature point is calculated to generate dense three-dimensional point cloud data containing the three-dimensional coordinate data of all effective feature points. The set of effective feature points is a collection of real surface feature points. Each effective feature point possesses two-dimensional pixel coordinates, orientation, and feature description information determined in the original image sequence data. The original image sequence data is a series of images captured by a drone using an oblique photography camera from multiple perspectives of a building, following a pre-set flight path and altitude range. Each image contains the recorded three-dimensional spatial coordinates of the drone camera at the time of capture. Calculating the spatial three-dimensional coordinates of each effective feature point involves using the two-dimensional pixel coordinates determined in the original image sequence data, combined with the three-dimensional spatial coordinates recorded by the drone camera when capturing each original image, and intrinsic and extrinsic parameters such as the camera's lens focal length and imaging parameters, to calculate the spatial three-dimensional coordinates of the effective feature points using photogrammetric intersection calculation methods. The photogrammetric intersection calculation method is as follows: First, select at least two images from different viewpoints containing the same valid feature points from the original image sequence data, and determine the two-dimensional pixel coordinates of the valid feature points in each image; based on the spatial position coordinates, shooting attitude angle, and camera focal length of the corresponding UAV camera for each image, establish a coordinate transformation relationship from the image coordinate system to the actual spatial coordinate system; based on the coordinate transformation relationship, use the two-dimensional pixel coordinates of the valid feature points in at least two images to calculate the three-dimensional spatial coordinate data of the valid feature points using the spatial forward intersection calculation algorithm; the spatial forward intersection calculation algorithm is as follows: set the spatial position coordinates of the UAV camera as (Xc, Yc, Zc), the two-dimensional pixel coordinates of the valid feature points in the image as (x, y), and the focal length of the UAV camera as f, then the valid feature points... The three-dimensional spatial coordinates (X, Y, Z) can be determined through the following calculation steps: First, the position of the feature point in the camera coordinate system is determined based on the two-dimensional coordinates (x, y) of the image and the camera focal length f. Specifically, the coordinates along the x-axis are (x·d), the coordinates along the y-axis are (y·d), and the coordinates along the z-axis are the focal length f, where d represents the actual size of a single pixel. Then, based on the camera shooting posture information and spatial position coordinates, the feature point coordinates in the camera coordinate system are transformed into three-dimensional coordinates in the actual spatial coordinate system using a three-dimensional rotation and translation matrix, thereby obtaining the three-dimensional coordinate data of each effective feature point. This process is repeated for all effective feature points one by one, ultimately obtaining a set of three-dimensional spatial coordinate data for all effective feature points. The three-dimensional spatial coordinate data of all effective feature points are then combined to form dense three-dimensional point cloud data. Dense three-dimensional point cloud data is a collection of point data composed of the three-dimensional spatial coordinates of a large number of effective feature points. Each point has spatial position data. For example, the dense three-dimensional point cloud data obtained from the above method for the exterior of a building may contain tens of thousands or even more effective feature point coordinates, fully reflecting the precise geometric shape and spatial distribution characteristics of the building's exterior.

[0037] Based on dense 3D point cloud data, a spatial triangular mesh partitioning method is used to construct triangular mesh topological connection relationships between adjacent effective feature points; Dense 3D point cloud data is a dataset containing the 3D coordinate information of all effective feature points of a building. The spatial triangulation method employed is the Delaunay triangulation method, which analyzes the spatial adjacency of effective feature points in the dense 3D point cloud data. By determining the distance and spatial adjacency relationships between each effective feature point, the topological connection relationship of the triangular mesh between adjacent effective feature points is constructed. Specifically: First, the nearest points in space for each effective feature point are determined, i.e., the spatial distance between each effective feature point and its adjacent feature points is calculated. The corresponding coordinate components of the spatial 3D coordinates of two effective feature points are subtracted to obtain the coordinate differences in the X, Y, and Z axes. The squares of each difference are then summed and the square root is taken to obtain the spatial distance between the two effective feature points. Distance; using the above methods, determine multiple neighboring valid feature points for each valid feature point; based on the Delaunay triangulation method, specifically the criterion that no other valid feature point lies inside the circumsphere of the triangular mesh, determine the triangular topological relationship between adjacent valid feature points. That is, if three valid feature points satisfy the above circumsphere criterion, then the three valid feature points form a spatial triangular mesh topological connection relationship; using the above methods, gradually expand and confirm the topological connection relationship between all valid feature points until a triangular mesh topological structure network covering all valid feature points is formed; finally, a complete topological network composed of spatial triangular meshes between valid feature points is obtained.

[0038] Based on the three-dimensional coordinate data corresponding to the effective feature points in the dense three-dimensional point cloud data, the spatial position and geometric shape of each triangular mesh are determined, and a three-dimensional mesh model of the building that meets the surveying accuracy requirements is generated. The three-dimensional coordinates of effective feature points in dense three-dimensional point cloud data are obtained through photogrammetric intersection calculation. Based on the spatial triangular mesh topology network, the spatial position and geometry of each triangular mesh are determined using the spatial three-dimensional coordinate data of effective feature points. Specifically, the spatial three-dimensional coordinates of the three effective feature points contained in each triangular mesh are used as vertex coordinates, and the spatial position and geometry of the triangular mesh are formed by connecting the coordinates of the three vertices. For example, if a triangular mesh is composed of three vertices with effective feature point spatial coordinates (100m, 200m, 50m), (105m, 205m, 55m), and (102m, 203m, 53m), then the triangular mesh is the triangular surface formed by directly connecting these three points in space. The spatial position and geometry of all triangular meshes are determined using the above method, and a complete three-dimensional surface mesh model is constructed based on the spatial position and geometry information. The three-dimensional mesh model is a complete spatial mesh structure composed of all connected triangular meshes, reflecting the actual spatial position and geometric features of various parts of the building surface. The three-dimensional mesh model of the building generated by the above method can fully meet the requirements of surveying accuracy and ensure the accuracy and authenticity of the building's real structure and spatial position.

[0039] Example 2 The difference between Embodiment 2 and Embodiment 1 is that this embodiment introduces a UAV-based oblique photogrammetry 3D modeling system.

[0040] Figure 2 A schematic diagram of the UAV-based oblique photogrammetry 3D modeling system of the present invention is provided. The UAV-based oblique photogrammetry 3D modeling system includes: Feature point extraction module: acquires the original image sequence data obtained by UAV oblique photography, extracts the feature point information of the overlapping area of ​​adjacent images in the original image sequence data, and outputs the initial image feature point set; Feature classification module: Based on the initial image feature point set, extract the real surface feature points of direct incident light imaging and the false reflection feature points caused by dynamic mutual reflection of multiple mirrors to generate a composite feature set; The ghost image analysis module analyzes the spatial propagation trajectory characteristics of false reflection feature points in the composite feature set by utilizing the difference between the incident angle of light and the reflection propagation path, and generates reflection ghost image trajectory data. Feature matching module: Based on spatial topological constraints and texture continuity feature analysis, it generates real feature point matching constraint data for real surface feature points in the composite feature set. The ghost image removal module identifies and removes false reflection feature points from the composite feature set based on the matching constraint data between the reflection ghost image trajectory data and the real feature points, thereby obtaining a set of effective feature points. 3D Reconstruction Module: Reconstructs 3D point clouds based on a set of effective feature points to generate 3D mesh models of buildings that meet the accuracy requirements of surveying and mapping.

[0041] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. The semiconductor medium can be a solid-state drive.

[0042] Those skilled in the art will recognize that the modules and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0043] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and modules described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0044] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or modules may be electrical, mechanical, or other forms.

[0045] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0046] In addition, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module.

[0047] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0048] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0049] In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A UAV-based oblique photogrammetry 3D modeling method, characterized in that, Includes the following steps: S1: Acquire the raw image sequence data obtained by the UAV during oblique photography, extract the feature point information of the overlapping area of ​​adjacent images in the raw image sequence data, and output the initial image feature point set; S2: Based on the initial image feature point set, extract the real surface feature points of direct incident light imaging and the false reflection feature points caused by dynamic mutual reflection of multiple mirrors to generate a composite feature set; S3: For false reflection feature points in the composite feature set, analyze the spatial propagation trajectory characteristics by utilizing the difference between the incident angle of light and the reflection propagation path, and generate reflection phantom trajectory data; S4: For real surface feature points in the composite feature set, generate real feature point matching constraint data based on spatial topological constraints and texture continuity feature analysis; S5: Based on the reflection phantom trajectory data and the matching constraint data of real feature points, identify and eliminate false reflection feature points in the composite feature set to obtain the effective feature point set; S6: Reconstruct 3D point clouds based on the effective feature point set to generate a 3D mesh model of a building that meets the accuracy requirements of surveying and mapping.

2. The UAV-based oblique photogrammetry 3D modeling method according to claim 1, characterized in that, S1 specifically refers to: By controlling the drone to sequentially collect multi-view building images within a preset flight path and altitude range, the original image sequence data is obtained; Spatial alignment is performed on the original image sequence data to identify the spatial overlap region between each group of adjacent images; Image feature points are extracted within spatially overlapping regions. The pixel coordinates, orientation, and feature description information of each feature point in the original image are obtained, and an initial image feature point set containing feature point information of all spatially overlapping regions is constructed.

3. The UAV-based oblique photogrammetry 3D modeling method according to claim 2, characterized in that, S2 specifically refers to: Based on the initial set of image feature points, and combined with the geometric relationship between the spatial position of the UAV camera, the lens viewpoint, and the normal direction of the building surface, the light propagation path of the image feature points is determined; Image feature points that meet the condition of directly reaching the drone camera lens from the building surface are determined to be real surface feature points; Image feature points that meet the condition that light reaches the drone camera lens after being reflected twice or multiple times by the glass curtain wall of a building are identified as false reflection feature points. A composite feature set is formed based on real surface feature points and false reflection feature points.

4. The UAV-based oblique photogrammetry 3D modeling method according to claim 3, characterized in that, S3 specifically refers to: Based on the false reflection feature points in the composite feature set, and combined with the spatial position coordinates and lens view of the UAV camera, the incident angle and reflection angle of the incident and reflected light rays on the glass curtain wall surface of the building corresponding to each false reflection feature point are determined. Based on the incident angle and reflection angle of the light, the spatial trajectory path of the light propagation at each false reflection feature point is calculated and tracked to obtain the spatial trajectory path data of all false reflection feature points. By combining spatial trajectory path data, the similarity characteristics of false reflection feature points on the spatial propagation path are determined, and reflection ghost trajectory data is generated.

5. The UAV-based oblique photogrammetry 3D modeling method according to claim 4, characterized in that, S4 specifically refers to: Based on the real surface feature points in the composite feature set, a spatial adjacency relationship is constructed, and the three-dimensional geometric distance and two-dimensional pixel distance between each real surface feature point and its nearest neighbor real surface feature point are calculated to form a spatial adjacency matrix. For the corresponding pairs of real surface feature points in the spatial adjacency matrix, extract the texture blocks from their respective original images, calculate the texture similarity, and generate a texture similarity matrix. The spatial adjacency matrix and texture similarity matrix are jointly thresholded and filtered, and a true feature point matching map is constructed. Output the true feature point matching constraint data based on the true feature point matching map.

6. The UAV-based oblique photogrammetry 3D modeling method according to claim 5, characterized in that, S5 specifically refers to: Based on the matching constraint data of the reflected phantom trajectory data and the real feature points, the spatial propagation trajectory differences between the false reflected feature points and the real surface feature points are determined. Based on the differences in spatial propagation trajectories, feature type classification is performed on all image feature points in the composite feature set. Remove false reflection feature points whose spatial propagation trajectory differences exceed a predetermined difference threshold; Preserve real surface feature points whose spatial propagation trajectory differences are within a predetermined difference threshold range; Based on the retained real surface feature points, a set of effective feature points is formed.

7. The UAV-based oblique photogrammetry 3D modeling method according to claim 6, characterized in that, S6 specifically refers to: Based on the two-dimensional pixel coordinates of effective feature points in the original image sequence data and the spatial position of the UAV camera, the spatial three-dimensional coordinate data of each effective feature point is calculated to generate dense three-dimensional point cloud data containing the three-dimensional coordinate data of all effective feature points. Based on dense 3D point cloud data, a spatial triangular mesh partitioning method is used to construct triangular mesh topological connection relationships between adjacent effective feature points; Based on the 3D coordinate data corresponding to the effective feature points in the dense 3D point cloud data, the spatial position and geometry of each triangular mesh are determined, and a 3D mesh model of the building that meets the surveying accuracy requirements is generated.

8. A UAV-based oblique photogrammetry 3D modeling system, used to implement the UAV-based oblique photogrammetry 3D modeling method according to any one of claims 1-7, characterized in that, include: Feature point extraction module: acquires the original image sequence data obtained by UAV oblique photography, extracts the feature point information of the overlapping area of ​​adjacent images in the original image sequence data, and outputs the initial image feature point set; Feature classification module: Based on the initial image feature point set, extract the real surface feature points of direct incident light imaging and the false reflection feature points caused by dynamic mutual reflection of multiple mirrors to generate a composite feature set; The ghost image analysis module analyzes the spatial propagation trajectory characteristics of false reflection feature points in the composite feature set by utilizing the difference between the incident angle of light and the reflection propagation path, and generates reflection ghost image trajectory data. Feature matching module: Based on spatial topological constraints and texture continuity feature analysis, it generates real feature point matching constraint data for real surface feature points in the composite feature set. The ghost image removal module identifies and removes false reflection feature points from the composite feature set based on the reflection ghost image trajectory data and the matching constraint data of real feature points, thereby obtaining a set of effective feature points. 3D Reconstruction Module: Reconstructs 3D point clouds based on a set of effective feature points to generate 3D mesh models of buildings that meet the accuracy requirements of surveying and mapping.

Images