Urban building three-dimensional model refined construction method based on oblique photography

Abstract

The invention belongs to the technical field of three-dimensional modeling, and discloses a city building three-dimensional model refined construction method based on oblique photography. Comprising the steps of collecting inclined image data of a to-be-modeled urban area and performing data cleaning to obtain a modeling image data set; carrying out projection consistency analysis, identifying a boundary dislocation region and removing the boundary dislocation region to obtain a dislocation removal data set; shielding area identification is carried out, shielding compensation is carried out based on an identification result, and an enhanced three-dimensional point data set is output; executing detail reservation to obtain a triangular facet data set; normal optimization is carried out, plane adjustment is carried out on the triangular surface data set based on an optimization result, and a three-dimensional grid data set is obtained; constructing an initial three-dimensional model, and performing map mapping on the initial three-dimensional model to obtain a detail map set; constructing a fine building three-dimensional model in combination with the detail map set and the initial three-dimensional model; and the boundary definition, the structural integrity and the texture fineness of the urban building three-dimensional model are improved.

Description

Technical Field

[0001] This invention relates to the field of 3D modeling technology, and more specifically, to a method for refining 3D models of urban buildings based on oblique photogrammetry. Background Technology

[0002] Oblique photogrammetry is a novel aerial photogrammetry technique that can acquire complete texture information of building tops and facades. Because it can be combined with modeling techniques to generate high-precision 3D building models with realistic textures, it has been widely used in many fields. However, traditional model building methods can only achieve basic 3D model building functions, and are often limited when facing complex urban building scenes, and still face many technical bottlenecks. In the multi-view image fusion stage of urban architecture, oblique images from different shooting angles are spatially aligned through feature matching. For example, the east-facing facade is matched using SIFT features, while the south-facing facade is extracted using the SURF operator. However, traditional model building methods only focus on the number of feature points and ignore projection consistency analysis, which can easily lead to the accumulation of matching errors and the formation of jagged edges in the spliced ​​contours. In addition, facade components such as balcony railings in urban architecture often exhibit occlusion differences from different perspectives. For example, railings are clearly visible in the front view but are occluded by balcony slabs in the oblique view. However, traditional model building methods directly fuse point clouds in these areas with inconsistent occlusion, resulting in broken, intersecting, or deformed railing reconstructions that fail to restore the true facade component structure. At the same time, the roof areas of urban buildings often have parapets. For components such as walls, skylights, and equipment platforms, traditional modeling methods tend to group areas with minimal undulation into flat planes during the triangular mesh simplification process. This can easily lead to the incorrect filtering of detailed roof components, resulting in distorted flat roof models that compromise the structural integrity of the model. Furthermore, when multi-view images are stitched together to generate triangular meshes, differences in normal vector directions can occur. Traditional modeling methods estimate the normal vectors of triangular faces based on only a single viewpoint without considering multi-view fusion, leading to significant jumps in the normal vector directions of adjacent triangular faces. Moreover, some urban building facades often feature decorative details such as reliefs or moldings. Traditional modeling methods do not optimize texture mapping for high-frequency detail areas, resulting in blurred decorative textures or skewed UV unwrapping directions, affecting the refined representation of urban building 3D models.

[0003] In view of this, the present invention proposes a method for refining the 3D model of urban buildings based on oblique photography to solve the above problems. Summary of the Invention

[0004] To overcome the aforementioned shortcomings of the prior art and to achieve the above objectives, the present invention provides the following technical solution: a method for refining the 3D model of urban buildings based on oblique photogrammetry, comprising: S1. Collect oblique image data of the urban area to be modeled and perform data cleaning to obtain the modeling image dataset; S2. Perform projection consistency analysis on the modeling image dataset to identify boundary misalignment regions and remove them from the modeling image dataset to obtain the misalignment removal dataset; S3. Identify occlusion regions in the misaligned culling dataset, compensate for occlusion based on the identification results, and output an enhanced 3D point dataset. S4. Perform detail preservation on the enhanced 3D point dataset to obtain the triangular face dataset; S5. Perform normal optimization on the triangular face dataset, and adjust the triangular face dataset in plane based on the optimization results to obtain a three-dimensional mesh dataset; S6. Construct an initial 3D model based on a 3D mesh dataset, and perform texture mapping on the initial 3D model to obtain a detail texture set; combine the detail texture set and the initial 3D model to construct a detailed architectural 3D model.

[0005] Furthermore, the method for performing projection consistency analysis includes: Extract multi-view image data belonging to the same building from the modeling image dataset, extract feature points for each image in the multi-view image data, and associate and match the feature points of each image with the feature points of other images to output a set of feature point pairs. Calculate the fundamental matrix between the image pairs corresponding to each feature point pair, and derive the projection matrix between the corresponding image pairs based on the fundamental matrix; Identify the common viewing region of each image pair, and transform the common viewing region of the corresponding image pair to a unified reference coordinate plane based on the projection matrix to obtain the common viewing image block; Extract the overlapping regions between co-image blocks and calculate the gradient direction difference of the overlapping regions. Draw the gradient direction map between the corresponding image blocks based on the gradient direction difference. If there are pixels in the gradient direction map whose gradient direction difference is greater than a preset direction threshold, and the number of such pixels exceeds a preset pixel ratio, then the region corresponding to the image block is determined to be a boundary misalignment region.

[0006] Furthermore, the method for identifying the occlusion area includes: Filter out misaligned facade images in the dataset that cover the same building facade, and identify the building facade component areas in the facade image set. Obtain the shooting posture information for each facade image, and calculate the angle between the shooting direction and the viewing angle of the building facade in the corresponding facade image based on the shooting posture information; Edge extraction is performed on the building facade component area, and the outline pixel chain of the facade component is output; the edge density and edge connectivity coefficient of the outline pixel chain of the facade component are calculated; at the same time, the texture dispersion and orientation consistency coefficient of the building facade component area are calculated based on the gray value of the building facade component area; the visibility score of the corresponding building facade component area is calculated based on the edge density, edge connectivity coefficient, texture dispersion and orientation consistency coefficient. Filter facade images in the facade image set that contain the same building facade component area, and extract facade images with visibility scores lower than the fitted visibility score threshold as low-score images; If the viewing angle of the low-resolution image is greater than a preset angle threshold, the building facade component area corresponding to the low-resolution image is determined to be an occluded area.

[0007] Furthermore, the method for performing occlusion compensation includes: Extract the corresponding building facade component areas that are not marked as occlusion areas in the misalignment removal dataset and use them as compensation reference areas; Extract the edge coordinate point set of the compensation reference area and project it into 3D space to output the preliminary point cloud completion point set of the corresponding occluded area; Regions where the spacing between edge coordinates in the preliminary point cloud completion point set is greater than a preset spacing threshold are identified as gap regions; edge reconstruction fitting is performed on the gap regions to output an enhanced point cloud completion point set; The original misalignment removal dataset is transformed into a spatial coordinate point set, and the enhanced point cloud completion point set is merged with it to obtain an enhanced 3D point dataset.

[0008] Furthermore, the methods for preserving the execution details include: Extract a subset of 3D point coordinates corresponding to the building roof area from the enhanced 3D point dataset, and calculate the rate of curvature change of each 3D point coordinate in the building roof area based on this subset of 3D point coordinates. Spatial clustering is performed on the corresponding 3D point coordinate subsets based on the rate of curvature change to obtain a curvature feature point subset; a curvature feature point subset in which the rate of curvature change of all curvature feature points in the point set is higher than the preset curvature threshold is selected, and it is determined whether the curvature feature points in the curvature feature point subset form a closed region. If a closed region is formed, the closed region is marked as a detail component region. Set detailed scale modeling parameters, and perform triangular meshing on the detailed component region based on these detailed scale modeling parameters to obtain the meshed component region; Transition triangles are generated at the junctions between the mesh component area and the remaining areas of the building roof area, and boundary stitching is performed. At the same time, labels are added to maintain the topological separation between the mesh component area and the remaining areas. Subset the 3D point coordinates of all stitched building roof areas into a triangular facet dataset.

[0009] Furthermore, the method for performing normal optimization includes: Extract the normal vector of each triangle in the triangle face dataset, and calculate the angle between the normal vectors of each pair of adjacent triangles sharing the same edge. If the angle between the normal vectors exceeds the preset angle jump threshold, the region formed by the pair of adjacent triangles sharing the same edge is determined as an abnormal normal transition region. The centroid of each triangular face in the abnormal transition region of the normal is identified. Based on the centroid, the multi-view local region image corresponding to the triangular face is matched, and the projection position of the centroid in the local region image of each view is calculated. Calculate the gray-level gradient direction of the local region image for each viewpoint, and calculate the normal estimation vector at the projection position based on the shooting direction and gray-level gradient direction under the corresponding viewpoint. Combine the estimated normal vectors of all viewpoints into a set of candidate normal vectors for the corresponding triangular face, calculate the angle between each candidate normal vector and the original normal vector, and select the candidate normal vector with the smallest angle and the smallest angle with the shooting direction as the target normal vector. Replace the normal vector of the corresponding triangle with the target normal vector, and perform normal direction smoothing on another adjacent triangle that is in the abnormal normal transition region; use the normal vectors of all updated triangles as the optimization result.

[0010] Furthermore, the method of performing planar adjustment includes: Based on the updated normal vectors of the triangles, the vertex positions between adjacent triangles are adjusted to ensure that the normal directions of adjacent triangles are continuous and the face boundaries are flat, and a 3D mesh dataset is output.

[0011] Furthermore, the method for performing texture mapping includes: Identify the local region image corresponding to each triangle face in the initial 3D model, project the triangle face onto each corresponding local region image, and calculate the pixel ratio of the projected region in each local region image and the gray-level gradient intensity of the projected region. The texture image score is calculated based on pixel ratio and grayscale gradient intensity, and the local region image with the highest texture image score is selected as the main texture image. Extract the image texture from the main texture image and generate texture coordinate expansion results to associate and map them with the corresponding triangle faces; Identify the triangular facets belonging to the exterior facade detail component area, perform local texture magnification on the triangular facets and adjust the texture resolution to ensure that the texture clarity is not lower than the preset clarity threshold; Output all adjusted triangular surface texture data as a detail map set.

[0012] Furthermore, the method for constructing a detailed 3D architectural model includes: The surface texture data of each triangle in the detail map set is bound to the corresponding triangle in the initial 3D model; all bound triangles are integrated to obtain a detailed architectural 3D model.

[0013] A system for refining 3D urban building models based on oblique photogrammetry, used to implement a method for refining 3D urban building models based on oblique photogrammetry, is characterized by including: The data acquisition module collects oblique image data of the urban area to be modeled and performs data cleaning to obtain the modeling image dataset; The misalignment removal module performs projection consistency analysis on the modeling image dataset, identifies boundary misalignment regions and removes them from the modeling image dataset to obtain the misalignment removal dataset. The occlusion enhancement module identifies occlusion regions in the misaligned culling dataset, compensates for occlusion based on the identification results, and outputs an enhanced 3D point dataset. The detail preservation module performs detail preservation on the enhanced 3D point dataset to obtain a triangular face dataset. The normal adjustment module optimizes the normals of the triangular face dataset and performs planar adjustment on the triangular face dataset based on the optimization results to obtain a 3D mesh dataset. The modeling and construction module constructs an initial 3D model based on a 3D mesh dataset and performs texture mapping on the initial 3D model to obtain a detail texture set; it then combines the detail texture set and the initial 3D model to construct a detailed architectural 3D model; the various modules are connected via wired and / or wireless means.

[0014] The technical effects and advantages of this invention's method for refining 3D urban building models based on oblique photogrammetry are as follows: By collecting oblique imagery data of the urban area to be modeled and cleaning the data, a modeling image dataset was obtained. Based on this modeling image dataset, projection consistency analysis and misalignment removal, occlusion area identification and compensation, detail preservation, normal optimization, and texture mapping were successively performed to achieve refined construction of urban building 3D models based on oblique photogrammetry. Compared with existing experience, the removal of boundary misalignment areas reduced the accumulation of contour noise caused by multi-view matching errors. By determining whether the building facade component areas belong to occlusion areas and performing point cloud compensation, the fracture deformation of facade components such as balcony railings caused by inconsistent occlusion was avoided. By performing mesh generation on the detailed component areas, the detailed components of the roof area were preserved from being simplified and filtered by triangular faces. By selecting target normal vectors and performing normal direction smoothing, the fracture phenomenon caused by the normal jump of adjacent triangular faces was eliminated. By selecting the main texture image and performing local texture magnification on the exterior facade detailed component areas, the texture clarity and texture alignment accuracy of the exterior facade decorative details were ensured. The boundary clarity, structural integrity, and texture refinement of the urban building 3D model were improved. Attached Figure Description

[0015] Figure 1 This is a schematic diagram of the method for refining urban building 3D models based on oblique photography according to the present invention; Figure 2 This is a schematic diagram of the urban building 3D model refinement system based on oblique photography according to the present invention. Detailed Implementation

[0016] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0017] Example 1 Please see Figure 1 As shown in this embodiment, the method for refining the 3D model of urban buildings based on oblique photogrammetry includes: S1. Collect oblique image data of the urban area to be modeled and perform data cleaning to obtain the modeling image dataset; S2. Perform projection consistency analysis on the modeling image dataset to identify boundary misalignment regions and remove them from the modeling image dataset to obtain the misalignment removal dataset; S3. Identify occlusion regions in the misaligned culling dataset, compensate for occlusion based on the identification results, and output an enhanced 3D point dataset. S4. Perform detail preservation on the enhanced 3D point dataset to obtain the triangular face dataset; S5. Perform normal optimization on the triangular face dataset, and adjust the triangular face dataset in plane based on the optimization results to obtain a three-dimensional mesh dataset; S6. Construct an initial 3D model based on a 3D mesh dataset, and perform texture mapping on the initial 3D model to obtain a detail texture set; combine the detail texture set and the initial 3D model to construct a detailed architectural 3D model.

[0018] In this embodiment, the oblique image data refers to oblique images of buildings in a certain urban area that need to be modeled, taken using oblique photography technology from different perspectives, used to cover the facade structure and roof area of ​​urban buildings; wherein image distortion correction is performed on these oblique image data to achieve data cleaning, resulting in a higher quality modeling image dataset.

[0019] Methods for performing projection consistency analysis include: Extract multi-view image data belonging to the same building from the modeling image dataset, extract feature points for each image in the multi-view image data, and associate and match the feature points of each image with the feature points of other images to output a set of feature point pairs.

[0020] This involves extracting multi-view image data from various perspectives of the same target building from the modeling image dataset, and in this embodiment, using operators such as SIFT or SURF to detect feature points in each image, including corner points and edge intersections, etc.; by constructing feature point pairs from each image and feature points in other images that are in the same physical location within the same building as the feature point, all feature point pairs are integrated to obtain a set of feature point pairs.

[0021] The fundamental matrix between the image pairs corresponding to each feature point pair is calculated, and the projection matrix between the corresponding image pairs is derived based on the fundamental matrix. The fundamental matrix is ​​used to describe the polar geometric relationship between two images. In this embodiment, the fundamental matrix between the two images is constructed using the eight-point algorithm. The projection matrix of the matrix is ​​constructed by rotating and translating the fundamental matrix.

[0022] The common viewing region of each image pair is identified, and the common viewing region of the corresponding image pair is transformed to a unified reference coordinate plane based on the projection matrix to obtain a common viewing image block. The common viewing region refers to the image overlap area in which two images in each image pair simultaneously cover the spatial area of ​​the same building. By using the projection matrix as the projection reference of the common viewing region, the common viewing region is mapped to the set unified reference coordinate plane through perspective transformation to obtain the common viewing image block of the region, thus realizing the spatial alignment of the same region in different images.

[0023] The overlapping regions between co-view image blocks are extracted, and the gradient direction difference of the overlapping regions is calculated. Based on the gradient direction difference, the gradient direction map between the corresponding image blocks is drawn. Specifically, by extracting the overlapping regions of the projection areas in multiple co-view image blocks and performing gradient direction calculation, the gradient direction difference of each corresponding pixel is obtained and drawn into a gradient direction map. This gradient direction map is used to reflect whether the edge representation of the same building area is consistent in different images, and is used to identify splicing errors and direction mismatches.

[0024] If there are pixels in the gradient direction map whose gradient direction difference is greater than a preset direction threshold, and the number of such pixels exceeds a preset pixel ratio, then the region corresponding to the image block is determined to be a boundary misalignment region. The preset direction threshold for the gradient direction difference and the preset pixel ratio for the number of pixels are set based on the relevant theoretical knowledge of oblique photography and the known relevant information of the corresponding building. When there are pixels in the gradient direction map of the overlapping area of ​​the image pair whose number exceeds the preset pixel ratio and has a direction deviation greater than the preset direction threshold, it indicates that there is a difference in edge position or splicing misalignment between the corresponding image pairs. Therefore, the corresponding region is determined to be a boundary misalignment region.

[0025] Methods for identifying occluded areas include: The process involves filtering out facade images from the misalignment removal dataset that cover the same building facade, and identifying the building facade component regions within these images. This is achieved by extracting images from the misalignment removal dataset that share the same target building and whose coverage areas contain the same target building facade. The building facade component regions refer to areas on urban building facades where protruding structures such as railings, balconies, and bay windows are located. These component regions are identified based on the contour information of these structures.

[0026] The shooting posture information of each facade image is obtained. Based on the shooting posture information, the angle between the shooting direction and the viewing angle of the building facade in the corresponding facade image is calculated. The shooting posture information refers to the direction vector of the camera device and related shooting parameters when the image is taken. The viewing angle is obtained by calculating the angle between the direction vector of the shooting direction and the building facade. This angle is used to reflect the viewing angle of the building facade.

[0027] Edge extraction is performed on the building facade component area, and the outline pixel chain of the facade component is output. In this embodiment, the pixel chain composed of the specific outline pixels of the building facade component area is extracted by the edge detection algorithm as the data basis for subsequent operations.

[0028] Calculate the edge density and edge connectivity coefficient of the pixel chain of the facade component outline. Edge density refers to the ratio of the number of edge pixels to the area formed by the pixel chain of the facade component outline, representing the density of line distribution in that area. The formula for calculating the edge connectivity coefficient is: ;in Represents the edge connectivity coefficient; This indicates the number of edge pixels in the region corresponding to the pixel chain of the facade component outline; It refers to the number of connected segments obtained by performing connected component detection on the corresponding region of the pixel chain of the facade component outline; the larger the value of the edge connectivity coefficient, the more concentrated and coherent the edges are, and vice versa, it indicates that there are broken edges or skipped edges, and the structural integrity is poor.

[0029] Simultaneously, based on the grayscale values ​​of the building facade component area, the texture dispersion and orientation consistency coefficient of the building facade component area are calculated. The texture dispersion refers to the statistical variance of the grayscale values ​​corresponding to the building facade component area. The larger the value, the richer the texture, and vice versa. The orientation consistency coefficient is the mean of the ratio of the main direction of the corresponding building facade component area extracted by the edge detection operator to the gradient direction of all pixels. This value is used to represent the distribution of texture orientation.

[0030] The visibility score of the corresponding building facade component area is calculated based on edge density, edge connectivity coefficient, texture dispersion, and orientation consistency coefficient. The visibility score of the corresponding building facade component area is obtained by weighted summation of edge density, edge connectivity coefficient, texture dispersion, and orientation consistency coefficient. The weights used in the weighting are set based on historical experience in identifying occluded areas.

[0031] Facade images containing the same building facade component area are selected from the facade image set. Facade images with visibility scores lower than the fitted visibility score threshold are extracted as low-scoring images. In this embodiment, for each building facade component area, the visibility score distribution trend curve of the area is fitted using a fitting algorithm in the facade images of the same group, and a score that can represent the average performance level is extracted from it as the fitted visibility score threshold. If the visibility score of a facade image is significantly lower than the threshold, the corresponding facade image is regarded as a low-scoring image.

[0032] If the viewing angle of a low-resolution image is greater than a preset angle threshold, the building facade component area corresponding to the low-resolution image is determined as an occluded area. The preset angle threshold is set based on the relevant theoretical knowledge of oblique photography. When there is a low-resolution image with a low visibility score, and the angle between its corresponding shooting direction and the building facade is greater than the preset angle threshold, it can be considered that the building facade component area in the low-resolution image is occluded or blurred under the current shooting direction, and therefore the area is determined as an occluded area.

[0033] Methods for occlusion compensation include: The corresponding building facade component areas that are not marked as occluded areas in the misalignment culling dataset are extracted as compensation reference areas. Specifically, the building facade component areas corresponding to the occluded areas in the misalignment culling dataset that are in good viewpoints in other images and are not blurred due to occlusion are extracted as compensation reference areas, which serve as the data basis for subsequent point cloud compensation.

[0034] Extract the edge coordinate point set of the compensation reference area and project it into three-dimensional space. Output the preliminary point cloud completion point set of the corresponding occluded area. The edge pixel coordinate points of the building facade components are extracted from the compensation reference area to form the edge coordinate point set. Combine the corresponding building information to calculate the three-dimensional spatial position of these coordinates and project them into three-dimensional space. The projected three-dimensional coordinate points are used as the preliminary point cloud completion point set.

[0035] Regions where the distance between edge coordinates in the initial point cloud completion point set is greater than a preset distance threshold are identified as gap regions. The preset distance threshold is set based on occlusion compensation experience. If the distance between several consecutive edge coordinates in a certain region exceeds the preset distance threshold, it is considered that there is a gap in the projected 3D space at that location.

[0036] Edge reconstruction fitting is performed on the gap regions to output an enhanced point cloud complete point set. In this embodiment, the fitting algorithm is used to fill in the missing structural points in these gap regions, thereby forming the completed enhanced point cloud complete point set.

[0037] The original misalignment culling dataset is transformed into a spatial coordinate point set, and the augmented point cloud completion point set is merged with it to obtain an augmented 3D point dataset. The image data in the misalignment culling dataset is represented using spatial coordinate points and reconstructed into a dense point cloud. The augmented point cloud completion point set is spatially stitched and fused with this dense point cloud to form an augmented 3D point dataset that includes the construction of an occlusion compensation structure.

[0038] Methods for retaining execution details include: Extract a subset of 3D point coordinates corresponding to the roof area of ​​buildings in the enhanced 3D point dataset. Calculate the rate of curvature change of each 3D point coordinate in the roof area based on this subset. The subset of 3D point coordinates refers to the point cloud data corresponding to the roof area of ​​urban buildings in the enhanced 3D point dataset. Calculate the rate of curvature change for each 3D point coordinate in the subset. This value reflects the degree of undulation of the corresponding point in the roof area structure.

[0039] Based on the rate of curvature change, spatial clustering is performed on the corresponding subset of three-dimensional point coordinates to obtain a subset of curvature feature points. In this embodiment, the spatial clustering algorithm is used to cluster the three-dimensional point coordinates whose rate of curvature change belongs to different curvature threshold intervals to obtain several subsets of curvature feature points. The rate of curvature change of the three-dimensional point coordinates in each subset of curvature feature points belongs to the same curvature threshold interval.

[0040] Select a subset of curvature feature points in the set of curvature feature points whose rate of curvature change is higher than the preset curvature threshold, and determine whether the curvature feature points in the subset of curvature feature points form a closed region. If a closed region is formed, mark the closed region as a detail component region.

[0041] Based on the theoretical knowledge of oblique photogrammetry, a preset curvature threshold is set. A subset of feature points in the curvature feature point subset is selected, and the curvature change rate of all curvature feature points in the subset is higher than the curvature threshold. Furthermore, a boundary contour tracking algorithm is used to detect whether the curvature feature points in the subset are closed. If a closed region is formed, the region is identified as a detailed component region, such as a small skylight or a parapet wall, etc.

[0042] Set detailed scale modeling parameters, and perform triangular meshing on the detailed component region based on these detailed scale modeling parameters to obtain the meshed component region.

[0043] In this embodiment, the detailed scale modeling parameters include more refined parameter settings such as the minimum side length limit of the triangular face, the allowable value of the face depth fitting error, and the upper limit of the rate of change of adjacent angles. Compared with general areas, detailed component areas require denser meshing and higher curvature response fitting accuracy to preserve the true geometric features of the component objects. At the same time, the triangular meshing is achieved by using a surface reconstruction algorithm to obtain several independent high-precision triangular face areas as local mesh units for restoring the real structural structure, which is the meshed component area.

[0044] Transition triangles are generated at the junctions between the mesh component area and the remaining areas of the building roof area, and boundary stitching is performed. At the same time, labels are added to maintain the topological separation between the mesh component area and the remaining areas.

[0045] The transition triangle is an interpolated surface with continuously varying size and smooth connection constructed using a set of pixels inside and outside the boundary of the mesh-divided component region. This avoids structural gaps caused by resolution differences. In this embodiment, a topology stitching algorithm is used to connect the mesh-divided component region with other adjacent regions, and a certain mark is added to the mesh-divided component region to indicate that the region cannot be simplified, thereby achieving topological separation and preventing the region from being mistakenly merged as a region that can be simplified during the modeling process.

[0046] Subset the 3D point coordinates of all stitched building roof areas into a triangular facet dataset.

[0047] Methods for performing normal optimization include: Extract the normal vector of each triangle in the triangle facet dataset, and calculate the angle between the normal vectors of each pair of adjacent triangles sharing the same edge. A triangle sharing the same edge is a triangle that shares two vertices. The angle between the normal vectors is obtained by extracting the normal vectors of the two triangles respectively and calculating the angle between them. This angle is used to reflect the degree of directional discontinuity between the spatial triangle faces.

[0048] If the angle between the normal vectors exceeds the preset angle jump threshold, the region formed by the pair of adjacent triangles sharing the same edge is determined as an abnormal normal transition region. The preset angle jump threshold is set based on the theoretical knowledge of model construction. If the angle between the normal vectors is higher than the preset angle jump threshold, the corresponding region is determined as an abnormal normal transition region, thus identifying the region with a visual sense of broken lines in the model splicing.

[0049] The centroid of each triangular face in the abnormal transition region of the normal is identified. Based on the centroid, the local region image corresponding to the triangular face is matched with the local region image of the triangular face from multiple perspectives. The projection position of the centroid in the local region image of each perspective is calculated. The centroid position of the triangular face is calculated as the typical position of the face. The centroid position is mapped to the image containing the region of the triangular face from different shooting perspectives to realize the mapping from the centroid position to the image pixel.

[0050] The gray-level gradient direction of the local region image is calculated for each viewpoint. At the projection position, the normal estimation vector under the corresponding viewpoint is calculated based on the shooting direction and the gray-level gradient direction. Specifically, by calculating the gray-level gradient direction of the local region image corresponding to the abnormal transition region of the normal under each viewpoint, the gray-level gradient directions of all pixels are aggregated to the projection position to form the gradient principal direction vector. The normal estimation vector under that viewpoint is obtained by performing a vector cross product between this vector and the direction vector corresponding to the shooting direction.

[0051] Combine the estimated normal vectors of all viewpoints into a set of candidate normal vectors for the corresponding triangular face. Calculate the angle between each candidate normal vector and the original normal vector. Select the candidate normal vector with the smallest angle and the smallest angle with the shooting direction as the target normal vector.

[0052] The estimated normal vectors of all viewpoints belonging to the corresponding abnormal transition region are combined into a set of candidate normal vectors. The angle between each candidate normal vector and the normal vector of the corresponding triangular face is calculated. The candidate normal vector with the smallest angle and the one that best matches the shooting direction is selected as the target normal vector. This normal vector also takes into account the degree of matching between the surface normal and the real visible direction.

[0053] The normal vector of the corresponding triangle is replaced by the target normal vector, and the normal direction smoothing is performed on another adjacent triangle in the normal anomalous transition region. The original normal vector of the corresponding triangle is replaced by the target normal vector, and the normal vector of the adjacent triangle in the normal anomalous transition region is updated by a linear weighted algorithm, so that the region presents a smoother visual change.

[0054] The normal vectors of all updated triangle faces are used as the optimization result.

[0055] Methods for adjusting a plane include: Based on the updated normal vectors of the triangular faces, the vertex positions between adjacent triangular faces are adjusted to ensure the continuity of the normal directions of adjacent triangular faces and the flatness of the face boundaries. A 3D mesh dataset is output, in which the shared vertices of triangular faces with the same edge are used as adjustable vertices. With the minimum angle between the normal vectors as the constraint condition, the component can ensure that the new vertex positions of the normal vector regions of each pair of adjacent triangular faces are consistent. At the same time, compression or stretching operations are performed to adjust the curvature of the boundary, enhance the continuity between the boundaries, and make it visually flatter.

[0056] Methods for performing texture mapping include: Identify the local region image corresponding to each triangle face in the initial 3D model, project the triangle face onto each corresponding local region image, and calculate the pixel ratio of the projected region in each local region image and the gray-level gradient intensity of the projected region.

[0057] This process involves projecting the spatial coordinates of each triangular facet onto the corresponding two-dimensional pixel position in each local region image containing that triangular facet; calculating the ratio of the number of pixels occupied by the triangular facet after projection to the total number of pixels in the image to obtain the pixel ratio, which reflects the accuracy of the image's ability to represent the model facet; extracting the gray-level gradient within the projection region and calculating the mean gray-level gradient of that region to obtain the gray-level gradient intensity.

[0058] The texture image score is calculated based on pixel ratio and grayscale gradient intensity, and the local region image with the highest texture image score is selected as the main texture image.

[0059] The texture image score is obtained by weighted summation of pixel ratio and grayscale gradient intensity. It should be noted that the pixel ratio and grayscale gradient intensity have been normalized to the same order of magnitude before the calculation. The calculated texture image score is used to represent the clarity of texture and the consistency of direction in the image. Therefore, the local area image with the highest texture image score is selected as the main texture image to avoid problems such as texture blurring caused by arbitrarily assigning images in the UV map.

[0060] The image texture of the main texture image is extracted, and the texture coordinate unfolding result is generated and associated with the corresponding triangle face. Specifically, the texture fragment of the triangle face region is obtained by extracting the pixel information of the main texture image corresponding to the triangle face, and the UV unfolding coordinate of the texture of the region is constructed based on the existing texture unfolding theory, so that the texture fragment is attached to the triangle face accordingly, thereby realizing the association mapping between the texture and the corresponding triangle face region.

[0061] Identify triangular faces belonging to the exterior facade detail component area, perform local texture magnification on the triangular faces and adjust the texture resolution to ensure that the texture clarity is not lower than the preset clarity threshold.

[0062] The method involves identifying the triangular faces corresponding to the detailed components of the building facade based on the original building-related information. The detailed component areas of the facade refer to high-precision decorative areas used to decorate the exterior of the building, such as reliefs, decorative strips, or decals. In this embodiment, an image region magnification algorithm is used to adjust the corresponding triangular faces to improve the texture resolution of the area, while ensuring that the texture clarity of the area is not lower than the clarity threshold set based on the modeling requirements.

[0063] Output all adjusted triangular surface texture data as a detail map set.

[0064] Methods for constructing detailed 3D architectural models include: The surface texture data of each triangle in the detail map set is texture-bound to the corresponding triangle in the initial 3D model. The surface texture data includes the UV map coordinate information and the pixel value of the area. The texture binding between the map and the corresponding area of ​​the model is achieved by overlaying these maps onto the triangles of the corresponding area of ​​the initial 3D model.

[0065] By integrating all the bound triangles, a detailed 3D architectural model is obtained. This model encapsulates all texture-bound triangle sets with the original initial 3D model into a unified model, maintaining consistency in triangle distribution, texture information, and component labels, and outputs a high-precision detailed 3D architectural model.

[0066] This embodiment acquires a modeling image dataset by collecting oblique image data of the urban area to be modeled and performing data cleaning. Based on this modeling image dataset, it performs projection consistency analysis and misalignment removal, occlusion area identification and compensation, detail preservation, normal optimization, and texture mapping, achieving refined construction of a 3D urban building model based on oblique photogrammetry. Compared with existing experience, identifying and removing boundary misalignment areas reduces contour noise accumulation caused by multi-view matching errors. By determining whether building facade component areas belong to occlusion areas and performing point cloud compensation, it avoids fracture deformation of facade components such as balcony railings caused by inconsistent occlusion. By performing mesh generation on detailed component areas, it preserves the detailed components of the roof area from being simplified and filtered by triangular faces. By filtering target normal vectors and performing normal direction smoothing, it eliminates the fracture phenomenon caused by the normal jump of adjacent triangular faces. By filtering the main texture image and performing local texture magnification on the facade detailed component areas, it ensures the texture clarity and texture alignment accuracy of the facade decorative details. This improves the boundary clarity, structural integrity, and texture refinement of the 3D urban building model.

[0067] Example 2 Please see Figure 2 As shown, for parts not described in detail in this embodiment, please refer to the description in Embodiment 1. A system for refining 3D urban building models based on oblique photogrammetry is provided, including: The data acquisition module collects oblique image data of the urban area to be modeled and performs data cleaning to obtain the modeling image dataset; The misalignment removal module performs projection consistency analysis on the modeling image dataset, identifies boundary misalignment regions and removes them from the modeling image dataset to obtain the misalignment removal dataset. The occlusion enhancement module identifies occlusion regions in the misaligned culling dataset, compensates for occlusion based on the identification results, and outputs an enhanced 3D point dataset. The detail preservation module performs detail preservation on the enhanced 3D point dataset to obtain a triangular face dataset. The normal adjustment module optimizes the normals of the triangular face dataset and performs planar adjustment on the triangular face dataset based on the optimization results to obtain a 3D mesh dataset. The modeling and construction module constructs an initial 3D model based on a 3D mesh dataset and performs texture mapping on the initial 3D model to obtain a detail texture set; it then combines the detail texture set and the initial 3D model to construct a detailed architectural 3D model; the various modules are connected via wired and / or wireless means.

[0068] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

[0069] All formulas in this manual are dimensionless and calculated numerically. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters and thresholds in the formulas are set by those skilled in the art according to the actual situation.

[0070] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

Claims

1. A method for refining 3D urban building models based on oblique photogrammetry, characterized in that, include: S1. Collect oblique image data of the urban area to be modeled and perform data cleaning to obtain the modeling image dataset; S2. Perform projection consistency analysis on the modeling image dataset to identify boundary misalignment regions and remove them from the modeling image dataset to obtain the misalignment removal dataset; S3. Identify occlusion regions in the misaligned culling dataset, compensate for occlusion based on the identification results, and output an enhanced 3D point dataset. S4. Perform detail preservation on the enhanced 3D point dataset to obtain the triangular face dataset; S5. Perform normal optimization on the triangular face dataset, and adjust the triangular face dataset in plane based on the optimization results to obtain a three-dimensional mesh dataset; S6. Construct an initial 3D model based on a 3D mesh dataset, and perform texture mapping on the initial 3D model to obtain a detail texture set; combine the detail texture set and the initial 3D model to construct a detailed architectural 3D model.

2. The method for refining urban architectural 3D models based on oblique photogrammetry according to claim 1, characterized in that, The methods for performing projection consistency analysis include: Extract multi-view image data belonging to the same building from the modeling image dataset, extract feature points for each image in the multi-view image data, and associate and match the feature points of each image with the feature points of other images to output a set of feature point pairs. Calculate the fundamental matrix between the image pairs corresponding to each feature point pair, and derive the projection matrix between the corresponding image pairs based on the fundamental matrix; Identify the common viewing region of each image pair, and transform the common viewing region of the corresponding image pair to a unified reference coordinate plane based on the projection matrix to obtain the common viewing image block; Extract the overlapping regions between co-image blocks and calculate the gradient direction difference of the overlapping regions. Draw the gradient direction map between the corresponding image blocks based on the gradient direction difference. If there are pixels in the gradient direction map whose gradient direction difference is greater than a preset direction threshold, and the number of such pixels exceeds a preset pixel ratio, then the region corresponding to the image block is determined to be a boundary misalignment region.

3. The method for refining the 3D model of urban buildings based on oblique photogrammetry according to claim 2, characterized in that, The methods for identifying occluded areas include: Filter out misaligned facade images in the dataset that cover the same building facade, and identify the building facade component areas in the facade image set. Obtain the shooting posture information for each facade image, and calculate the angle between the shooting direction and the viewing angle of the building facade in the corresponding facade image based on the shooting posture information; Edge extraction is performed on the building facade component area, and the outline pixel chain of the facade component is output; the edge density and edge connectivity coefficient of the outline pixel chain of the facade component are calculated; at the same time, the texture dispersion and orientation consistency coefficient of the building facade component area are calculated based on the gray value of the building facade component area; the visibility score of the corresponding building facade component area is calculated based on the edge density, edge connectivity coefficient, texture dispersion and orientation consistency coefficient. Filter facade images in the facade image set that contain the same building facade component area, and extract facade images with visibility scores lower than the fitted visibility score threshold as low-score images; If the viewing angle of the low-resolution image is greater than a preset angle threshold, the building facade component area corresponding to the low-resolution image is determined to be an occluded area.

4. The method for refining urban architectural 3D models based on oblique photogrammetry according to claim 3, characterized in that, The methods for performing occlusion compensation include: Extract the corresponding building facade component areas that are not marked as occlusion areas in the misalignment removal dataset and use them as compensation reference areas; Extract the edge coordinate point set of the compensation reference area and project it into 3D space to output the preliminary point cloud completion point set of the corresponding occluded area; Regions where the spacing between edge coordinates in the preliminary point cloud completion point set is greater than a preset spacing threshold are identified as gap regions; edge reconstruction fitting is performed on the gap regions to output an enhanced point cloud completion point set; The original misalignment removal dataset is transformed into a spatial coordinate point set, and the enhanced point cloud completion point set is merged with it to obtain an enhanced 3D point dataset.

5. The method for refining the 3D model of urban buildings based on oblique photogrammetry according to claim 4, characterized in that, The methods for preserving execution details include: Extract a subset of 3D point coordinates corresponding to the building roof area from the enhanced 3D point dataset, and calculate the rate of curvature change of each 3D point coordinate in the building roof area based on this subset of 3D point coordinates. Spatial clustering is performed on the corresponding 3D point coordinate subsets based on the rate of curvature change to obtain a curvature feature point subset; a curvature feature point subset in which the rate of curvature change of all curvature feature points in the point set is higher than the preset curvature threshold is selected, and it is determined whether the curvature feature points in the curvature feature point subset form a closed region. If a closed region is formed, the closed region is marked as a detail component region. Set detailed scale modeling parameters, and perform triangular meshing on the detailed component region based on these detailed scale modeling parameters to obtain the meshed component region; Transition triangles are generated at the junctions between the mesh component area and the remaining areas of the building roof area, and boundary stitching is performed. At the same time, labels are added to maintain the topological separation between the mesh component area and the remaining areas. Subset the 3D point coordinates of all stitched building roof areas into a triangular facet dataset.

6. The method for refining the 3D model of urban buildings based on oblique photogrammetry according to claim 5, characterized in that, The methods for performing normal optimization include: Extract the normal vector of each triangle in the triangle face dataset, and calculate the angle between the normal vectors of each pair of adjacent triangles sharing the same edge. If the angle between the normal vectors exceeds the preset angle jump threshold, the region formed by the pair of adjacent triangles sharing the same edge is determined as an abnormal normal transition region. The centroid of each triangular face in the abnormal transition region of the normal is identified. Based on the centroid, the multi-view local region image corresponding to the triangular face is matched, and the projection position of the centroid in the local region image of each view is calculated. Calculate the gray-level gradient direction of the local region image for each viewpoint, and calculate the normal estimation vector at the projection position based on the shooting direction and gray-level gradient direction under the corresponding viewpoint. Combine the estimated normal vectors of all viewpoints into a set of candidate normal vectors for the corresponding triangular face, calculate the angle between each candidate normal vector and the original normal vector, and select the candidate normal vector with the smallest angle and the smallest angle with the shooting direction as the target normal vector. Replace the normal vector of the corresponding triangle with the target normal vector, and perform normal direction smoothing on another adjacent triangle that is in the abnormal normal transition region; use the normal vectors of all updated triangles as the optimization result.

7. The method for refining urban architectural 3D models based on oblique photogrammetry according to claim 6, characterized in that, The methods for performing planar adjustment include: Based on the updated normal vectors of the triangles, the vertex positions between adjacent triangles are adjusted to ensure that the normal directions of adjacent triangles are continuous and the face boundaries are flat, and a 3D mesh dataset is output.

8. The method for refining the 3D model of urban buildings based on oblique photogrammetry according to claim 7, characterized in that, The methods for performing texture mapping include: Identify the local region image corresponding to each triangle face in the initial 3D model, project the triangle face onto each corresponding local region image, and calculate the pixel ratio of the projected region in each local region image and the gray-level gradient intensity of the projected region. The texture image score is calculated based on pixel ratio and grayscale gradient intensity, and the local region image with the highest texture image score is selected as the main texture image. Extract the image texture from the main texture image and generate texture coordinate expansion results to associate and map them with the corresponding triangle faces; Identify the triangular facets belonging to the exterior facade detail component area, perform local texture magnification on the triangular facets and adjust the texture resolution to ensure that the texture clarity is not lower than the preset clarity threshold; Output all adjusted triangular surface texture data as a detail map set.

9. The method for refining the 3D model of urban buildings based on oblique photogrammetry according to claim 8, characterized in that, The methods for constructing detailed 3D architectural models include: The surface texture data of each triangle in the detail map set is bound to the corresponding triangle in the initial 3D model; all bound triangles are integrated to obtain a detailed architectural 3D model.

10. A system for refining 3D urban building models based on oblique photogrammetry, used to implement the method for refining 3D urban building models based on oblique photogrammetry as described in any one of claims 1-9, characterized in that, include: The data acquisition module collects oblique image data of the urban area to be modeled and performs data cleaning to obtain the modeling image dataset; The misalignment removal module performs projection consistency analysis on the modeling image dataset, identifies boundary misalignment regions and removes them from the modeling image dataset to obtain the misalignment removal dataset. The occlusion enhancement module identifies occlusion regions in the misaligned culling dataset, compensates for occlusion based on the identification results, and outputs an enhanced 3D point dataset. The detail preservation module performs detail preservation on the enhanced 3D point dataset to obtain a triangular face dataset. The normal adjustment module optimizes the normals of the triangular face dataset and performs planar adjustment on the triangular face dataset based on the optimization results to obtain a 3D mesh dataset. The modeling and construction module constructs an initial 3D model based on a 3D mesh dataset and performs texture mapping on the initial 3D model to obtain a detail texture set; it then combines the detail texture set and the initial 3D model to construct a detailed architectural 3D model; the various modules are connected via wired and / or wireless means.

Images