Oblique photography and three-dimensional laser point cloud fused building model construction method

Abstract

The invention discloses a building model construction method based on oblique photography and three-dimensional laser point cloud fusion, and relates to the technical field of three-dimensional modeling. Comprising the following steps: S1, acquiring inclined image data and three-dimensional laser point cloud data; s2, preprocessing the collected data; s3, learning a corresponding relation between the features and performing feature fusion; s4, performing semantic annotation on the data, and combining an instance segmentation technology to realize rapid modeling; and S5, carrying out simplification processing on the building model. According to the invention, by setting a high-overlapping-degree photographing data acquisition mode, the integrity of building information is ensured; the oblique photography data and the three-dimensional laser point cloud data are subjected to feature correspondence based on the improved generative adversarial network, the precision of fusion data is improved, feature fusion is carried out on the oblique photography data and the three-dimensional laser point cloud data based on a layering and staged mode, the defect of oblique photography in the aspect of obtaining depth information is made up, and the accuracy of the depth information is improved. And meanwhile, the model has an accurate shape and a real appearance.

Description

Technical Field

[0001] The present invention relates to the technical field of three-dimensional modeling, and in particular to a method for constructing a building model by fusing oblique photography with three-dimensional laser point cloud. Background Art

[0002] In conventional methods, drone photography and 3D laser scanners process data separately. For example, drones perform oblique photogrammetry and post-processing software modeling separately; 3D laser scanners perform building scanning and post-point cloud stitching modeling separately.

[0003] Oblique photography can obtain image information of buildings from multiple angles, with high resolution and rich texture information. Through the multi-lens camera mounted on the aircraft, image data of a large area can be quickly obtained, with relatively low cost and flexible operation. It can intuitively reflect the appearance, color and other characteristics of the building, and provide basic texture materials for building a realistic 3D model.

[0004] 3D laser scanning technology can accurately obtain the 3D spatial coordinate information of buildings, and the point cloud data has high-precision 3D geometric information. It can accurately describe the shape, structure and spatial position relationship of the building, and has a strong ability to capture the details of the building's outline, height and complex structure. Whether it is a regular shape or an irregular shape building, detailed and accurate data can be obtained, making up for the shortcomings of traditional measurement methods in obtaining 3D information.

[0005] However, although oblique photography can provide rich textures, it has certain limitations in obtaining accurate three-dimensional structure and depth information of buildings, and is greatly affected by factors such as lighting and occlusion. For example, in areas with dense buildings, the texture of some areas may be missing or inaccurate due to mutual occlusion. Although the three-dimensional laser point cloud has high geometric accuracy, it lacks intuitive texture and color information, and the generated model is relatively simple in visual effect. Models built solely on point clouds lack realistic presentation, which is not conducive to intuitive display and application.

[0006] In complex urban planning and other aspects, building models with higher precision and higher restoration are needed. For example, in the field of urban planning, high-precision and realistic building models are needed for spatial analysis, planning and design. The fusion of oblique photography and 3D laser point cloud can provide urban planners with more accurate building information, such as building height, area, spacing, etc., which is helpful for rational planning of urban layout, infrastructure construction, etc. In terms of architectural heritage protection, the fusion model can accurately record the current status of the building and provide reliable data support for subsequent restoration, protection and research, while retaining the appearance and texture characteristics of the building and accurately presenting its structural information. In real estate development, it can be used to display the real effects and surrounding environment of the real estate, helping buyers to understand the real estate information more intuitively, and also providing a basis for developers to carry out project planning and design optimization.

[0007] Therefore, there is an urgent need for a method to fuse the two data to construct a building model. There are methods to fuse the two data in the existing technology, but the fusion accuracy of the two data in the existing registration algorithm needs to be improved in complex scenes; and the traditional method of accurately mapping the texture information of oblique photography to the model constructed by three-dimensional laser point cloud is prone to model distortion, deformation or missing problems, which affects the visualization effect of the model.

[0008] In order to solve the above problems, the present invention proposes a building model construction method by fusing oblique photography with three-dimensional laser point cloud. Summary of the invention

[0009] The purpose of the present invention is to propose a building model construction method by fusing oblique photography with three-dimensional laser point cloud, which can effectively solve the problems in the background technology.

[0010] In order to achieve the above object, the present invention adopts the following technical solutions: The method for constructing a building model by integrating oblique photography and three-dimensional laser point cloud comprises the following steps: S1: Use drones for oblique photography to obtain multi-angle oblique image data of buildings, and use 3D laser scanners to obtain 3D laser point cloud data of buildings; S2: pre-processing the collected oblique image data and 3D laser point cloud data; S3: The improved generative adversarial network is used to automatically learn the feature correspondence between oblique images and 3D laser point clouds, and the texture features, color features of oblique images and geometric features of 3D laser point clouds are fused from low resolution to high resolution based on a multi-stage hierarchical feature fusion method; S4: Based on semantic segmentation technology, semantic annotation is performed on oblique images and point cloud data to distinguish different parts of buildings. At the same time, combined with instance segmentation technology, repeated structures in buildings are identified to achieve rapid modeling and parameter adjustment; S5: Simplify the constructed building model using model simplification and compression technology to generate the final constructed building model.

[0011] Preferably, in S1, different types of drones are used to carry oblique photography cameras to take pictures from several heights and angles; several three-dimensional laser scanners are used to scan buildings from different positions; and data of buildings under construction or with dynamic changes are continuously collected within a specific time period.

[0012] Preferably, during the shooting process, the oblique photography camera sets the heading overlap to 75-80%, the lateral overlap to 70-75%, and performs close-up photography on the pre-divided key areas and blocked areas.

[0013] Preferably, in S2, radiation correction, geometric correction, image registration and image enhancement processing are performed on the collected oblique image data; point cloud denoising, point cloud filtering, point cloud registration and point cloud simplification processing are performed on the collected three-dimensional laser point cloud data.

[0014] Preferably, in S3, the feature correspondence between the oblique image and the three-dimensional laser point cloud is automatically learned based on the improved adversarial generative network, and the generative network of the improved adversarial generative network includes an image feature extraction branch, a point cloud feature extraction branch and a feature fusion branch, respectively realizing the extraction of oblique image information and three-dimensional laser point cloud information through an image Transformer module, a point cloud Transformer module and a CNN module, and further fusing them; The image feature extraction branch consists of 2 convolution blocks and 4 image Transformer blocks; the point cloud feature extraction branch consists of 2 convolution blocks and 4 point cloud Transformer blocks; the feature fusion branch consists of 4 convolution blocks and 1 image reconstruction module; The convolution blocks are composed of 1 convolution layer, 1 batch normalization layer and an activation function; the size of the convolution kernel in the convolution layer is set to 3*3, the step size is 1, and the number of convolutions is 128; The image transformer block cuts the input image feature map into several small blocks; then performs linear operations to obtain a feature vector with a uniform shape; then inputs the feature vector into the swin transformer module, the feature vector is first processed by layer normalization calculation, and then input into the hybrid attention mechanism; The hybrid attention mechanism includes a channel attention mechanism and a spatial attention mechanism; The channel attention mechanism performs global average pooling and maximum pooling operations on the input feature data to obtain the description features and , these two features are added by a shared multi-layer perceptron and processed by the Sigmoid function; ; Finally, the channel weight coefficient With the input feature data Multiply to get channel attention feature data ; in, is the activation function; is a shared multi-layer perceptron; The spatial attention module then focuses on the input feature data Perform average pooling and maximum pooling operations to obtain descriptive features respectively. and , these two features are horizontally spliced ​​and aggregated, and a convolution kernel of size 7*7 is passed in to operate; to obtain ; Finally, through the spatial weight coefficient With the input feature data Multiply them to get the mixed attention feature data ; After feature mapping by the swin transformer module, the shape of the output feature vector is changed into the image feature data and 3D laser point cloud feature data that are the same as the original input feature data, that is, the output feature data of the image transformer block and the point cloud transformer block; The discriminant network builds a recursive discriminant network to discriminate the output of the generating network. The recursive discriminant network integrates a multi-stage attention mechanism for comprehensive discrimination, as follows: No. The stages are as follows: in, For the A weight matrix; For the The input of the attention module; For the The output of the attention module; For the convolution kernels; is a connector; To improve the output of the generative adversarial network; The discriminator of each stage in the recursive discriminant network is composed of 5 convolutions, the size of the convolution kernel of the convolution block is set to 3*3, and the step size is 1; the discriminant block is composed of a fully connected block and a discriminant block layer and an activation function; The target structure of the improved adversarial generative network is also sent as input to the recursive discriminant network, the deviation between the two is calculated, and the deviation is returned to each layer of the generative network together with the loss function, the network parameters are continuously optimized, and finally high-quality fusion features are generated; The loss function dynamically adjusts the activity based on the performance of the generator and discriminator networks.

[0015] Preferably, in S3, the upper and lower limits of the resolution in different intervals are pre-divided, and combined with the multi-stage method of the above-mentioned multi-stage attention mechanism, the texture features of the oblique image and the geometric features of the point cloud are subjected to multi-stage hierarchical feature fusion in a low-resolution to high-resolution manner.

[0016] Preferably, the S4 also combines the multi-stage approach of the multi-stage attention mechanism to track and simulate dynamically changing buildings in stages.

[0017] Preferably, the semantic segmentation technology described in S4 searches for the neighboring points of each point in the three-dimensional point cloud based on the adaptive KNN nearest neighbor algorithm. Each point and its neighboring points form a local adaptive reference point set. Then, in combination with geometric constraints, a micro-cutting plane is fitted using the least squares method. At the same time, the micro-cutting plane is optimized based on the gradient descent algorithm. The adaptive reference point set is projected onto the optimized micro-cutting plane, and the maximum angle between each projected point and the corresponding connecting line of its neighboring points is determined to identify the boundary features of the building point cloud.

[0018] Preferably, in S5, real-time rendering technology is also used to realize online browsing and interaction of the building model.

[0019] Compared with the prior art, the present invention provides a building model construction method by integrating oblique photography with three-dimensional laser point cloud, which has the following beneficial effects: The present invention ensures the integrity of building information by setting a high-overlapping photographic data collection method; based on an improved adversarial generative network, the oblique photography data and the three-dimensional laser point cloud data are feature-matched to improve the accuracy of the fused data; based on a layered and staged approach, the oblique image data and the three-dimensional laser point cloud data are feature-fused to make up for the shortcomings of oblique photography in obtaining depth information, while allowing the model to have an accurate shape and a real appearance, thereby improving the visualization effect; constructing a building model by fusing oblique photography with three-dimensional laser point clouds can reduce the workload of post-processing to a certain extent. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 This is a flow chart of the method mentioned in Example 1 of the present invention; Figure 2 This is a schematic diagram of the network framework generated in Example 1 of the present invention; Figure 3 This is a schematic diagram of the recursive discriminant network framework mentioned in Example 1 of the present invention. DETAILED DESCRIPTION

[0021] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0022] The present invention ensures the integrity of building information by setting a high-overlapping photography data collection method; based on an improved adversarial generative network, the oblique photography data and the three-dimensional laser point cloud data are feature-matched to improve the accuracy of the fused data; based on a layered and staged approach, the oblique image data and the three-dimensional laser point cloud data are feature-fused to make up for the shortcomings of oblique photography in obtaining depth information, while making the model have an accurate shape and a real appearance, improving the visualization effect; constructing a building model by fusing oblique photography with three-dimensional laser point clouds can reduce the workload of post-processing to a certain extent. Specifically, it includes the following contents. Embodiment 1:

[0023] See also Figure 1-Figure 3 The method for constructing a building model by integrating oblique photography with three-dimensional laser point cloud of the present invention comprises the following steps: S1: Use drones for oblique photography to obtain multi-angle oblique image data of buildings, and use 3D laser scanners to obtain 3D laser point cloud data of buildings; the details are as follows: Different types of drones equipped with oblique photography cameras are used to shoot from multiple heights and angles to obtain richer image information. At the same time, multiple 3D laser scanners are equipped to scan buildings from different positions to improve the coverage and accuracy of point cloud data. During the shooting process, the oblique photography camera sets the heading overlap to 75-80% and the lateral overlap to 70-75%, and close-up photography is performed for pre-divided key areas and blocked areas.

[0024] It is also possible to use ground-based mobile measurement systems to collect data around buildings to complement aerial data.

[0025] During a specific period of time, continuous data collection is performed on buildings that are under construction or undergoing dynamic changes. For example, for large-scale construction projects, oblique photography and point cloud scanning are performed at regular intervals to record the construction process of the building and provide a data basis for the dynamic update of the model.

[0026] S2: Preprocess the collected oblique image data and 3D laser point cloud data; the details are as follows: The collected oblique image data is subjected to radiation correction to eliminate the influence of factors such as illumination and atmosphere on the image, making the grayscale value of the image more accurate. Then geometric correction is performed to eliminate the geometric deformation of the image and make it consistent with the geometric position of the real object. Multiple images are also registered to align them in space for subsequent fusion and analysis. Finally, the image is enhanced, such as histogram equalization and contrast stretching, to improve the contrast and clarity of the image.

[0027] The collected 3D laser point cloud data is initially denoised to remove noise points and outliers, thus improving the accuracy and quality of the point cloud. Filtering is then performed, such as mean filtering and median filtering, to smooth the point cloud surface and remove noise and details. Multiple groups of 3D laser point clouds are also registered to align them in space for subsequent fusion and analysis. Finally, the 3D laser point cloud is simplified, such as downsampling and gridding, to reduce the number of point clouds and improve processing speed.

[0028] It is also possible to combine multi-scale analysis methods to carry out targeted processing of noise at different scales to further improve data quality.

[0029] S3: The improved adversarial generative network is used to automatically learn the feature correspondence between oblique images and 3D laser point clouds, and the texture features, color features of oblique images and geometric features of 3D laser point clouds are fused from low resolution to high resolution based on a multi-stage hierarchical feature fusion method; the details are as follows: Reference Figure 2 Based on the improved generative adversarial network, the feature correspondence between the oblique image and the 3D laser point cloud is automatically learned. The generative network of the improved generative adversarial network includes an image feature extraction branch, a point cloud feature extraction branch and a feature fusion branch. The image Transformer module, the point cloud Transformer module and the CNN module are used to extract the oblique image information and the 3D laser point cloud information, and further fuse them. The image feature extraction branch consists of 2 convolution blocks and 4 image Transformer blocks; the point cloud feature extraction branch consists of 2 convolution blocks and 4 point cloud Transformer blocks; the feature fusion branch consists of 4 convolution blocks and 1 image reconstruction module; The convolution blocks are composed of one convolution layer, one batch normalization layer and an activation function. The size of the convolution kernel in the convolution layer is set to 3*3, the step size is 1, and the number of convolutions is 128. The image transformer block cuts the input image feature map into several small blocks; then performs linear operations to obtain a feature vector with a uniform shape; then the feature vector is input into the swin transformer module, the feature vector is first processed by layer normalization calculation, and then input into the hybrid attention mechanism; The hybrid attention mechanism includes channel attention mechanism and spatial attention mechanism; The channel attention mechanism performs global average pooling and maximum pooling operations on the input feature data to obtain the descriptive features. and , these two features are added by a shared multi-layer perceptron and processed by the Sigmoid function; ; Finally, the channel weight coefficient With the input feature data Multiply to get channel attention feature data ; in, is the activation function; is a shared multi-layer perceptron; The spatial attention module then focuses on the input feature data Perform average pooling and maximum pooling operations to obtain descriptive features respectively. and , these two features are horizontally spliced ​​and aggregated, and a convolution kernel of size 7*7 is passed in to operate; to obtain ; Finally, through the spatial weight coefficient With the input feature data Multiply them to get the mixed attention feature data ; After feature mapping by the swin transformer module, the shape of the output feature vector is changed into the image feature data and 3D laser point cloud feature data that are the same as the original input feature data, that is, the output feature data of the image transformer block and the point cloud transformer block; Reference Figure 3 , the discriminant network builds a recursive discriminant network to discriminate the output of the generating network. The recursive discriminant network integrates the multi-stage attention mechanism for comprehensive discrimination, as follows: The first stage is as follows: The details of the second stage are as follows: No. The stages are as follows: No. The stages are as follows: in, For the A weight matrix; For the The input of the attention module; For the The output of the attention module; For the convolution kernels; is a connector; To improve the output of the generative adversarial network; The discriminator of each stage in the recursive discriminant network consists of 5 convolution blocks and 1 discriminant block. The size of the convolution kernel of the convolution block is set to 3*3 and the step size is 1; the discriminant block consists of a fully connected layer and an activation function; The optimal target structure of the improved adversarial generative network is also sent as input to the recursive discriminant network, and the deviation between the two is calculated. The deviation is returned to each layer of the generative network together with the loss function, and the network parameters are continuously optimized to finally generate high-quality fusion features. Among them, the loss function dynamically adjusts the activity according to the performance of the generative network and the discriminant network.

[0030] The upper and lower limits of the resolution are pre-divided into different intervals, and combined with the multi-stage method of the multi-stage attention mechanism mentioned above, the texture features of the oblique image and the geometric features of the point cloud are fused in a multi-stage hierarchical manner from low resolution to high resolution. The specific steps are as follows: After the processing steps of S1 and S2, the edges, corners and other features are extracted from the low-resolution oblique image data, and the geometric features such as planes and line segments are extracted from the 3D laser point cloud data. These features are used for matching and association, for example, by establishing feature descriptors and performing similarity measurements to find corresponding feature point pairs. Based on the matched feature point pairs, preliminary registration and fusion are performed to determine the spatial correspondence between the oblique image and the point cloud.

[0031] A preliminary building model framework is constructed based on the fused low-resolution data. This framework reflects the overall shape, volume, and distribution of the main structures of the building. For example, for a building with a simple shape, the main framework of the rectangular shape can be quickly outlined; for a complex building, the approximate orientation relationship of different structural parts can also be determined.

[0032] Then introduce higher resolution oblique images and point cloud data blocks. First, based on the improved adversarial generative network, the new data is matched with the existing coarse model. In the feature matching, the coordinate transformation parameters between the new data and the coarse model are continuously adjusted.

[0033] The detailed information in the high-resolution data is integrated into the coarse model. For example, the building surface texture details, the outline information of small parts such as windows obtained from the high-resolution oblique image, and the subtle concave and convex structure of the building facade and the precise shape of the decorative components obtained from the high-resolution point cloud are gradually added to the coarse model to refine it.

[0034] As the resolution continues to increase and data fusion deepens, the model is optimized. This includes smoothing the model surface to remove uneven areas caused by data fusion or errors; repairing possible holes, gaps and other defects in the model; and optimizing the model's geometric structure to make it more in line with the actual building's mechanical structure and aesthetic requirements.

[0035] Finally, the building model is constructed from coarse to fine, and a three-dimensional building model with rich details, precise geometric structure and realistic texture appearance is obtained, which can be used for subsequent analysis, display or other applications.

[0036] Based on the above multi-stage hierarchical feature fusion steps, the accuracy of data fusion can be improved and the construction accuracy of the building model can be improved.

[0037] S4: Based on semantic segmentation technology, semantic annotation is performed on oblique images and point cloud data to distinguish different parts of buildings. At the same time, combined with instance segmentation technology, repeated structures in buildings are identified to achieve rapid modeling and parameter adjustment; the details are as follows: Based on semantic segmentation technology, the oblique images and point cloud data are semantically annotated to distinguish different parts of the building. Specifically, the neighboring points of each point in the three-dimensional point cloud are searched based on the adaptive KNN nearest neighbor algorithm. Each point and its neighboring points form a local adaptive reference point set. Then, combined with geometric constraints, a micro-cutting plane is fitted using the least squares method. At the same time, the micro-cutting plane is optimized based on the gradient descent algorithm, and then the adaptive reference point set is projected onto the optimized micro-cutting plane. The maximum angle between each projected point and the corresponding connecting line with its neighboring points is determined to identify the boundary features of the building point cloud.

[0038] The traditional KNN algorithm uses a fixed k value, but in point clouds of different regions or densities, the choice of k value may affect the accuracy of the results. The introduction of an adaptive k value selection method can automatically adjust the k value according to the density and features of the local point cloud, which can improve the accuracy of boundary feature recognition.

[0039] When fitting the micro-cutting plane, the fitting accuracy can be improved by combining geometric constraints, such as normal vector constraints of boundary points, distance constraints between boundary points, etc. Optimization algorithms, such as gradient descent or genetic algorithms, can also be used to optimize the parameters of the micro-cutting plane to meet the geometric constraints.

[0040] It also combines instance segmentation technology to identify repeated structures in buildings and store them in the graphics library. When building the same structural model, it can be directly called, which can greatly shorten the model building time. For buildings with dynamic changes, such as deformable structures or buildings affected by external forces, it can also combine multi-stage mechanisms to adopt dynamic modeling methods to track and simulate dynamically changing buildings.

[0041] By fusing the texture features of oblique images with the geometric features of 3D laser point clouds, a highly realistic texture map can be generated, reducing the workload of later manual modification.

[0042] Based on the above building model construction steps, the coordinates of the image control points and the reference points are compared and calculated for the 6 reference points in the randomly pre-constructed building model. The calculation results can be referred to Table 1: Table 1 Coordinate comparison of image control points and reference points

[0043] It can be seen from the table that the mean square error in the X direction is about ±0.313m, the mean square error in the Y direction is about ±0.474m, and the mean square error in the Z direction is about ±0.050m, which has high accuracy.

[0044] S5: Use model simplification and compression technology to simplify the constructed building model to generate the final constructed building model. The details are as follows: The completed building model can be stored, and an identification model can be built to identify unimportant edges or vertices in the building model and fold them to reduce the complexity of the model. For example, edges or vertices can be determined based on the length of the edge, the angle with the adjacent face, and the importance in the overall shape of the model; or based on the distance between vertices. During the folding process, the position of the vertices in the folded model needs to be recalculated to maintain the general shape of the model.

[0045] For model structures at different levels in the building model, we can combine the actual architecture of the building and render them in layers. We can dynamically select the appropriate LOD level based on the distance between the observation value and the building to ensure the visual effect of the model.

[0046] You can also compress the texture of the model, using texture compression algorithms such as DXT, ETC, etc. to reduce the storage space of texture images. By quantizing and encoding texture colors, the size of texture data can be greatly reduced while maintaining visual quality.

[0047] Finally, the simplified and compressed model is optimized and checked for errors or incomplete geometry. The model is checked and repaired using model checking tools.

[0048] Verify the visual quality and accuracy of the model to ensure that the simplification and compression process has not significantly affected the appearance and structure of the model. You can verify the quality of the model by comparing the model rendering effects before and after simplification, or by conducting actual application tests.

[0049] The final building model is exported to a format suitable for the target application. In the application, the model parameters such as LOD level, texture resolution, etc. can be further adjusted according to actual needs to achieve the best performance and visual effects.

[0050] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A method for constructing a building model by integrating oblique photography with three-dimensional laser point cloud, characterized in that: The steps include: S1: Use drones for oblique photography to obtain multi-angle oblique image data of buildings, and use 3D laser scanners to obtain 3D laser point cloud data of buildings; S2: pre-processing the collected oblique image data and 3D laser point cloud data; S3: The improved generative adversarial network is used to automatically learn the feature correspondence between oblique images and 3D laser point clouds, and the texture features, color features of oblique images and geometric features of 3D laser point clouds are fused from low resolution to high resolution based on a multi-stage hierarchical feature fusion method; S4: Based on semantic segmentation technology, semantic annotation is performed on oblique images and point cloud data to distinguish different parts of buildings. At the same time, combined with instance segmentation technology, repeated structures in buildings are identified to achieve rapid modeling and parameter adjustment; S5: Simplify the constructed building model using model simplification and compression technology to generate the final constructed building model.

2. The method for constructing a building model by integrating oblique photography with three-dimensional laser point cloud according to claim 1, characterized in that: In S1, different types of drones are used to carry oblique photography cameras to take pictures from several heights and angles; at the same time, several three-dimensional laser scanners are used to scan the buildings from different positions; and within a specific time period, data is continuously collected for buildings that are under construction or have dynamic changes.

3. The method for constructing a building model by integrating oblique photography with three-dimensional laser point cloud according to claim 2, characterized in that: During the shooting process, the oblique photography camera sets the heading overlap to 75-80%, the lateral overlap to 70-75%, and performs close-up photography for the pre-divided key areas and blocked areas.

4. The method for constructing a building model by integrating oblique photography with three-dimensional laser point cloud according to claim 1, characterized in that: In S2, radiation correction, geometric correction, image registration and image enhancement processing are performed on the collected oblique image data; point cloud denoising, point cloud filtering, point cloud registration and point cloud simplification processing are performed on the collected three-dimensional laser point cloud data.

5. The method for constructing a building model by integrating oblique photography and three-dimensional laser point cloud according to claim 1, characterized in that: In S3, the feature correspondence between the oblique image and the three-dimensional laser point cloud is automatically learned based on the improved adversarial generative network. The generative network of the improved adversarial generative network includes an image feature extraction branch, a point cloud feature extraction branch and a feature fusion branch. The image Transformer module and the point cloud Transformer module are combined with the CNN module to realize the extraction of oblique image information and three-dimensional laser point cloud information, and further fuse them. The image feature extraction branch consists of 2 convolution blocks and 4 image Transformer blocks; the point cloud feature extraction branch consists of 2 convolution blocks and 4 point cloud Transformer blocks; the feature fusion branch consists of 4 convolution blocks and 1 image reconstruction module; The convolution blocks are composed of 1 convolution layer, 1 batch normalization layer and an activation function; the size of the convolution kernel in the convolution layer is set to 3*3, the step size is 1, and the number of convolutions is 128; The image transformer block cuts the input image feature map into several small blocks; then performs linear operations to obtain a feature vector with a uniform shape; then inputs the feature vector into the swin transformer module, the feature vector is input into the swin transformer module, firstly processed by layer normalization calculation, and then input into the hybrid attention mechanism; The hybrid attention mechanism includes a channel attention mechanism and a spatial attention mechanism; The channel attention mechanism performs global average pooling and maximum pooling operations on the input feature data to obtain the description features and , these two features are added by a shared multi-layer perceptron and processed by the Sigmoid function; ; Finally, the channel weight coefficient With the input feature data Multiply to get channel attention feature data ; in, is the activation function; is a shared multi-layer perceptron; The spatial attention module then focuses on the input feature data Perform average pooling and maximum pooling operations to obtain descriptive features respectively. and , these two features are horizontally spliced ​​and aggregated, and a convolution kernel of size 7*7 is passed in to operate; to obtain ; Finally, through the spatial weight coefficient With the input feature data Multiply them to get the mixed attention feature data ; After feature mapping by the swin transformer module, the shape of the output feature vector is changed into the image feature data and 3D laser point cloud feature data that are the same as the original input feature data, that is, the output feature data of the image transformer block and the point cloud transformer block; The discriminant network builds a recursive discriminant network to discriminate the output of the generating network. The recursive discriminant network integrates a multi-stage attention mechanism for comprehensive discrimination, as follows: No. The stages are as follows: in, For the A weight matrix; For the The input of the attention module; For the The output of the attention module; For the convolution kernels; is a connector; To improve the output of the generative adversarial network; The discriminator of each stage in the recursive discriminant network consists of 5 convolution blocks and 1 discriminant block. The size of the convolution kernel of the convolution block is set to 3*3 and the step size is 1; the discriminant block consists of a fully connected layer and an activation function; The target structure of the improved adversarial generative network is also sent as input to the recursive discriminant network, the deviation between the two is calculated, and the deviation is returned to each layer of the generative network together with the loss function, the network parameters are continuously optimized, and finally high-quality fusion features are generated; The loss function dynamically adjusts the activity based on the performance of the generator and discriminator networks.

6. The method for constructing a building model by integrating oblique photography with three-dimensional laser point cloud according to claim 5, characterized in that: In S3, the upper and lower limits of the resolution are pre-divided into different intervals, and combined with the multi-stage method of the multi-stage attention mechanism, the texture features of the oblique image and the geometric features of the point cloud are fused in a multi-stage hierarchical manner from low resolution to high resolution.

7. The method for constructing a building model by integrating oblique photography with three-dimensional laser point cloud according to claim 6, characterized in that: The S4 also combines the multi-stage approach of the multi-stage attention mechanism to track and simulate dynamically changing buildings in stages.

8. The method for constructing a building model by integrating oblique photography with three-dimensional laser point cloud according to claim 7, characterized in that: The semantic segmentation technology described in S4 searches for the neighboring points of each point in the three-dimensional point cloud based on the adaptive KNN nearest neighbor algorithm. Each point and its neighboring points form a local adaptive reference point set. Then, in combination with geometric constraints, a micro-cutting plane is fitted using the least squares method. At the same time, the micro-cutting plane is optimized based on the gradient descent algorithm. The adaptive reference point set is then projected onto the optimized micro-cutting plane, and the maximum angle between each projected point and the corresponding connecting line with its neighboring points is determined to identify the boundary features of the building point cloud.

9. The method for constructing a building model by integrating oblique photography and three-dimensional laser point cloud according to claim 1, characterized in that: In S5, real-time rendering technology is also used to realize online browsing and interaction of building models.

Images