High-precision oblique photography monomerization method

Abstract

The invention discloses a high-precision oblique photography monomerization method. The method comprises the following steps of S1, data acquisition and arrangement, S2, data preprocessing, S3, feature extraction, S4, model segmentation, S5, monomerization model optimization, and S6, model quality evaluation and updating. According to the method, more comprehensive and detailed features can be extracted, more accurate segmentation is realized, in texture feature extraction, SIFT, SURF and BRIEF algorithms are combined, stable texture features can be extracted, the extraction efficiency is improved, during model segmentation, a region growing algorithm is combined with multiple features such as colors, textures and geometric shapes, and DBSCAN and a hierarchical clustering algorithm are combined, so that the segmentation efficiency is improved, and the segmentation efficiency is improved. According to the method provided by the invention, by combining semantic segmentation models, such as, U-Net, MaskR-CNN and FCN, a segmentation result can more accurately reflect real boundaries and details of an object, the model segmented by the method is finer and more complete compared with a model segmented by a common method, and richer information can be provided for subsequent analysis and application.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a high-precision oblique photography singulation method. Background Art

[0002] With the development of digital technology, oblique photography real-scene three-dimensional models have been widely used in many fields such as urban planning, architectural design, and cultural relics protection.

[0003] However, most of the existing oblique photography real-scene three-dimensional models are presented in the form of an overall model, which makes it difficult to carry out accurate identification, efficient management and in-depth analysis of individual objects in the model when facing complex scenes. The traditional singularization method is inefficient in solving the above problems. When processing large-scale data, it requires a lot of manpower and time costs, and the accuracy performance is poor. It is difficult to meet application scenarios with strict requirements on details. The degree of automation is also at a low level and relies heavily on manual intervention, resulting in the entire process being cumbersome and error-prone. In summary, the traditional singularization method can no longer meet the current growing demand for refined three-dimensional model applications. Therefore, the present invention proposes a high-precision oblique photography singularization method. Summary of the Invention

[0004] The purpose of the present invention is to solve the shortcomings of the prior art and to propose a high-precision oblique photography singulation method.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] A high-precision oblique photography singulation method comprises the following steps:

[0007] S1. Data Collection and Collation: Use oblique photography equipment to capture the target area from multiple angles to obtain rich image data. Use LiDAR and other equipment to collect point cloud data of the target area. Collate the collected image data and point cloud data, categorize and store them according to time, shooting location, and other information to prepare for subsequent processing.

[0008] S2. Data preprocessing: De-noise the image data to remove noise caused by factors such as the shooting environment and equipment; enhance blurred images to improve image clarity; remove outliers from point cloud data to ensure accuracy; streamline data to reduce data volume and improve processing efficiency; and fuse the preprocessed image data with the point cloud data to form a unified data set;

[0009] S3. Feature extraction: Extract key features for individualization from the fused data. For image data, these features include texture and color features of the object. For point cloud data, these features include geometric features such as shape, size, and spatial position. Deep learning algorithms are used to optimize and filter the extracted features to improve their recognition and representativeness.

[0010] S4. Model Segmentation: Use region growing and clustering algorithms to perform preliminary segmentation on the fused data, dividing the model into multiple approximate regions. Combined with deep learning semantic segmentation models such as U-Net and Mask R-CNN, perform fine segmentation on the initially segmented regions, accurately identify the boundaries of each object, and achieve individual segmentation of the model.

[0011] S5. Individualized model optimization: Perform geometric correction on the individualized model to correct geometric deformations caused by the model acquisition and processing process to ensure model accuracy. Optimize the model's texture to remove problems such as texture stretching and distortion to make the model's appearance more realistic. Add attribute information, such as the object's name, type, and purpose, to enrich the model's semantic information.

[0012] S6. Model quality assessment and update: Develop model quality assessment indicators, such as individualization accuracy, model integrity, texture quality, etc., conduct a comprehensive assessment of the individualized model, and make targeted adjustments and optimizations to unqualified parts based on the assessment results. Regularly collect new image data and point cloud data, and use incremental update technology to update the individualized model in real time or regularly to ensure consistency between the model and the real scene.

[0013] Preferably, in step S1, the oblique photography equipment can be a multi-rotor drone equipped with a multi-lens oblique camera. According to the size and complexity of the target area, the flight route and shooting parameters are reasonably planned, and the lidar equipment selects appropriate accuracy and range to ensure that the collected point cloud data can accurately reflect the terrain and object characteristics of the target area.

[0014] Preferably, in step S2, image denoising can adopt methods such as Gaussian filtering and median filtering, image enhancement can use techniques such as histogram equalization and contrast stretching, outliers in point cloud data can be removed by statistical filtering and radius filtering, and data simplification can be achieved through voxel grid downsampling algorithm.

[0015] Preferably, in step S3, texture feature extraction adopts algorithms such as scale-invariant feature transformation and accelerated robust features, color feature extraction is analyzed through RGB color space, HSV color space, etc., geometric feature extraction uses information such as normal vector and curvature of point cloud, and deep learning algorithm uses convolutional neural network for feature optimization, which is trained through a large amount of labeled data to improve the accuracy of feature extraction.

[0016] Preferably, in step S4, the region growing algorithm merges adjacent pixels or point clouds into one region based on similar features of objects, such as color, texture, geometric shape, etc. The clustering algorithm can adopt algorithms such as DBSCAN to perform clustering according to the density distribution of data points. During the training process, the semantic segmentation model uses a large number of oblique photography real-scene 3D model samples for training, continuously optimizes model parameters, and improves segmentation accuracy.

[0017] Preferably, in step S5, geometric correction adopts a correction method based on control points, multiple corresponding control points are selected in the model and the real scene, and geometric correction is achieved through coordinate transformation. Texture optimization uses an image deformation algorithm to stretch and distort the texture. The addition of attribute information can be achieved through manual annotation or extracting matching information from a relevant database.

[0018] Preferably, in step S6, the model quality assessment index is reasonably set according to different application scenarios and requirements, and the incremental update technology adopts feature matching and model fusion algorithms to match and fuse the newly collected data with the original model to achieve efficient updating of the model.

[0019] The present invention has the following beneficial effects:

[0020] 1. For different application scenarios, such as urban areas, mountainous areas, old residential communities, and large industrial parks, the code supports flexible adjustment of data collection equipment parameters and flight route planning. For example, in complex mountainous terrain, fixed-wing drones can set higher flight altitudes and larger route spacing to quickly cover large areas. In old urban communities, multi-rotor drones can lower their flight altitudes, reduce route spacing, and finely collect details of buildings and facilities within the community. This customized data collection method is difficult to achieve with ordinary methods, which often lack flexibility and find it difficult to take into account the special needs of different scenarios.

[0021] 2. It can extract more comprehensive and detailed features and achieve more accurate segmentation. In texture feature extraction, the combination of SIFT, SURF and BRIEF algorithms can not only extract stable texture features, but also improve the extraction efficiency. In model segmentation, the region growing algorithm combines multiple features such as color, texture and geometric shape, DBSCAN and hierarchical clustering algorithms are combined, as well as semantic segmentation models such as U-Net, Mask R-CNN and FCN. The segmentation results can more accurately reflect the real boundaries and details of the object. Compared with the segmentation models of ordinary methods, the models are more refined and complete, and can provide richer information for subsequent analysis and application.

[0022] 3. The entire code implementation process incorporates deep learning algorithms and automated processing procedures. Convolutional neural networks are used for feature optimization. Through training with a large amount of labeled data, they can automatically learn more representative features. In model segmentation, the semantic segmentation model is also based on deep learning, and model parameters are continuously optimized to improve segmentation accuracy. These automated and intelligent processing methods reduce manual intervention, not only improving processing efficiency but also reducing human errors.

[0023] 4. The code uses incremental update technology. Through feature matching and model fusion algorithms, it can quickly match and fuse newly collected data with the original model to achieve real-time or periodic updates of the model. This enables the model to promptly reflect changes in real-world scenarios, such as new buildings in urban construction and terrain changes in mountainous areas. Ordinary methods may require a large amount of data collection and processing work to be re-performed when updating the model, which is inefficient and difficult to meet the requirements for real-time and accuracy of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 This is a step diagram of a high-precision oblique photography singulation method proposed by the present invention;

[0025] Figure 2 This is a diagram showing some codes for data preprocessing in a high-precision oblique photography singulation method proposed in the present invention;

[0026] Figure 3 This is a partial code display diagram of feature extraction in a high-precision oblique photography singulation method proposed in the present invention;

[0027] Figure 4 This is a partial code display diagram of the region growing algorithm in the high-precision oblique photography singulation method proposed in the present invention;

[0028] Figure 5 This is a partial code display diagram for the semantic segmentation model definition in the high-precision oblique photography singulation method proposed in this invention. DETAILED DESCRIPTION

[0029] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0030] A high-precision oblique photography singulation method comprises the following steps:

[0031] S1. Data Collection and Collation: Use oblique photography equipment to capture the target area from multiple angles to obtain rich image data. Use LiDAR and other equipment to collect point cloud data of the target area. Collate the collected image data and point cloud data, categorize and store them according to time, shooting location, and other information to prepare for subsequent processing.

[0032] S2. Data preprocessing: De-noise the image data to remove noise caused by factors such as the shooting environment and equipment; enhance blurred images to improve image clarity; remove outliers from point cloud data to ensure accuracy; streamline data to reduce data volume and improve processing efficiency; and fuse the preprocessed image data with the point cloud data to form a unified data set;

[0033] S3. Feature extraction: Extract key features for individualization from the fused data. For image data, these features include texture and color features of the object. For point cloud data, these features include geometric features such as shape, size, and spatial position. Deep learning algorithms are used to optimize and filter the extracted features to improve their recognition and representativeness.

[0034] S4. Model Segmentation: Use region growing and clustering algorithms to perform preliminary segmentation on the fused data, dividing the model into multiple approximate regions. Combined with deep learning semantic segmentation models such as U-Net and Mask R-CNN, perform fine segmentation on the initially segmented regions, accurately identify the boundaries of each object, and achieve individual segmentation of the model.

[0035] S5. Individualized model optimization: Perform geometric correction on the individualized model to correct geometric deformations caused by the model acquisition and processing process to ensure model accuracy. Optimize the model's texture to remove problems such as texture stretching and distortion to make the model's appearance more realistic. Add attribute information, such as the object's name, type, and purpose, to enrich the model's semantic information.

[0036] S6. Model Quality Assessment and Update: Develop model quality assessment indicators, such as singulation accuracy, model integrity, and texture quality. Conduct a comprehensive assessment of the singulated model and, based on the assessment results, make targeted adjustments and optimizations to any unqualified parts. Regularly collect new image data and point cloud data, and utilize incremental update technology to perform real-time or periodic updates to the singulated model to ensure consistency between the model and the real-world scenario.

[0037] For oblique photography, a multi-rotor drone equipped with a multi-lens oblique camera can be used. The flight route and shooting parameters should be reasonably planned according to the size and complexity of the target area. The LiDAR equipment should be of appropriate accuracy and range to ensure that the collected point cloud data can accurately reflect the terrain and object characteristics of the target area.

[0038] Image denoising can be done using methods such as Gaussian filtering and median filtering. Image enhancement can be done using techniques such as histogram equalization and contrast stretching. Outliers in point cloud data can be removed using statistical filtering and radius filtering. Data simplification can be achieved using voxel grid downsampling algorithms.

[0039] Texture feature extraction uses algorithms such as scale-invariant feature transformation and accelerated robust features. Color feature extraction is analyzed using RGB color space and HSV color space. Geometric feature extraction uses information such as the normal vector and curvature of the point cloud. The deep learning algorithm uses convolutional neural networks for feature optimization and is trained with a large amount of labeled data to improve the accuracy of feature extraction.

[0040] The region growing algorithm merges adjacent pixels or point clouds into a region based on similar features of objects, such as color, texture, and geometry. Clustering algorithms such as DBSCAN can be used to cluster data points based on their density distribution. During the training process, the semantic segmentation model uses a large number of real-life 3D model samples from oblique photography to continuously optimize model parameters and improve segmentation accuracy.

[0041] Geometric correction uses a control point-based correction method. Multiple corresponding control points are selected in the model and the real scene, and geometric correction is achieved through coordinate transformation. Texture optimization uses image deformation algorithms to stretch and distort textures. Attribute information can be added through manual annotation or by extracting matching information from relevant databases.

[0042] Model quality assessment indicators are reasonably set according to different application scenarios and requirements. The incremental update technology uses feature matching and model fusion algorithms to match and fuse newly collected data with the original model to achieve efficient model updating.

[0043] Example 1: Data collection and processing based on multi-rotor drones and ground-based lidar

[0044] Step 1: Data collection;

[0045] A multi-rotor drone equipped with a five-lens tilting camera was used to collect data in urban areas. Based on the distribution of urban buildings and the complexity of the terrain, the planned flight altitude was 100 meters, the route spacing was set to 50 meters, the lateral overlap was 70%, and the heading overlap was 80%. The ground-based lidar used was the RIEGLVZ-400i, with an accuracy of 5mm and a range of 400 meters, to conduct supplementary scans of ground buildings and facilities.

[0046] Step 2: Data preprocessing;

[0047] Image denoising uses Gaussian filtering, and its formula is: Where G(x,y) is a Gaussian function, σ is the standard deviation, and the σ value is adjusted according to the image noise. It is generally between 1 and 3. Image enhancement uses histogram equalization to make the image grayscale distribution uniform by adjusting the grayscale histogram of the image. The formula is: Among them, s k is the gray value after equalization, L is the total number of gray levels, n is the total number of pixels in the image, j is the number of pixels of gray level j. Statistical filtering is used to remove outliers from point cloud data. The average distance from each point to its neighboring points is calculated and compared with the set threshold to remove outliers. Data simplification is done through the voxel grid downsampling algorithm to divide the point cloud data into a uniform voxel grid, and only one representative point is retained in each voxel grid.

[0048] Step 3: Feature extraction;

[0049] Texture feature extraction uses the SIFT algorithm to extract stable texture features by constructing a scale space, detecting extreme points, and calculating feature descriptors. Color feature extraction is analyzed in the RGB color space and the RGB value distribution of each pixel is calculated. Geometric feature extraction uses the normal vector of the point cloud. The normal vector calculation formula is: in, and are two vectors on the point cloud surface, is the normal vector.

[0050] Step 4: Model segmentation;

[0051] The region growing algorithm merges adjacent pixels or point clouds into one region based on similar features of color and geometric shape. The clustering algorithm uses the DBSCAN algorithm to cluster according to the density distribution of data points. Its core formula is: N Eps (p)={q∈D|dist(p,q)≤Eps}, where N Eps(p) is the Eps domain of point p, D is the dataset, dist(p,q) is the distance between points p and q, and Eps is the domain radius. The segmentation model uses U-Net, which is trained using a large number of real-life 3D model samples of urban buildings taken through oblique photography, and the model parameters are continuously optimized.

[0052] Step 5: Optimize the monomer model;

[0053] Geometric correction uses a correction method based on control points. Multiple corresponding control points are selected in the model and the real scene, and geometric correction is achieved through coordinate transformation. The coordinate transformation formula is:

[0054] Among them, (x, y, z) is the original coordinate, (x′, y′, z′) is the corrected coordinate, and a ij is the rotation matrix element, t x ,t y ,t z It is the translation vector. Texture optimization uses image deformation algorithm to stretch and distort the texture. The addition of attribute information is achieved through manual annotation and extraction of matching information from the urban geographic information database.

[0055] Step 6: Model quality assessment and update;

[0056] The model quality assessment indicators are set at 95% individual accuracy, 90% model integrity, and texture quality with no obvious visual defects. The incremental update technology uses feature matching and model fusion algorithms to match and fuse the newly collected data with the original model to achieve efficient model updating.

[0057] Example 2: Oblique Photography and UAV LiDAR Data Processing for Mountainous Terrain

[0058] Step 1: Data collection;

[0059] A fixed-wing drone equipped with a four-lens tilt camera was selected, and the flight altitude was set to 200 meters. According to the undulating mountain terrain, the route spacing was planned to be 80 meters, with a lateral overlap of 75% and a heading overlap of 85%. The drone lidar used Velodyne VLP-16, with an accuracy of 10mm and a range of 100 meters, to scan the mountainous terrain and vegetation.

[0060] Step 2: Data preprocessing;

[0061] Image denoising uses median filtering, which sorts the pixel values ​​within the window and takes the middle value as the filtered pixel value. Image enhancement uses contrast stretching technology to enhance the contrast of the image by adjusting the grayscale range of the image. The formula is: Among them, s is the original grayscale value, s min and smax are the minimum and maximum values ​​of the original grayscale value, L is the total number of grayscale levels, radius filtering is used to remove outliers from point cloud data, a radius range is set, and points with too few points within the radius are removed. Data simplification is done through the voxel grid downsampling algorithm, and the voxel grid size is adjusted according to the density of the mountain point cloud data.

[0062] Step 3: Feature extraction;

[0063] Texture feature extraction uses the SURF algorithm, which uses integral images to quickly calculate feature points and improve feature extraction efficiency. Color feature extraction is analyzed in the HSV color space, which is more conducive to distinguishing the color features of vegetation and terrain. Geometric feature extraction uses the curvature of the point cloud. The curvature calculation formula is: in, is the normal vector, is the tangent vector, is the curve vector and s is the arc length.

[0064] Step 4: Model segmentation;

[0065] The region growing algorithm divides the terrain and vegetation in mountainous areas into regions based on texture and geometric shape similarity features. The DBSCAN algorithm is used for clustering according to the density distribution of mountain point cloud data. The semantic segmentation model uses Mask R-CNN, which is trained using a large number of real-life 3D model samples from oblique photography of mountainous areas to improve the segmentation accuracy of complex terrain and vegetation in mountainous areas.

[0066] Step 5: Optimize the monomer model;

[0067] Geometric correction adopts a correction method based on control points. Corresponding control points are selected in the mountain model and the real scene, and geometric correction is achieved through coordinate transformation. Texture optimization uses image deformation algorithm to stretch and distort the mountain texture to make it fit the terrain better. Attribute information is added through manual labeling and extraction of matching information from the mountain geographic information database, such as vegetation type, terrain and landform name, etc.

[0068] Step 6: Model quality assessment and update;

[0069] The model quality assessment indicators are set at 90% individual accuracy, 85% model integrity, and texture quality that meets the visual effects of mountainous terrain and vegetation. The incremental update technology uses feature matching and model fusion algorithms to match and fuse newly collected mountain data with the original model to achieve regular updates of the model.

[0070] Example 3: Oblique Photography and Ground Mobile LiDAR Data Processing in Urban Old Community Renovation Projects

[0071] Step 1: Data collection;

[0072] The multi-rotor drone is equipped with a three-lens tilt camera. The flight altitude is set to 80 meters for old urban communities. According to the community layout, the route spacing is planned to be 40 meters, the lateral overlap is 70%, and the heading overlap is 80%. The ground mobile lidar uses FARO Focus S350, with an accuracy of 3mm and a range of 350 meters, to scan the roads, buildings and facilities inside the community.

[0073] Step 2: Data preprocessing;

[0074] Image denoising uses a combination of Gaussian filtering and median filtering. Gaussian filtering is first used to smooth the noise, and then median filtering is used to remove isolated noise points. Image enhancement uses a combination of histogram equalization and contrast stretching to make the image clearer. Point cloud data outliers are removed by combining statistical filtering and radius filtering to improve the accuracy of point cloud data. Data simplification is achieved through a voxel grid downsampling algorithm. The voxel grid parameters are adjusted according to the characteristics of the cell point cloud data.

[0075] Step 3: Feature extraction;

[0076] Texture feature extraction uses an algorithm that combines SIFT and SURF, taking advantage of their strengths and overcoming their weaknesses to improve the accuracy of texture feature extraction. Color feature extraction uses a comprehensive analysis in RGB and HSV color spaces to more comprehensively describe the color characteristics of objects. Geometric feature extraction uses the normal vector and curvature information of the point cloud, combined with the geometric shape characteristics of community buildings and facilities for analysis.

[0077] Step 4: Model segmentation;

[0078] The region growing algorithm divides the buildings, roads and facilities within the community into regions based on similar features such as color, texture and geometric shape. The clustering algorithm adopts the DBSCAN algorithm to cluster according to the density distribution of the community point cloud data. The semantic segmentation model uses a combination of U-Net and Mask R-CNN, and is trained using a large number of real-life 3D model samples of oblique photography of old urban communities to improve segmentation accuracy.

[0079] Step 5: Optimize the monomer model;

[0080] Geometric correction adopts a correction method based on control points. Corresponding control points are selected in the community model and the real scene, and geometric correction is achieved through coordinate transformation. Texture optimization uses an image deformation algorithm to stretch and distort the community texture to make it more realistically reflect the actual situation of the community. Attribute information is added through manual labeling and extracting matching information from the community renovation-related database, such as the age of the building, renovation plan, etc.

[0081] Step 6: Model quality assessment and update;

[0082] The model quality assessment indicators are set at 95% individual accuracy, 90% model integrity, and texture quality that meets the needs of the community renovation project. The incremental update technology uses feature matching and model fusion algorithms to match and fuse the newly collected data from the community renovation process with the original model to achieve real-time model updates.

[0083] From the perspective of Example 1, Example 2 and Example 3, it can be seen that ordinary photography methods have significant advantages when used for individualization of real-scene three-dimensional models through oblique photography. In terms of data acquisition, by relying on high-resolution SLR cameras with different lenses, and adopting surround, layered and different height setting shooting methods, images can be acquired from all directions and angles to meet the needs of complex scenes; in the data processing stage, classic algorithms such as SIFT, SURF, Canny are used to accurately exert efforts in each link of preprocessing, feature extraction and model segmentation to effectively improve data quality; model optimization establishes a transformation model through control points to correct geometry, and uses a multi-resolution fusion algorithm to optimize textures and add attribute information to ensure model accuracy and practicality, and the implementation cost is low, the camera price is affordable, and the open source algorithm does not have high hardware requirements. High-quality individualized three-dimensional model construction can be achieved even under a limited budget, with extremely high cost performance.

[0084] Furthermore, the use of a high-resolution SLR camera paired with a wide-angle and telephoto lens provides great flexibility in the shooting process. The surround shooting method can record the target object from an all-round perspective, ensuring that image information from all sides is captured. For example, by shooting a group of photos every 30 degrees, the building's facade style, door and window layout, and decorative details can be fully presented. Layered shooting is designed to obtain depth information for different scenes. Starting with a long-range photo, you can quickly determine the overall scene layout, like drawing a macro map, providing a framework for subsequent shooting. Then gradually move in to take mid-range and close-up photos, like focusing with a magnifying glass, you can clearly capture the subtle details of the object, such as the carving texture and material texture on the building. Setting shooting points at different heights effectively solves the problem of obtaining object height information. When shooting high-rise buildings, shooting at different floors or using a tripod to adjust the shooting height can accurately record the structural changes at different heights of the building. The final image information obtained is comprehensive and rich, providing a solid data foundation for the subsequent construction of high-precision 3D models, which can well meet the diverse needs of complex scene modeling.

[0085] Furthermore, the image quality assessment algorithm combined with manual review and screening of photos can quickly eliminate blurred, obscured, and poor-quality photos, ensuring that the image data entering the subsequent processing flow has high availability. SIFT is combined with the RANSAC algorithm for image alignment and stitching. The SIFT algorithm can extract scale-invariant features in images and has good adaptability to rotation, scaling, brightness changes, etc. Even under different shooting angles and lighting conditions, it can accurately find corresponding feature points between images.

[0086] The RANSAC algorithm effectively removes mismatched points through random sampling and consistency verification, ensuring the accuracy of image alignment, thereby achieving high-quality image stitching and generating a complete panoramic image. In terms of feature extraction, the SURF algorithm quickly extracts texture features. Its calculation method based on integral images greatly improves the efficiency of feature extraction and can obtain a large amount of stable texture information in a short time. It is analyzed separately in multiple color spaces (RGB and HSV). The RGB space is convenient for statistically analyzing color distribution from the perspective of basic color components, while the HSV space focuses more on describing color characteristics from the aspects of hue, saturation and brightness. The combination of the two can comprehensively and deeply characterize the color characteristics of objects. The Canny edge detection algorithm is used to extract the edge contour of the object. The algorithm can accurately outline the edge of the object through steps such as Gaussian filtering to smooth the image, calculate the gradient amplitude and direction, refine the edge by non-maximum suppression, and perform double threshold detection and edge connection, providing an accurate basis for the subsequent calculation of geometric parameters such as the object's perimeter, area, and aspect ratio.

[0087] In the model segmentation stage, threshold segmentation is combined with the region growing algorithm. The appropriate threshold is set according to the extracted color and geometric features to preliminarily segment the object and background areas. The region growing algorithm starts from the seed point and merges regions based on the texture and color similarity of adjacent pixels. At the same time, constraints such as region size and shape regularity are set to avoid excessive or incorrect merging, thereby achieving accurate segmentation of objects in complex scenes and effectively improving the effect and quality of data processing.

[0088] At the same time, in the geometric correction stage of model optimization, multiple control points with known positions are carefully selected, such as the corners of buildings, intersections of roads, etc. The coordinates of these control points in the actual scene are precisely known. By measuring their coordinates in the image, an affine transformation model or a perspective transformation model is established. The affine transformation model can correct linear transformations such as translation, rotation, scaling and shearing of the image, while the perspective transformation model can handle more complex projection transformations, such as perspective effects such as near big and far small caused by shooting angles. By using these models to geometrically correct the image, the geometric deformation caused by shooting angles and lens distortion can be effectively eliminated, making the geometric shape of the model closer to the real object.

[0089] In terms of texture optimization, the multi-resolution fusion algorithm plays an important role. The algorithm decomposes the image into sub-images of different resolutions, and fuses the textures at the joints at different resolutions. Rough fusion is performed at low resolution to ensure the consistency of the overall texture, and fine fusion is performed at high resolution to retain the detailed information of the texture, so that the texture transition is natural and the clarity and quality of the texture are improved. By manually labeling and consulting relevant materials, attribute information is added to the segmented objects, such as the name, purpose, and construction year of the building, which greatly enriches the semantic information of the model, making the model not only accurate in geometry and appearance, but also with richer connotations, enhancing the practicality of the model and meeting the diverse needs of different users for model information.

[0090] It should be noted that ordinary photography methods have many shortcomings in the individualization of oblique photography real-scene three-dimensional models. In data collection, only orthophotos can be obtained, with a single angle, limited coverage and low efficiency, making it difficult to fully display complex objects and large-scale scenes. During data processing, image alignment and splicing are difficult, feature extraction is not comprehensive and accurate, and model segmentation accuracy is low. In terms of model accuracy, geometric correction and texture optimization effects are poor, and the addition of attribute information is limited, resulting in insufficient practicality of the model. Due to the above shortcomings, the application scenarios are difficult to meet the needs of fields with high requirements for accuracy and integrity.

[0091] Table 1: Data comparison of various embodiments and comparative examples

[0092]

[0093]

[0094] It can be clearly seen from the comparison of the above tables that the above content explains in many aspects the advantages of the key codes of the single-code method of oblique photography real-scene three-dimensional model compared with ordinary methods. In data collection, it can flexibly adjust equipment parameters and routes according to different scenarios, and its adaptability far exceeds that of ordinary methods. During feature extraction and model segmentation, multiple algorithms are combined to obtain comprehensive and detailed features, accurately segment objects, and make the model more detailed. The processing flow is integrated with deep learning to achieve automation, reduce manual intervention, and improve efficiency and accuracy. The model update uses incremental update technology to quickly integrate new data to ensure that the model reflects real-life changes in real time. The code structure also has good scalability, which is convenient for adding new algorithm functions to meet the needs of technological development.

[0095] Specifically, in each embodiment, key code examples and corresponding analysis are written in Python.

[0096] Furthermore, Figure 2As shown in the figure, for data preprocessing, image denoising uses methods such as Gaussian filtering and median filtering, which can effectively remove image noise while retaining image details, improving image quality, and providing a clear data basis for subsequent processing. Point cloud denoising removes outliers through statistical filtering, radius filtering, etc. to ensure the accuracy of point cloud data; the voxel grid downsampling algorithm simplifies data, reducing the amount of data while retaining the point cloud geometric features to the greatest extent, thereby improving the efficiency of subsequent processing.

[0097] Furthermore, Figure 3 As shown in the figure, during feature extraction, texture feature extraction uses algorithms such as SIFT and SURF, which can extract stable and rich texture features, have good invariance to rotation and scale changes, and have high computational efficiency, providing texture information for model segmentation. Color feature extraction comprehensively analyzes in multiple color spaces and fully describes the color characteristics of objects, which helps to distinguish different objects. Geometric feature extraction uses information such as the normal vector and curvature of the point cloud to accurately describe the geometric shape characteristics of the object, providing a geometric basis for model construction.

[0098] Furthermore, Figure 4 、 Figure 5 As shown in the figure, the model segmentation adopts the region growing algorithm based on the similarity of multiple features of point clouds, which can divide regions more accurately and segment objects in complex scenes more finely. The clustering algorithm adopts the combination of DBSCAN and hierarchical clustering, which can adapt to data with different density distributions and accurately separate different clusters of objects. The combination of semantic segmentation models such as U-Net and MaskR-CNN greatly improves the segmentation accuracy through a large number of sample training, meeting the requirements of high-precision individual segmentation tasks.

[0099] The above description is only a preferred specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any technician familiar with the technical field, within the technical scope disclosed by the present invention, who makes equivalent replacements or changes based on the technical solution and inventive concept of the present invention, should be covered by the scope of protection of the present invention.

Claims

1. A high-precision oblique photography singulation method, characterized in that: The following steps are involved: S1. Data Collection and Collation: Use oblique photography equipment to capture the target area from multiple angles to obtain rich image data. Use LiDAR and other equipment to collect point cloud data of the target area. Collate the collected image data and point cloud data, categorize and store them according to time, shooting location, and other information to prepare for subsequent processing. S2. Data preprocessing: De-noise the image data to remove noise caused by factors such as the shooting environment and equipment; enhance blurred images to improve image clarity; remove outliers from point cloud data to ensure accuracy; streamline data to reduce data volume and improve processing efficiency; and fuse the preprocessed image data with the point cloud data to form a unified data set; S3. Feature extraction: Extract key features for individualization from the fused data. For image data, these features include texture and color features of the object. For point cloud data, these features include geometric features such as shape, size, and spatial position. Deep learning algorithms are used to optimize and filter the extracted features to improve their recognition and representativeness. S4. Model Segmentation: Use region growing and clustering algorithms to perform preliminary segmentation on the fused data, dividing the model into multiple approximate regions. Combined with deep learning semantic segmentation models such as U-Net and Mask R-CNN, perform fine segmentation on the initially segmented regions, accurately identify the boundaries of each object, and achieve individual segmentation of the model. S5. Individualized model optimization: Perform geometric correction on the individualized model to correct geometric deformations caused by the model acquisition and processing process to ensure model accuracy. Optimize the model's texture to remove problems such as texture stretching and distortion to make the model's appearance more realistic. Add attribute information, such as the object's name, type, and purpose, to enrich the model's semantic information. S6. Model quality assessment and update: Develop model quality assessment indicators, such as individualization accuracy, model integrity, texture quality, etc., conduct a comprehensive assessment of the individualized model, and make targeted adjustments and optimizations to unqualified parts based on the assessment results. Regularly collect new image data and point cloud data, and use incremental update technology to update the individualized model in real time or regularly to ensure consistency between the model and the real scene.

2. The high-precision oblique photography singulation method according to claim 1, characterized in that: In step S1, the oblique photography equipment can be a multi-rotor drone equipped with a multi-lens oblique camera. The flight route and shooting parameters are reasonably planned according to the size and complexity of the target area. The lidar equipment selects appropriate accuracy and range to ensure that the collected point cloud data can accurately reflect the terrain and object characteristics of the target area.

3. The high-precision oblique photography singulation method according to claim 1, characterized in that: In step S2, image denoising can be performed using methods such as Gaussian filtering and median filtering, image enhancement can be performed using techniques such as histogram equalization and contrast stretching, outliers in point cloud data can be removed using methods such as statistical filtering and radius filtering, and data simplification can be achieved through a voxel grid downsampling algorithm.

4. The high-precision oblique photography singulation method according to claim 1, characterized in that: In step S3, texture feature extraction adopts algorithms such as scale-invariant feature transformation and accelerated robust feature, color feature extraction is analyzed through RGB color space, HSV color space, etc., geometric feature extraction uses information such as normal vector and curvature of point cloud, and deep learning algorithm uses convolutional neural network for feature optimization, which is trained through a large amount of labeled data to improve the accuracy of feature extraction.

5. The high-precision oblique photography singulation method according to claim 1, characterized in that: In step S4, the region growing algorithm merges adjacent pixels or point clouds into one region based on similar features of objects, such as color, texture, and geometric shape. The clustering algorithm can use algorithms such as DBSCAN to cluster according to the density distribution of data points. During the training process, the semantic segmentation model uses a large number of oblique photography real-scene 3D model samples for training, continuously optimizes model parameters, and improves segmentation accuracy.

6. The high-precision oblique photography singulation method according to claim 1, characterized in that: In step S5, geometric correction adopts a correction method based on control points, selects multiple corresponding control points in the model and the real scene, and implements geometric correction through coordinate transformation. Texture optimization utilizes an image deformation algorithm to stretch and distort the texture. The addition of attribute information can be achieved through manual annotation or extracting matching information from a relevant database.

7. The high-precision oblique photography singulation method according to claim 1, characterized in that: In step S6, the model quality evaluation index is reasonably set according to different application scenarios and requirements. The incremental update technology adopts feature matching and model fusion algorithms to match and fuse the newly collected data with the original model to achieve efficient model update.

Images