Digital twin terrain rapid construction method based on oblique photography

Abstract

The invention discloses a digital twin terrain rapid construction method based on oblique photography, and relates to the technical field of geographic model image processing, and the method comprises the steps: data collection based on oblique photography: employing an oblique photography mode to collect bidirectional terrain image data at two sides of a path in a cross manner; image processing: performing preprocessing, feature extraction and classification on the terrain image data acquired in the two directions to generate training data; model training: constructing and training a deep learning model, and identifying and classifying topographic features in the bidirectional topographic image data; digital twinborn modeling: constructing a digital elevation model and a digital surface model, and obtaining digital three-dimensional terrains on the two sides of the flight path; and result verification: combining the digital three-dimensional terrains on the two sides of the path, and verifying with an actual site. According to the method, the deep learning model is constructed and trained, and digital twin modeling is performed according to the training result, so that the construction speed and precision of the terrain model can be greatly improved.

Description

Technical Field

[0001] The present invention relates to the technical field of geographic model image processing, and in particular to a method for quickly constructing digital twin terrain based on oblique photography. Background Art

[0002] Digital twin technology has received extensive attention in recent years, and its importance is reflected in many fields such as urban management, environmental monitoring, and traffic planning. The core concept of digital twins is to reflect the state of the real world in real time in order to make effective decisions and management. In this context, digital twin terrain construction based on oblique photography has gradually become a research hotspot. Oblique photography technology provides an important basis for terrain modeling by obtaining multi-angle and multi-resolution ground images. However, traditional digital twin terrain construction methods usually include steps such as data acquisition, image processing, three-dimensional modeling, and result verification. These methods often rely on manual intervention, the processing process is complex, and the professional level of operators is required to be high. Common traditional methods include: Stereo pair measurement: Generate a three-dimensional model by matching and measuring dual-view images. Laser scanning: Use a laser rangefinder to obtain high-precision point cloud data. Ground survey: Rely on manual measurement to obtain basic terrain information.

[0003] Although traditional technologies are effective to a certain extent, they still have the following problems: low efficiency, manual operation and multiple data collections lead to overall low efficiency. High cost, high equipment and labor costs, especially in data acquisition in large areas. Precision problems, errors caused by manual intervention affect the accuracy of the model. Difficult data processing, massive data processing and analysis requires a lot of time and resources. Summary of the invention

[0004] The purpose of this invention is to construct a digital twin terrain construction method based on oblique photography to improve modeling efficiency and accuracy. By introducing oblique photography and deep learning technology, it is possible to achieve automated, high-precision terrain data processing and modeling, providing more reliable data support for urban management and decision-making.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions: A method for rapidly constructing digital twin terrain based on oblique photography, comprising: Step 1: Data collection based on oblique photography: Plan the flight path of the UAV and use oblique photography to cross-collect terrain image data in both directions on both sides of the path; Step 2: Image processing: preprocessing, feature extraction and classification of the terrain image data collected bidirectionally to generate training data; Step 3: Model training: Build and train a deep learning model to identify and classify terrain features in the bidirectional terrain image data; Step 4: Digital twin modeling: Build a digital elevation model and a digital surface model based on the results generated by the deep learning model to obtain the digital three-dimensional terrain on both sides of the flight path; Step 5: Result verification: Merge the digital 3D terrain on both sides of the path and verify it with the actual site.

[0006] Furthermore, in step 1, the data acquisition method based on oblique photography includes the following steps: Step 11. Equipment selection: Choose high-performance drones, laser scanners, and high-resolution cameras; Step 12, generating a collection path: according to the established UAV flight path, image data collection paths with the same path are set equidistantly on both sides of the UAV flight path; Step 13, data acquisition: Use a high-performance drone, laser scanner and high-resolution camera to perform cross-oblique photography on both sides of the drone flight path along the image data acquisition path, and simultaneously collect 3D scanning data.

[0007] Furthermore, in step 2, the image processing includes the following steps: Step 21, image preprocessing: performing pixel value standardization processing and grayscale conversion on the image frame; Step 22, feature extraction and classification: use edge detection algorithm to extract ground object features in image frame image data, and use deep learning model to perform feature classification on image data; Step 23: Data labeling: Use labeling tools to label the types of objects in the image data.

[0008] Furthermore, in step 3, the model training includes the following steps: Step 31: Model selection: Select deep learning architecture; Step 32, training: firstly, data segmentation is performed to divide the terrain scanning image data set in the terrain image data into a training set, a validation set and a test set, and then hyperparameter adjustment is performed to obtain the best training model; Step 33: Training evaluation: After each training cycle, evaluate the performance of the model to avoid overfitting; Step 34: Model verification and adjustment: Use different performance indicators to evaluate the effectiveness of model training and obtain verification results, and adjust and optimize the model based on the verification results.

[0009] Furthermore, in step 4, the digital twin modeling includes the following steps: Step 41, point cloud generation: extract corresponding 3D point cloud data from the image in combination with the deep learning classification result. The generation of the 3D point cloud data includes the following steps: feature point matching, matching feature points under different viewing angles through image processing algorithms; 3D reconstruction, using the corresponding relationship to generate a 3D point cloud model through triangulation method; Step 42, digital elevation model: extracting elevation information of the terrain scanning image data from the point cloud data, and generating a digital elevation model through an interpolation algorithm; Step 43, digital surface model: data fusion, combining multi-source point cloud data, fusing the ground feature data in the image frame data into the digital elevation model to construct complete surface information; Step 44, model texture mapping: texture selection, selecting the area of ​​ground feature from the original image as texture; mapping processing, mapping the selected texture according to the geometric features of the model surface.

[0010] Further, in step 5, the result verification includes the following steps: Step 51, assess the accuracy: ground control point comparison, using the ground control points with known coordinates to compare with the model results to assess the accuracy of the spatial position; spatial analysis, using the accuracy analysis tool to calculate the height and position errors; Step 52: Display verification results: Display the verification results in the form of a map.

[0011] Furthermore, in step 12, the laser scanner and the high-resolution camera are equidistantly and obliquely arranged on both sides of the drone, and the laser scanner and the high-resolution camera on both sides are arranged relatively to each other for cross-collecting oblique photography data in opposite directions.

[0012] Furthermore, the UAV flight uses GNSS data for positioning. When the UAV is flying, the laser scanner and the high-resolution camera form a collection unit distributed on the image data collection path on both sides of the UAV flight path.

[0013] Furthermore, the overlap rate of the images cross-collected by the collection units on both sides of the drone is not less than zero.

[0014] Furthermore, in step 21, the image preprocessing further includes the following steps: Step 211, image denoising: using a filtering algorithm to remove noise in the image; Step 212: Image enhancement: Improve the contrast of the image through histogram equalization.

[0015] Compared with the prior art, the method for rapidly constructing digital twin terrain based on oblique photography proposed by the present invention has the following beneficial effects: 1. The present invention uses oblique photography to cross-collect terrain image data in both directions on both sides of the path, constructs and trains a deep learning model, and establishes digital twin modeling for the terrain image data collected in both directions based on the training results, which can greatly improve the construction speed and accuracy of the terrain model.

[0016] 2. The present invention greatly reduces the manual participation process and improves data processing efficiency, thereby enabling the rapid construction of digital twin terrain.

[0017] 3. When classifying data, the present invention adopts multi-source data integration, integrates data from different sources, provides more comprehensive terrain information, and cross-collects terrain image data in both directions on both sides of the path through oblique photography to form digital three-dimensional terrain on both sides of the path, which can make the digital twin terrain more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is the overall flow chart of the method in the invention; Figure 2 The invention is a flow chart of data collection based on oblique photography in the method; Figure 3 The invention is a flowchart of image processing in the method; Figure 4 The invention is a flowchart of the model training in the method; Figure 5 A flowchart of digital twin modeling in the present method in the invention; Figure 6 It is a flow chart of the result verification in the method in the invention; Figure 7 This is a schematic diagram of cross-collection of bidirectional terrain images using oblique photography in this method. DETAILED DESCRIPTION

[0019] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0020] Reference Figure 1-Figure 7 A method for rapidly constructing digital twin terrain based on oblique photography includes five steps: data acquisition based on oblique photography, image processing, model training, digital twin modeling and result verification.

[0021] The specific process is as follows: S1. Data collection based on oblique photography: Plan the flight path of the UAV and use oblique photography to cross-collect terrain image data in both directions on both sides of the path.

[0022] Data collection based on oblique photography: using drones to collect cross-bidirectional terrain image data using methods including oblique photography. The data collection method based on oblique photography specifically includes the following steps: S11. Equipment selection: Choose high-performance drones, laser scanners, and high-resolution cameras. To ensure the quality and efficiency of data collection, choose high-performance drones, such as the DJI Phantom series or the Mavic series. The following factors should be considered when selecting equipment: high resolution, which can capture more detailed geographic features; good stability, which can still operate stably under adverse weather conditions; easy to carry, suitable for different collection environments.

[0023] S12. Generate collection paths: According to the established UAV flight path, image data collection paths with the same path are set equidistantly on both sides of the UAV flight path, at least a left image data collection path and a right image data collection path are obtained, laser scanners and high-resolution cameras are set equidistantly and obliquely on both sides of the UAV, and the laser scanners and high-resolution cameras on both sides are relatively set to cross-collect opposite oblique photography data. In order to improve the flight accuracy of the UAV, the UAV flight uses GNSS data for positioning. When the UAV is flying, the laser scanner and the high-resolution camera form a collection unit distributed on the image data collection path on both sides of the UAV flight path. When the flight path is formulated, the target area is covered as much as possible. The overlap rate of the images cross-collected by the collection units on both sides of the UAV is not less than zero, ensuring that data collection is carried out under good meteorological conditions to improve image quality, and real-time monitoring of the data collection process to ensure that all images are recorded completely without omissions; S13. Data collection: Use high-performance drones, laser scanners and high-resolution cameras to perform cross-oblique photography on both sides of the drone flight path along the image data collection path, and simultaneously collect three-dimensional scanning data. During the data collection stage, multiple data types need to be generated, including: image data, the original images taken by a high-resolution camera, three-dimensional scanning data, three-dimensional data scanned by a laser scanner, GNSS data, for extremely high positioning accuracy; bystander data, environmental information collected during the collection process.

[0024] S2, Image processing: Preprocess, extract features and classify the terrain image data collected in both directions to generate training data. The data processing stage involves basic operations such as image denoising and enhancement for subsequent analysis.

[0025] Specifically, in step S2, the image processing includes the following steps: S21, image preprocessing: performing pixel value standardization processing and grayscale conversion on the image frame; S22. Feature extraction and classification: Use edge detection algorithms to extract ground features in image frame data for subsequent classification. Use deep learning models to classify image data. Traditional machine learning methods, such as SVM and decision trees, can also be used to provide support for subsequent modeling. S23. Data labeling: Use labeling tools, such as manual labeling or semi-automatic labeling tools, to label the types of objects, such as buildings, roads, rivers, etc. This process lays the foundation for subsequent deep learning model training.

[0026] In order to further improve the image processing quality, image preprocessing also includes the following steps: S211, image denoising: using filtering algorithms, such as median filtering or bilateral filtering, to remove noise from the image; S212, Image enhancement: Improve the contrast of the image through techniques such as histogram equalization, making subsequent feature extraction more effective.

[0027] S3. Model training: Build and train a deep learning model to identify and classify terrain features in bidirectional terrain image data.

[0028] In step S3, model training includes the following steps: S31. Model selection: Choose a deep learning architecture, such as U-Net, ResNet or a more complex convolutional neural network, that can effectively handle image segmentation and classification tasks. The model selection should be based on the following points: Model processing power: the ability to learn large-scale data, robustness: performance in different scenarios, computational efficiency: the speed of training and inference; S32, training: firstly, data segmentation is performed to divide the terrain scanning image data set in the terrain image data into a training set, a validation set and a test set, and then hyperparameter adjustment is performed to adjust the learning rate, batch size and other hyperparameters according to the performance of the model to obtain the best training effect; S33, training evaluation: after each training cycle, evaluate the performance of the model to avoid overfitting; S34. Model verification and adjustment: Use different performance indicators, such as precision, recall, F1-score, etc., to evaluate the effect of model training and obtain verification results, and further adjust and optimize the model based on the verification results.

[0029] S4. Digital twin modeling: Based on the results generated by the deep learning model, a digital elevation model (DEM) and a digital surface model (DSM) are constructed to obtain the digital three-dimensional terrain on both sides of the flight path. The purpose is to achieve three-dimensional visualization of the terrain.

[0030] In step S4, digital twin modeling includes the following steps: S41, point cloud generation: Combined with the deep learning classification results, the corresponding 3D point cloud data is extracted from the image. The generation of 3D point cloud data includes the following steps: feature point matching, matching feature points under different perspectives through image processing algorithms; 3D reconstruction, using the correspondence relationship, generating a 3D point cloud model through triangulation method; S42, Digital Elevation Model (DEM): Extract elevation information of terrain scanning image data from point cloud data, and generate digital elevation model through interpolation algorithm, such as TIN or IDW. DEM can effectively reflect terrain undulations and provide a basis for subsequent analysis and application. S43, Digital Surface Model DSM: In the process of generating DSM, it is necessary to integrate different types of objects, taking into account the height information of buildings, vegetation and other objects. The generation process of DSM includes: data fusion, combining multi-source point cloud data, fusing the object feature data in the image frame data into the digital elevation model, and constructing complete surface information. The surface expression adopts appropriate algorithms, such as B-spline and NURBS for surface expression; S44. Model texture mapping: Add texture mapping to the generated digital elevation model and digital surface model to improve the visualization of the model. The steps of texture mapping are as follows: texture selection, select the area of ​​ground features from the original image as the texture, mapping processing, map the selected texture according to the geometric features of the model surface to ensure realism.

[0031] S5. Result verification: merge the digital 3D terrain on both sides of the path and verify it with the actual site. The merging is done with the planned UAV flight path as the center line, and the overlapping areas of the cross-collected images are fitted or the same areas on one side are deleted.

[0032] In step S5, the method for verifying the accuracy of the generated digital twin terrain model includes: S51. Evaluate the accuracy: compare the ground control points (GCPs), use the ground control points with known coordinates to compare with the model results, and evaluate the accuracy of the spatial position; perform spatial analysis, and calculate the height and position errors through accuracy analysis tools such as RMSE and MAE; S52. Display of verification results: Display the verification results in the form of charts or maps, provide detailed data support and visualization effects, and ensure the transparency and credibility of the results.

[0033] This method is used in many occasions, for example: 1. Support urban planning decisions through the generated digital twin terrain model. For example, analyze the impact of urban green space layout on climate and optimize traffic flow design; 2. Use the acquired terrain and object information for environmental monitoring, regularly update the model to monitor terrain changes, and support environmental protection and restoration work; 3. When natural disasters occur, quickly build an updated digital twin terrain model to provide basic data support for post-disaster recovery.

[0034] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A method for rapidly constructing digital twin terrain based on oblique photography, characterized in that: include: Step 1: Data collection based on oblique photography: Plan the flight path of the UAV and use oblique photography to cross-collect terrain image data in both directions on both sides of the path; Step 2: Image processing: preprocessing, feature extraction and classification of the terrain image data collected bidirectionally to generate training data; Step 3: Model training: Build and train a deep learning model to identify and classify terrain features in the bidirectional terrain image data; Step 4: Digital twin modeling: Build a digital elevation model and a digital surface model based on the results generated by the deep learning model to obtain the digital three-dimensional terrain on both sides of the flight path; Step 5: Result verification: Merge the digital 3D terrain on both sides of the path and verify it with the actual site.

2. According to claim 1, a method for rapidly constructing digital twin terrain based on oblique photography is characterized in that: In step 1, the data acquisition method based on oblique photography includes the following steps: Step 11. Equipment selection: Choose high-performance drones, laser scanners, and high-resolution cameras; Step 12, generating a collection path: according to the established UAV flight path, image data collection paths with the same path are set equidistantly on both sides of the UAV flight path; Step 13, data acquisition: Use a high-performance drone, laser scanner and high-resolution camera to perform cross-oblique photography on both sides of the drone flight path along the image data acquisition path, and simultaneously collect 3D scanning data.

3. The method for rapidly constructing digital twin terrain based on oblique photography according to claim 2, characterized in that: In step 2, the image processing comprises the following steps: Step 21, image preprocessing: performing pixel value standardization processing and grayscale conversion on the image frame; Step 22, feature extraction and classification: use edge detection algorithm to extract ground object features in image frame image data, and use deep learning model to perform feature classification on image data; Step 23: Data labeling: Use labeling tools to label the types of objects in the image data.

4. The method for rapidly constructing digital twin terrain based on oblique photography according to claim 3 is characterized in that: In step 3, the model training includes the following steps: Step 31: Model selection: Select deep learning architecture; Step 32, training: firstly, data segmentation is performed to divide the terrain scanning image data set in the terrain image data into a training set, a validation set and a test set, and then hyperparameter adjustment is performed to obtain the best training model; Step 33: Training evaluation: After each training cycle, evaluate the performance of the model to avoid overfitting; Step 34: Model verification and adjustment: Use different performance indicators to evaluate the effectiveness of model training and obtain verification results, and adjust and optimize the model based on the verification results.

5. The method for rapidly constructing digital twin terrain based on oblique photography according to claim 4 is characterized in that: In step 4, the digital twin modeling includes the following steps: Step 41, point cloud generation: extract corresponding 3D point cloud data from the image in combination with the deep learning classification result. The generation of the 3D point cloud data includes the following steps: feature point matching, matching feature points under different viewing angles through image processing algorithms; 3D reconstruction, using the corresponding relationship to generate a 3D point cloud model through triangulation method; Step 42, digital elevation model: extracting elevation information of the terrain scanning image data from the point cloud data, and generating a digital elevation model through an interpolation algorithm; Step 43, digital surface model: data fusion, combining multi-source point cloud data, fusing the ground feature data in the image frame data into the digital elevation model to construct complete surface information; Step 44, model texture mapping: texture selection, selecting the area of ​​ground feature from the original image as texture; mapping processing, mapping the selected texture according to the geometric features of the model surface.

6. The method for rapidly constructing digital twin terrain based on oblique photography according to claim 5, characterized in that: In step 5, the result verification includes the following steps: Step 51, assess the accuracy: ground control point comparison, using the ground control points with known coordinates to compare with the model results to assess the accuracy of the spatial position; spatial analysis, using the accuracy analysis tool to calculate the height and position errors; Step 52: Display verification results: Display the verification results in the form of a map.

7. The method for rapidly constructing digital twin terrain based on oblique photography according to claim 2, characterized in that: In step 12, the laser scanner and the high-resolution camera are arranged equidistantly and obliquely on both sides of the drone, and the laser scanner and the high-resolution camera on both sides are arranged relatively to each other for cross-collecting oblique photography data in opposite directions.

8. The method for rapidly constructing digital twin terrain based on oblique photography according to claim 7, characterized in that: The UAV is positioned using GNSS data during flight. When the UAV is flying, the laser scanner and the high-resolution camera form a collection unit that is distributed on an image data collection path on both sides of the UAV flight path.

9. The method for rapidly constructing digital twin terrain based on oblique photography according to claim 8, characterized in that: The overlapping rate of images cross-collected by the collection units on both sides of the UAV is not less than zero.

10. The method for rapidly constructing digital twin terrain based on oblique photography according to claim 3, characterized in that: In step 21, the image preprocessing further includes the following steps: Step 211, image denoising: using a filtering algorithm to remove noise in the image; Step 212, image enhancement: improve the contrast of the image through histogram equalization.

Images