Man-machine close photography data processing and three-dimensional reconstruction method, system and device and storage medium

Abstract

The invention discloses a man-machine close photography data processing and three-dimensional reconstruction method, system and device and a storage medium, and belongs to the technical field of three-dimensional modeling, and the method comprises the steps: obtaining close photography image data and space auxiliary positioning data of a target area, constructing an original data set for image registration and three-dimensional modeling, and generating three-dimensional model data; a digital orthoimage is generated, texture mapping operation is executed according to camera azimuth parameters, and a three-dimensional model grid with textures is generated; performing data field binding on the structured observation data and the three-dimensional model grid, and constructing an observation data index structure based on the spatial position of a model patch to form a spatial expression model; and defining an interface data format and an access protocol parameter for the field structure and the index system to complete data access interface registration. According to the method, the problem of difficulty in spatial organization and unified management of multi-source data in the prior art is solved, and the adaptability and practicability of the model in tasks such as monitoring, early warning and visual analysis are improved.

Description

Technical Field

[0001] This invention relates to the field of 3D modeling technology, specifically to a method, system, device, and storage medium for human-machine close-up photographic data processing and 3D reconstruction. Background Technology

[0002] Currently, in scenarios such as water conservancy project monitoring, safety inspection, and geographic information system (GIS) construction, relying on traditional terrestrial photogrammetry or high-altitude remote sensing imagery to acquire 3D geographic information models can no longer meet the comprehensive requirements for data accuracy, close-range coverage, and rapid modeling. On the one hand, traditional high-altitude remote sensing data suffers from insufficient resolution and loss of texture details, and is difficult to cover obscured areas and complex terrain structures; on the other hand, image data acquired through ground station deployment has a long acquisition cycle and low efficiency, making it difficult to adapt to the rapid response needs in large-scale or emergency scenarios.

[0003] Meanwhile, in the process of 3D modeling, image data and spatial observation data often suffer from inconsistent formats and spatiotemporal mismatch, causing a disconnect between modeling data and subsequent analysis. This severely restricts the application expansion of 3D models in simulation analysis, information fusion, and intelligent recognition. Therefore, there is an urgent need for a systematic technical method that combines UAV close-up photography, multi-angle imaging, spatial positioning calculation, and observation data integration to improve model accuracy, accommodate multi-source information, and meet timeliness requirements. Summary of the Invention

[0004] To address the aforementioned technical issues, a method for human-machine close-up photography data processing and 3D reconstruction is proposed, which includes acquiring close-up photography image data and spatial auxiliary positioning data of the target area, and constructing an original dataset with a multi-view structure.

[0005] Image registration and 3D modeling are performed on the original dataset to generate 3D model data with a spatial reference coordinate system;

[0006] A digital orthophoto is generated based on the 3D model data, and a texture mapping operation is performed according to the camera orientation parameters to generate a textured 3D model mesh.

[0007] Structured observation data is bound to the 3D model mesh for data field binding, and an observation data index structure is constructed based on the spatial location of model patches to form a spatial representation model;

[0008] Based on the field structure and indexing system of the spatial representation model, the interface data format and access protocol parameters are defined to complete the registration of the data access interface.

[0009] As a preferred embodiment of the human-machine close-up photography data processing and 3D reconstruction method described in this invention, the step of acquiring close-up photography image data and spatial auxiliary positioning data of the target area, and constructing an original dataset with a multi-view structure includes...

[0010] By using a drone platform equipped with oblique and orthophoto cameras, the target area is closely surveyed and mapped, acquiring oblique and vertical image data containing several shooting angles.

[0011] The corresponding three-dimensional position information is obtained by using an airborne GNSS receiver, and the attitude parameters consisting of pitch angle, yaw angle and roll angle are obtained by using an attitude calculation module.

[0012] The tilted images, vertical images, GNSS location information, and attitude parameters are then aligned and organized according to a unified timestamp to construct a close-up photographic raw dataset for 3D reconstruction.

[0013] As a preferred embodiment of the human-machine close-up photography data processing and 3D reconstruction method described in this invention, the step of performing image registration and 3D modeling on the original dataset to generate 3D model data with a spatial reference coordinate system includes:

[0014] Multi-view image matching is performed based on oblique and vertical images to extract corresponding feature points between images, and the relative camera pose matrix is ​​obtained by using structured bundle method or sparse reconstruction algorithm.

[0015] By combining the 3D position information obtained from GNSS with the heading, pitch, and roll angles obtained from attitude calculation, georeferenced transformation is performed on the camera pose to complete the spatial coordinate system one;

[0016] Under a unified spatial reference, dense point cloud reconstruction is performed to obtain spatial point cloud data covering the target area;

[0017] Noise filtering, boundary repair, and topological meshing are performed on the point cloud data to generate a three-dimensional mesh model with actual scale and geometric topology, thus forming the three-dimensional model data.

[0018] As a preferred embodiment of the human-machine close-up photography data processing and 3D reconstruction method described in this invention, the step of generating a digital orthophoto based on the 3D model data and performing a texture mapping operation according to the camera orientation parameters to generate a textured 3D model mesh includes:

[0019] Based on the orthophotos acquired by vertical photography and the 3D model, a spatial correspondence between the image and the model surface is established. The coordinates of the intersection point between the central ray of the image and the ground surface are calculated by using aerial photography solution parameters. The model surface is then projected onto the imaging plane for correction and transformation to generate a digital orthophoto in the corresponding coordinate system.

[0020] By utilizing the spatial index structure of the model mesh and the exterior orientation parameters of the camera, per-facet texture matching and mapping are performed to map the corresponding pixels in the oblique photographic image to the facets of the 3D model. The mapping relationship is encoded into a texture mapping index table and embedded into the topological data structure of the 3D model mesh.

[0021] As a preferred embodiment of the human-machine close-up photography data processing and 3D reconstruction method described in this invention, the method involves binding structured observation data with 3D model mesh data fields, and constructing an observation data index structure based on the spatial location of model patches to form a spatial representation model, which supports subsequent model calls or analysis.

[0022] Based on the spatial geometry of the three-dimensional mesh model, a unique spatial index numbering system is established. The water level observation data generated by GNSS-R reflection signals, the water level change data obtained by the three-frequency phase combination inversion, the temperature, humidity and wind speed data collected by meteorological sensors, the pore pressure and seepage rate data inside the dam collected by the seepage pressure monitoring instrument, and the intrusion detection event data output by the video intelligent recognition module are bound to the fields according to their corresponding spatial locations and timestamps.

[0023] A spatial database structure is used to encapsulate information in the 3D mesh model, and a reserved field is established for the model call interface through data structure nesting for input calls in analysis tasks.

[0024] As a preferred embodiment of the human-machine close-up photography data processing and 3D reconstruction method described in this invention, the step of defining the interface data format and access protocol parameters based on the field structure and indexing system of the spatial representation model, and completing the data access interface registration includes,

[0025] Based on the spatial index field, time field, and data type field set in the spatial representation model, a unified data structure template is designed as the data parameter standard for the access interface.

[0026] The GNSS-R water level observation data, three-frequency phase combination inverted water level data, environmental meteorological data, and video event data bound in the 3D model are encapsulated into data service units according to a unified template, and their access semantics and data boundaries are defined.

[0027] The service encapsulation method based on the REST protocol is adopted to register an independent interface path for each type of data service unit, and to define the access method (such as GET / POST), request parameter structure and response content format.

[0028] An interface configuration table is constructed to configure the calling permissions of interface paths, interface group tags, role access control, and cross-module call mapping relationships, thereby realizing a unified registration and permission management mechanism for interface services. This is a preferred solution for a human-machine close-up photography data processing and 3D reconstruction method described in this invention, wherein: the spatial expression model also includes dynamic data updating and temporal organization functions;

[0029] Configure data update interfaces for the GNSS-R reflected water level data, three-frequency phase combined water level data, environmental meteorological data, seepage pressure monitoring data and video event data bound in the spatial representation model, and define the corresponding data refresh cycle, data effective time window and historical version retention strategy;

[0030] Construct an observation data organization structure oriented towards time-series access, store all bound data according to a unified time field index, and support time-period query and historical version backtracking through a two-level data index structure;

[0031] When any type of observation data is updated through the interface, the system automatically reconstructs the corresponding index relationship and updates the mapping structure in the spatial representation model to maintain the timeliness and consistency of the model data.

[0032] Another objective of this invention is to provide a human-machine close-up photography data processing and 3D reconstruction system. This invention solves the problem of spatial organization and unified management of multi-source data in the prior art, and improves the adaptability and practicality of the model in tasks such as monitoring, early warning and visualization analysis.

[0033] As a preferred embodiment of the human-machine close-up photography data processing and 3D reconstruction system of the present invention, it is characterized by including a data acquisition module, a 3D modeling module, an observation data fusion module, and an interface service management module.

[0034] The data acquisition module controls the UAV platform to carry oblique and orthophoto cameras to perform close-range flight mapping of the target area, collect oblique and vertical images from multiple angles, and simultaneously receive position information obtained by the GNSS receiver and pitch, yaw and roll angle data output by the attitude calculation module, and perform unified timestamp alignment and organization to construct a multi-view structured original dataset.

[0035] The 3D modeling module performs multi-view image matching on the original dataset, extracts corresponding feature points, calculates camera pose and performs georeference transformation, and reconstructs dense point clouds and generates 3D meshes under a unified spatial reference system. The 3D modeling module integrates a pose calculation module to provide camera pose parameters, and nests a texture mapping submodule to map oblique photographic images to 3D model mesh patches through camera orientation parameters, constructs a visualized 3D model and generates a texture mapping index table.

[0036] The observation data fusion module is based on a spatial indexing system of a three-dimensional mesh model. It binds the water level observation data generated by GNSS-R reflection signals, the water level change data retrieved by the three-frequency phase combination inversion, the environmental data from meteorological sensors, the seepage pressure and flow velocity data obtained by the seepage pressure monitor in the dam body, and the event data output by the video intelligent recognition module to the corresponding spatial units and time fields, and encapsulates them into structured data objects and embeds them into the three-dimensional model to complete dynamic updates and historical version index maintenance.

[0037] The interface service management module constructs spatial index fields, time fields, and data type identifier fields based on a unified data structure template, encapsulates various observation data into accessible data service units, registers independent interface paths using an interface encapsulation method based on the REST protocol, sets request parameters and response format specifications, and manages interface groups, calling permissions, and cross-module service relationships through an interface configuration table to complete data service calls between the spatial representation model and external application modules.

[0038] A computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the human-machine close-up photography data processing and three-dimensional reconstruction method.

[0039] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the human-machine close-up photography data processing and three-dimensional reconstruction method.

[0040] The beneficial effects of this invention are as follows: Based on a multi-angle imaging device and spatial positioning device mounted on an unmanned aerial vehicle (UAV), this invention acquires multi-view close-up image data and high-precision positioning information covering the target area; it generates a 3D model with spatial reference through image registration and dense modeling, and constructs a unified expression structure bound to structured observation data; further, it designs a standardized interface to realize data interaction between the model and external task systems. This invention solves the problems of inefficient organization of close-up images under complex terrain conditions, difficulties in spatial alignment of model construction, insufficient coupling of multi-source data, and lack of subsequent analysis interfaces. It has the technical advantages of orderly image organization, high modeling accuracy, strong data fusion capability, and good integrability. Attached Figure Description

[0041] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0042] Figure 1 This is a flowchart illustrating a human-machine close-up photography data processing and 3D reconstruction method according to an embodiment of the present invention. Detailed Implementation

[0043] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of the present invention.

[0044] Example 1, referring to Figure 1 This is the first embodiment of the present invention, which provides a method for human-machine close-up photography data processing and 3D reconstruction, including:

[0045] S1 acquires close-up photographic image data and spatially assisted positioning data of the target area to construct a raw dataset with a multi-view structure.

[0046] By using a drone platform equipped with oblique and orthophoto cameras, the target area is closely surveyed and mapped, acquiring oblique and vertical image data containing several shooting angles.

[0047] The corresponding three-dimensional position information is obtained by using an airborne GNSS receiver, and the attitude parameters consisting of pitch angle, yaw angle and roll angle are obtained by using an attitude calculation module.

[0048] The tilted images, vertical images, GNSS location information, and attitude parameters are then aligned and organized according to a unified timestamp to construct a close-up photographic raw dataset for 3D reconstruction.

[0049] The 3D visualization management platform for reservoir dams consists of the following layers:

[0050] Data Acquisition Layer: This layer uses a sensor network to monitor various indicators of the reservoir dam in real time, including temperature, water level, and pressure. The sensor network then transmits the collected data to the data processing layer.

[0051] Data processing layer: This layer performs real-time analysis and processing of the collected data, including data cleaning, data fusion, and anomaly detection. Simultaneously, by establishing a mathematical model of the reservoir dam, real-time data is combined with model data to increase the model's accuracy.

[0052] Digital Twin Model Layer: Utilizing mathematical modeling and simulation techniques, the structure and operational status of the reservoir dam are digitized to form a digital twin model. This model reflects the real-time status of the reservoir dam and provides predictive and optimization analysis functions.

[0053] User Interface Layer: A 3D visualization interface presents the digital twin model to management personnel, enabling real-time monitoring and management of the reservoir dam. Management personnel can use the interface to perform data queries, alarm handling, and emergency response simulations.

[0054] The 3D visualization management platform for reservoir dams includes the following functional modules for real-time monitoring of the safety conditions around the reservoir dam:

[0055] (1) Smart Security: By using video surveillance and image recognition technology, when abnormal situations are detected, alarms are triggered in a timely manner and corresponding measures are taken. At the same time, through intelligent analysis algorithms, historical data is mined and analyzed to provide early warnings and risk assessments of security risks.

[0056] (2) Emergency Response Plan Simulation Demonstration: Based on a digital twin model, various accident scenarios of reservoir dams are simulated to demonstrate the effectiveness of emergency response plans. Managers can formulate, practice, and evaluate plans on the platform to improve their ability to respond to emergencies.

[0057] (3) Environmental monitoring: Environmental parameters around the reservoir dam, including temperature, humidity, and wind speed, are monitored in real time using equipment such as meteorological stations and water quality sensors. Through data analysis and model prediction, an assessment and early warning of the impact of environmental changes on the safety of the reservoir dam are provided.

[0058] (4) Intelligent Inspection: Utilizing intelligent equipment such as robots and drones, various parts of the reservoir dam are inspected and maintained. Through image recognition and sensor data analysis, the dam structure is monitored and evaluated, and inspection reports and maintenance recommendations are provided.

[0059] (5) Reservoir area flight positioning observation: Using aircraft such as drones to conduct flight positioning observation of the reservoir area of ​​the reservoir dam, and through the analysis of aerial images and remote sensing data, information on the vegetation and soil stability of the reservoir area is provided, which provides a basis for the environmental management and disaster prevention of the reservoir dam.

[0060] Building a 3D visualization management platform for reservoir dams using digital twin technology can improve dam safety, reduce operating costs, and provide a scientific basis for dam construction and maintenance decisions. With the continuous development and application of digital twin technology, reservoir dam management will usher in a more intelligent, efficient, and sustainable development era.

[0061] S2. Perform image registration and 3D modeling on the original dataset to generate 3D model data with a spatial reference coordinate system.

[0062] By using a drone platform equipped with oblique and orthophoto cameras, the target area is closely surveyed and mapped, acquiring oblique and vertical image data containing several shooting angles.

[0063] The corresponding three-dimensional position information is obtained by using an airborne GNSS receiver, and the attitude parameters consisting of pitch angle, yaw angle and roll angle are obtained by using an attitude calculation module.

[0064] The tilted images, vertical images, GNSS location information, and attitude parameters are then aligned and organized according to a unified timestamp to construct a close-up photographic raw dataset for 3D reconstruction.

[0065] In a preferred embodiment of the present invention, image registration and 3D modeling are performed on the original dataset to generate 3D model data with a spatial reference coordinate system.

[0066] 3D modeling mainly consists of two parts. One part involves modeling the various water conservancy objects within the reservoir management and protection area, primarily used to reflect the 3D outline of the project and enhance the sense of depth and realism of the engineering scene. The other part involves modeling the dam, hydraulic structures in the hub area, and electromechanical equipment. This part requires creating detailed models that accurately reproduce the size, shape, details, and colors of the actual buildings according to their proportions.

[0067] The reservoir management and protection area adopts an L2-level base plate, with base plate data acquired through satellite remote sensing imagery or UAV aerial photography. The DEM grid size is better than 15 meters, and the DOM resolution is better than 1 meter. Models of the reservoir dam and hydraulic structures in the key area are constructed using oblique photogrammetry combined with 3D laser scanning and other technologies, achieving a resolution better than 5 cm. The electromechanical equipment of the gate and valve chambers is modeled at LOD 3.0 level. The dynamic display of the reservoir dam's operation process is achieved through 2D and 3D map spatial visualization.

[0068] (1) Digital Elevation Model (DEM) of Reservoir Management and Protection Area

[0069] 1) Establishment Method

[0070] There are several methods for creating a DEM. These can be categorized by data source and acquisition method:

[0071] ① Data is obtained directly using instruments such as GPS and total station;

[0072] ②Acquired through photogrammetry based on aerial or space imagery, such as stereo coordinate instrument observation and aerial triangulation, analytical mapping, digital photogrammetry, etc.

[0073] ③ The adoption of automatic 3D reconstruction technology greatly reduces the cost of 3D data processing and improves the frequency and timeliness of data updates;

[0074] ④ Through 3D technologies such as oblique photography, BIM, laser point cloud, and multi-source heterogeneous data fusion, it supports the import, processing, and publication of various types of data, quickly constructs spatiotemporal data, and provides convenient GIS services for various industries;

[0075] ⑤ Through technologies such as lightweighting, LOD, and CPU parallel computing, we can quickly load massive amounts of data, giving you a smooth online 3D experience;

[0076] ⑥ Collect data from existing topographic maps, such as grid reading, manual tracking with a digitizer, and semi-automatic data collection with a scanner, and then generate a DEM through interpolation.

[0077] (7) It provides full-process services for massive spatiotemporal big data, including data acquisition, modeling, storage, analysis and intelligent applications; it adopts technologies such as distributed storage, real-time stream processing and big data fusion to improve the throughput of massive data and achieve orders-of-magnitude performance improvement. It provides rich analysis and visualization modules to meet all-round 3D GIS applications and services.

[0078] 2) Form.

[0079] There are various ways to organize and express data in digital elevation models, among which regular rectangular grids and irregular triangular grids are commonly used in land use engineering.

[0080] ① Regular rectangular grid. A regular rectangular grid is a dataset of planar coordinates (z, y) and equations (z) of topographic points arranged at equal intervals along the z and y axes on a Gaussian projection platform. The planar coordinates of any point P_{i, j} can be calculated based on the row and column numbers i and j of that point in the DEM and the basic information stored in the DEM file. The advantages of a rectangular grid DEM are its small storage size, compressible storage, and ease of use and management. It is suitable for small areas with minimal topographic variation.

[0081] ② Irregular Triangular Network (TIN). An Irregular Triangular Network (TIN) is a DEM represented by an irregular triangular network. Because each point constituting the TIN is original data, interpolation accuracy loss is avoided. Therefore, TINs can better estimate the feature points and lines of the terrain, and represent complex terrain more accurately than rectangular grids. However, TINs have a large data volume; in addition to storing their three-dimensional coordinates, the topological relationships of the network points must also be established. They are generally used for obtaining numerical values ​​through large-scale aerial surveying.

[0082] The commonly used algorithm is TIN. The advantages of TIN structured data are that it can describe surface morphology at different levels of resolution. Compared to grid data models, TIN models can represent more complex surfaces more accurately with less space and time at a specific resolution. Especially when the terrain contains a large number of features such as fault lines and tectonic lines, TIN models can better take these features into account.

[0083] 3) Data source.

[0084] In terms of data sources and collection methods, there are:

[0085] ① Data is obtained directly using instruments such as GPS and total station;

[0086] ②Acquired through photogrammetry based on aerial or space imagery, such as stereo coordinate instrument observation and aerial triangulation, analytical mapping, digital photogrammetry, etc.

[0087] ③ Collect data from existing topographic maps, such as grid reading, manual tracking with a digitizer, and semi-automatic data collection with a scanner, and then generate a DEM through interpolation.

[0088] 4) Resolution:

[0089] DEM resolution is a crucial indicator of the accuracy of a DEM's terrain depiction and a major factor determining its applicability. DEM resolution refers to the length of the smallest cell in the DEM. Because DEM data is discrete, the (X, Y) coordinates are represented by small squares, each marking its elevation. The length of this small square is the DEM's resolution. A smaller resolution value indicates higher resolution and more accurate terrain depiction, but the data volume also increases exponentially. Therefore, the creation and selection of a DEM must be based on a balance between accuracy and data volume, depending on the specific needs.

[0090] 5) Application.

[0091] Since DEM describes ground elevation information, it serves as the foundation for hydrological analyses such as catchment area analysis, river network analysis, rainfall analysis, flood storage calculation, and inundation analysis in flood prevention and disaster reduction.

[0092] (2) Digital orthophoto (DOM) of reservoir management and protection area.

[0093] Digital orthophoto (DOM): Image data generated by cropping aerial photographs and remote sensing images according to map sheet range after pixel correction. It is an image that simultaneously possesses map geometric accuracy and image features. It has high data accuracy, rich information, intuitive and realistic appearance, and is quick to obtain. It can be used as background control information for map analysis, and can also extract historical or latest information on natural resources and socio-economic development, providing a reliable basis for applications such as disaster prevention and public facility construction planning. It can also extract and derive new information to realize map revision and updating.

[0094] The main drawbacks of DOM are: low efficiency, slow parsing speed, and excessive memory consumption. It is almost impossible to use for large files. In addition, the low efficiency is also reflected in the large amount of time consumed.

[0095] (3) BIM model of reservoir dam and hub area.

[0096] Currently, modeling techniques are mainly divided into two types: one is to acquire real-world modeling data using drones equipped with digital cameras or LiDAR, and the other is to perform manual digital modeling using relevant 3D modeling software. The former is efficient and meets general accuracy requirements, but it is insufficient for delicate equipment such as electromechanical equipment, which necessitates supplementary manual modeling.

[0097] UAV oblique photogrammetry 3D modeling technology is a new surveying and mapping technique developed based on UAV oblique photogrammetry technology and photogrammetry theory. This technology utilizes a UAV equipped with a digital camera to collect multi-angle, omnidirectional, and full-coverage 3D data of a target object, constructing a 3D reality model, and optimizing the model to achieve rapid and efficient digital mapping. UAV 3D modeling technology is suitable for large areas where high precision is not required.

[0098] 3D modeling refers to the use of 3D modeling software such as Revit and 3DS Max to create a model with three-dimensional data based on the specific dimensions and spatial structure of a physical object. It is mainly used to construct models of electromechanical equipment such as valve chambers.

[0099] In one embodiment of the present invention, image registration and 3D modeling are performed on the original dataset to generate 3D model data with a spatial reference coordinate system.

[0100] By using a drone platform equipped with an oblique photography camera or LiDAR, low-altitude aerial surveys are conducted over the target area to acquire oblique image data or point cloud data with multiple perspectives. External 3D position information is obtained using a GNSS module, while camera attitude parameters (pitch angle, yaw angle, roll angle) are simultaneously acquired using a flight control system or attitude sensor. The above data are then organized and registered according to a unified timestamp to form a 3D point cloud or mesh model with preliminary spatial reference. Subsequently, the model is reconstructed and stitched together using mainstream 3D modeling tools (such as Pix4D, ContextCapture, etc.).

[0101] This alternative solution relies on readily available commercial 3D modeling software. While it is convenient to operate and has high modeling efficiency, making it suitable for modeling tasks with simple scene structures or moderate accuracy requirements, it has the following drawbacks:

[0102] For areas with dramatic elevation differences, repetitive textures, or severe equipment occlusion, the reconstruction results of oblique images exhibit local geometric distortions or modeling discontinuities; they cannot accurately reproduce the details of complex structures such as hydraulic structures and electromechanical equipment, limiting accuracy and completeness; when subsequently integrating structured observation data or applying it to precision engineering simulation analysis, the model lacks sufficient spatial constraints and data consistency support, making it difficult to meet the needs of refined scenarios; the black-box processing method of purchased modeling software limits the adjustment space of modeling parameters, which is not conducive to the optimization of customized modeling processes.

[0103] In contrast, the preferred embodiment of the present invention uses two model construction paths: "structural region modeling" and "functional equipment modeling". It uses an L2-level base plate to generate high-precision DEM and DOM, and uses remote sensing extraction, 3D laser scanning and manual fine modeling for different regions to perform fusion modeling. It also uses spatial reference coordinates for precision registration, which effectively improves the realism, accuracy and structural integrity of the model.

[0104] Therefore, the beneficial effects of this preferred technical solution are: it achieves high-precision, structurally clear, and coordinate-consistent 3D model reconstruction based on close proximity to photographic data, which is especially suitable for the realistic restoration and digital visual representation of dams, reservoirs and their auxiliary equipment, providing accurate spatial basic data support for subsequent model calling, dynamic analysis, monitoring and early warning, etc.

[0105] S3. Generate a digital orthophoto based on the three-dimensional model data, and perform a texture mapping operation according to the camera orientation parameters to generate a textured three-dimensional model mesh.

[0106] Based on the orthophotos acquired by vertical photography and the 3D model, a spatial correspondence between the image and the model surface is established. The coordinates of the intersection point between the central ray of the image and the ground surface are calculated by using aerial photography solution parameters. The model surface is then projected onto the imaging plane for correction and transformation to generate a digital orthophoto in the corresponding coordinate system.

[0107] By utilizing the spatial index structure of the model mesh and the exterior orientation parameters of the camera, per-facet texture matching and mapping are performed to map the corresponding pixels in the oblique photographic image to the facets of the 3D model. The mapping relationship is encoded into a texture mapping index table and embedded into the topological data structure of the 3D model mesh.

[0108] (1) Construction of three-dimensional scene.

[0109] The construction of 3D scenes is achieved by processing digital elevation model data (DEM), digital orthophoto data (DOM), model design data, and model photo data respectively to realize rapid 3D scene modeling. The main functions include riverbed terrain modeling, 3D scene editing, external model data conversion and input, and management of massive model data. It also provides a data source for simulation analysis operations.

[0110] Riverbed topography modeling: By combining DEM and DOM data, underwater topography, above-water topography, and even surrounding cities are modeled to generate a three-dimensional riverbed topography model.

[0111] 3D scene editing: Enables local modification and adjustment of 3D simulation scenes, adds other 3D models (such as bridges, buildings, docks, etc.) to the scene, and realizes the association and matching of ground features with external attribute information.

[0112] External model data conversion input: It can import data models (navigation marks, monitoring equipment, etc.) modeled by other tools (such as 3DMax) into the 3D scene.

[0113] Management of massive model data: Smooth navigation of massive model data is helpful for the management of 3D models of long river sections.

[0114] S4 binds structured observation data to the 3D model mesh with data fields, and constructs an observation data index structure based on the spatial location of model patches to form a spatial representation model.

[0115] Based on the spatial geometry of the three-dimensional mesh model, a unique spatial index numbering system is established. The water level observation data generated by GNSS-R reflection signals, the water level change data obtained by the three-frequency phase combination inversion, the temperature, humidity and wind speed data collected by meteorological sensors, the pore pressure and seepage rate data inside the dam collected by the seepage pressure monitoring instrument, and the intrusion detection event data output by the video intelligent recognition module are bound to the fields according to their corresponding spatial locations and timestamps.

[0116] A spatial database structure is used to encapsulate information in the 3D mesh model, and a reserved field is established for the model call interface through data structure nesting for input calls in analysis tasks.

[0117] Configure data update interfaces for the GNSS-R reflected water level data, three-frequency phase combined water level data, environmental meteorological data, seepage pressure monitoring data and video event data bound in the spatial representation model, and define the corresponding data refresh cycle, data effective time window and historical version retention strategy;

[0118] Construct an observation data organization structure oriented towards time-series access, store all bound data according to a unified time field index, and support time-period query and historical version backtracking through a two-level data index structure;

[0119] When any type of observation data is updated through the interface, the system automatically reconstructs the corresponding index relationship and updates the mapping structure in the spatial representation model to maintain the timeliness and consistency of the model data.

[0120] In a preferred embodiment of this invention, structured observation data is bound to a 3D model mesh, and an observation data index structure is constructed based on the spatial location of model patches to form a spatial representation model.

[0121] Based on the spatially registered 3D model mesh, spatial encoding is performed on the facet units that make up the mesh to construct a spatial index system with unique identifiers. This index system uses the geometric center coordinates, normal vector information, and topological relationships of the facets as core parameters to generate coded identifiers that are strongly correlated with spatial locations, serving as the basis for subsequent data binding and rapid retrieval.

[0122] Based on this, and in response to the needs of different parts and functional zones of the monitoring objects, the water level observation data obtained by GNSS-R reflection signal inversion, the water level change data solved by three-frequency phase combination, environmental meteorological elements (temperature, humidity, wind speed), the pore pressure and seepage rate data obtained by the seepage pressure monitoring instrument inside the dam body, and the intrusion alarm event data extracted by the video intelligent recognition module are collected according to their corresponding spatial location, equipment number and timestamp.

[0123] Then, through field mapping, the key fields (such as time, measurement value, sensor ID, observation type, etc.) in the above-mentioned structured observation data are bound at the field level to the patches with spatial index coding in the 3D model, forming a data-model mapping relationship containing structured semantics.

[0124] Furthermore, a spatial database (such as PostGIS or a custom BIM spatial database) is used for encapsulation and modeling in the model data storage layer. Nested data table structures are embedded in the model structure to support the mounting and partitioned storage of high-frequency observation data. At the same time, analysis interface fields are reserved for each spatial entity or patch node to support parameterized calls for subsequent analysis tasks such as seepage field simulation, abnormal state identification, or emergency plan simulation.

[0125] The preferred embodiment ensures spatial consistency, temporal consistency, and semantic consistency between observation data and model entities, realizing the integrated integration of the model's visual structure and actual observation information, and providing underlying support for building a unified, dynamically updated digital twin.

[0126] In this invention, an optional embodiment involves binding structured observation data with the 3D model mesh and constructing an observation data index structure based on the spatial location of model patches to form a spatial representation model. This is achieved by establishing attribute mounting rules oriented towards entity objects, using a 3D model built with BIM software (such as Revit or Navisworks), and mapping externally acquired sensor data files to the model component attribute fields in batches. Specifically, this includes manually associating multi-source observation data fields such as water level, meteorological data, and seepage pressure according to the building structural component ID or name, and attaching the observation time series as an external table file to the corresponding component's extended attribute tags. If the component name and sensor installation location are inconsistent, the binding is corrected by manually comparing the drawings and model numbers.

[0127] After data binding, by enabling timeline controls or script tools in the BIM software, the time-series changes of key components can be simulated, supporting data linkage display and analysis in limited scenarios. The model call interface mainly relies on BIM platform plugins or external APIs for access; the interface structure lacks a unified standard design, resulting in weak cross-system call capabilities.

[0128] Compared to the spatial encoding mechanism and structured field-level binding method based on three-dimensional mesh patches in the preferred embodiment of the present invention, the optional embodiment has the following disadvantages:

[0129] Data binding relies on component IDs or manual comparison, lacking precise geometric spatial indexes, making it difficult to support high-precision positioning and patch-level physical modeling; attribute data exists in component fields in the form of nested tables, making updates inconvenient and difficult to support high-frequency dynamic data inflow; the interface structure is closed, making it difficult to adapt to unstructured or heterogeneous observation data sources such as video recognition modules and GNSS-R signals; when the model is large or the amount of observation data is large, the rendering and linkage efficiency is low, failing to meet the requirements of real-time interaction and analysis tasks.

[0130] It should be further explained that:

[0131] The current preferred embodiment constructs a spatial indexing system at the mesh level of the 3D model, binding multi-source observation data by spatial location and timestamp, and encapsulating it in a nested spatial database format. This provides a unified, location-accurate, and formatted input interface for subsequent data retrieval and analysis tasks. Compared to alternative solutions that bind data solely based on component attributes, this embodiment offers significant advantages in spatial accuracy, data fusion completeness, and interface standardization. It effectively enhances the computability, interactivity, and multi-scenario adaptability of the 3D model, thereby better supporting monitoring and analysis needs in complex engineering environments.

[0132] Based on the field structure and indexing system of the spatial representation model, S5 defines the interface data format and access protocol parameters, and completes the registration of the data access interface.

[0133] Based on the spatial index field, time field, and data type field set in the spatial representation model, a unified data structure template is designed as the data parameter standard for the access interface.

[0134] The GNSS-R water level observation data, three-frequency phase combination inverted water level data, environmental meteorological data, and video event data bound in the 3D model are encapsulated into data service units according to a unified template, and their access semantics and data boundaries are defined.

[0135] The service encapsulation method based on the REST protocol is adopted to register an independent interface path for each type of data service unit, and to define the access method (such as GET / POST), request parameter structure and response content format.

[0136] An interface configuration table is constructed to configure the calling permissions of interface paths, interface group tags, role access control, and cross-module call mapping relationships, thereby realizing a unified registration and permission management mechanism for interface services.

[0137] Example 2 is the second embodiment of the present invention, which differs from the previous embodiment in that:

[0138] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0139] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0140] More specific examples of computer-readable media (a non-exhaustive list) include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which the program can be printed, because the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0141] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0142] Example 4 is the fourth embodiment of the present invention. This embodiment provides a human-machine close-up photography data processing and 3D reconstruction system, including a data acquisition module, a 3D modeling module, an observation data fusion module, and an interface service management module.

[0143] The data acquisition module controls the UAV platform to carry oblique and orthophoto cameras to perform close-range flight mapping of the target area, collect oblique and vertical images from multiple angles, and simultaneously receive position information obtained by the GNSS receiver and pitch, yaw and roll angle data output by the attitude calculation module, and perform unified timestamp alignment and organization to construct a multi-view structured original dataset.

[0144] The 3D modeling module performs multi-view image matching on the original dataset, extracts corresponding feature points, calculates camera pose and performs georeference transformation, and reconstructs dense point clouds and generates 3D meshes under a unified spatial reference system. The 3D modeling module integrates a pose calculation module to provide camera pose parameters, and nests a texture mapping submodule to map oblique photographic images to 3D model mesh patches through camera orientation parameters, constructs a visualized 3D model and generates a texture mapping index table.

[0145] The observation data fusion module is based on a spatial indexing system of a three-dimensional mesh model. It binds the water level observation data generated by GNSS-R reflection signals, the water level change data retrieved by the three-frequency phase combination inversion, the environmental data from meteorological sensors, the seepage pressure and flow velocity data obtained by the seepage pressure monitor in the dam body, and the event data output by the video intelligent recognition module to the corresponding spatial units and time fields, and encapsulates them into structured data objects and embeds them into the three-dimensional model to complete dynamic updates and historical version index maintenance.

[0146] The interface service management module constructs spatial index fields, time fields, and data type identifier fields based on a unified data structure template, encapsulates various observation data into accessible data service units, registers independent interface paths using an interface encapsulation method based on the REST protocol, sets request parameters and response format specifications, and manages interface groups, calling permissions, and cross-module service relationships through an interface configuration table to complete data service calls between the spatial representation model and external application modules.

[0147] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for human-machine close-up photography data processing and 3D reconstruction, characterized in that: include, Acquire close-up photographic image data and spatially assisted positioning data of the target area, and construct a raw dataset with a multi-view structure; Image registration and 3D modeling are performed on the original dataset to generate 3D model data with a spatial reference coordinate system; A digital orthophoto is generated based on the 3D model data, and a texture mapping operation is performed according to the camera orientation parameters to generate a textured 3D model mesh. Structured observation data is bound to the 3D model mesh for data field binding, and an observation data index structure is constructed based on the spatial location of model patches to form a spatial representation model; Based on the field structure and indexing system of the spatial representation model, the interface data format and access protocol parameters are defined to complete the registration of the data access interface.

2. The method for human-machine close-up photography data processing and 3D reconstruction as described in claim 1, characterized in that: The acquisition of close-up photographic image data and spatially assisted positioning data of the target area, and the construction of a raw dataset with a multi-view structure, includes... By using a drone platform equipped with oblique and orthophoto cameras, the target area is closely surveyed and mapped, acquiring oblique and vertical image data containing several shooting angles. The corresponding three-dimensional position information is obtained by using an airborne GNSS receiver, and the attitude parameters consisting of pitch angle, yaw angle and roll angle are obtained by using an attitude calculation module. The tilted images, vertical images, GNSS location information, and attitude parameters are then aligned and organized according to a unified timestamp to construct a close-up photographic raw dataset for 3D reconstruction.

3. The method for human-machine close-up photography data processing and 3D reconstruction as described in claim 2, characterized in that: The step of performing image registration and 3D modeling on the original dataset to generate 3D model data with a spatial reference coordinate system includes: Multi-view image matching is performed based on oblique and vertical images to extract corresponding feature points between images, and the relative camera pose matrix is ​​obtained by using structured bundle method or sparse reconstruction algorithm. By combining the 3D position information obtained from GNSS with the heading, pitch, and roll angles obtained from attitude calculation, georeferenced transformation is performed on the camera pose to complete the spatial coordinate system one; Under a unified spatial reference, dense point cloud reconstruction is performed to obtain spatial point cloud data covering the target area; Noise filtering, boundary repair, and topological meshing are performed on the point cloud data to generate a three-dimensional mesh model with actual scale and geometric topology, thus forming the three-dimensional model data.

4. The method for human-machine close-up photography data processing and 3D reconstruction as described in claim 3, characterized in that: The step of generating a digital orthophoto based on the 3D model data and performing a texture mapping operation according to the camera orientation parameters to generate a textured 3D model mesh includes: Based on the orthophotos acquired by vertical photography and the 3D model, a spatial correspondence between the image and the model surface is established. The coordinates of the intersection point between the central ray of the image and the ground surface are calculated by using aerial photography solution parameters. The model surface is then projected onto the imaging plane for correction and transformation to generate a digital orthophoto in the corresponding coordinate system. By utilizing the spatial index structure of the model mesh and the exterior orientation parameters of the camera, per-facet texture matching and mapping are performed to map the corresponding pixels in the oblique photographic image to the facets of the 3D model. The mapping relationship is encoded into a texture mapping index table and embedded into the topological data structure of the 3D model mesh.

5. The method for human-machine close-up photography data processing and 3D reconstruction as described in claim 4, characterized in that: The process involves binding structured observation data with 3D model mesh data fields and constructing an observation data index structure based on the spatial location of model patches to form a spatial representation model, which supports subsequent model calls or analysis. Based on the spatial geometry of the three-dimensional mesh model, a unique spatial index numbering system is established. The water level observation data generated by GNSS-R reflection signals, the water level change data obtained by the three-frequency phase combination inversion, the temperature, humidity and wind speed data collected by meteorological sensors, the pore pressure and seepage rate data inside the dam collected by the seepage pressure monitoring instrument, and the intrusion detection event data output by the video intelligent recognition module are bound to the fields according to their corresponding spatial locations and timestamps. A spatial database structure is used to encapsulate information in the 3D mesh model, and a reserved field is established for the model call interface through data structure nesting for input calls in analysis tasks.

6. The method for human-machine close-up photography data processing and three-dimensional reconstruction as described in claim 5, characterized in that: The field structure and indexing system based on the spatial representation model, defining the interface data format and access protocol parameters, and completing the data access interface registration include: Based on the spatial index field, time field, and data type field set in the spatial representation model, a unified data structure template is designed as the data parameter standard for the access interface. The GNSS-R water level observation data, three-frequency phase combination inverted water level data, environmental meteorological data, and video event data bound in the 3D model are encapsulated into data service units according to a unified template, and their access semantics and data boundaries are defined. The service encapsulation method based on the REST protocol is adopted to register an independent interface path for each type of data service unit, and to define the access method (such as GET / POST), request parameter structure and response content format. An interface configuration table is constructed to configure the calling permissions of interface paths, interface group tags, role access control, and cross-module call mapping relationships, thereby realizing a unified registration and permission management mechanism for interface services.

7. The method for human-machine close-up photography data processing and 3D reconstruction as described in claim 6, characterized in that: The spatial representation model also includes dynamic data updating and temporal organization functions; Configure data update interfaces for the GNSS-R reflected water level data, three-frequency phase combined water level data, environmental meteorological data, seepage pressure monitoring data and video event data bound in the spatial representation model, and define the corresponding data refresh cycle, data effective time window and historical version retention strategy; Construct an observation data organization structure oriented towards time-series access, store all bound data according to a unified time field index, and support time-period query and historical version backtracking through a two-level data index structure; When any type of observation data is updated through the interface, the system automatically reconstructs the corresponding index relationship and updates the mapping structure in the spatial representation model to maintain the timeliness and consistency of the model data.

8. A human-machine close-up photography data processing and 3D reconstruction system, employing the human-machine close-up photography data processing and 3D reconstruction method as described in any one of claims 1 to 7, characterized in that, It includes: a data acquisition module, a 3D modeling module, an observation data fusion module, and an interface service management module; The data acquisition module controls the UAV platform to carry oblique and orthophoto cameras to perform close-range flight mapping of the target area, collect oblique and vertical images from multiple angles, and simultaneously receive position information obtained by the GNSS receiver and pitch, yaw and roll angle data output by the attitude calculation module, and perform unified timestamp alignment and organization to construct a multi-view structured original dataset. The 3D modeling module performs multi-view image matching on the original dataset, extracts corresponding feature points, calculates camera pose and performs georeference transformation, and reconstructs dense point clouds and generates 3D meshes under a unified spatial reference system. The 3D modeling module integrates a pose calculation module to provide camera pose parameters, and nests a texture mapping submodule to map oblique photographic images to 3D model mesh patches through camera orientation parameters, constructs a visualized 3D model and generates a texture mapping index table. The observation data fusion module is based on a spatial indexing system of a three-dimensional mesh model. It binds the water level observation data generated by GNSS-R reflection signals, the water level change data retrieved by the three-frequency phase combination inversion, the environmental data from meteorological sensors, the seepage pressure and flow velocity data obtained by the seepage pressure monitor in the dam body, and the event data output by the video intelligent recognition module to the corresponding spatial units and time fields, and encapsulates them into structured data objects and embeds them into the three-dimensional model to complete dynamic updates and historical version index maintenance. The interface service management module constructs spatial index fields, time fields, and data type identifier fields based on a unified data structure template, encapsulates various observation data into accessible data service units, registers independent interface paths using an interface encapsulation method based on the REST protocol, sets request parameters and response format specifications, and manages interface groups, calling permissions, and cross-module service relationships through an interface configuration table to complete data service calls between the spatial representation model and external application modules.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the human-machine close-up photography data processing and three-dimensional reconstruction method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the human-machine close-up photography data processing and three-dimensional reconstruction method according to any one of claims 1 to 7.

Images