A Fast Reconstruction Method for Full-Scene Highway Reconstruction Based on Multi-Source Parallax Control and Adaptive Explicit Radiation Field

Abstract

This invention discloses a rapid reconstruction method for full-scene highway reconstruction based on multi-source parallax control and adaptive explicit radiance field, belonging to the field of 3D reconstruction technology. It includes: acquiring multi-source image data and calculating camera pose using differentiated strategies for scenes such as highways, bridges, and tunnels; representing the scene with an anisotropic set of 3D Gaussian spheres, and achieving accurate 3D-to-2D mapping through the covariance projection formula; iteratively optimizing Gaussian sphere properties using an adaptive density control strategy based on gradient threshold and volume constraints, automatically cloning / splitting in detail regions to improve accuracy; and achieving real-time rendering of over 60 FPS using tile-based parallel cardinality sorting and an alpha mixing formula. This method solves the problems of high cost, near-field blurring, disconnect between tunnel interiors and exteriors, and difficulty in real-time interaction associated with traditional methods, achieving rapid modeling and incremental updates within 24 hours, significantly improving reconstruction efficiency, detail fidelity, and user experience.

Description

Technical Field

[0001] This invention relates to the field of 3D reconstruction technology, specifically to a method for rapid 3D scene reconstruction for large infrastructure scenarios such as highways, bridges, and tunnels. This method utilizes multi-view fusion technology and spatial explicit radiation field technology for data acquisition, model training, and real-time rendering. Background Technology

[0002] In the planning, construction, management, and maintenance of highway projects throughout their entire lifecycle, high-precision digital reality models are a crucial foundation for achieving visualized management and intelligent decision-making. Currently, the reconstruction of 3D scenes for large-scale infrastructure mainly relies on the following existing technologies, but they all have significant limitations in practical applications: Traditional oblique photogrammetry and 3D laser scanning technology mainly involves field data collection via UAV oblique photogrammetry or ground / airborne LiDAR, followed by industry-specific data processing, including complex steps such as point cloud generation and registration, surface mesh construction, texture mapping and texturing, to ultimately generate a 3D reality model.

[0003] However, this technology has the following drawbacks: High cost and low efficiency. This technology relies on specialized data acquisition equipment (such as LiDAR) and a lot of manual data processing. For example, modeling a 30-kilometer-long road usually requires two professionals to spend 10 to 20 days to complete, resulting in extremely high equipment and labor costs. The lack of close-up details means that although the technology can restore the overall outline of a large scene, the surface texture of the model is often blurry and lacks high-frequency details when the viewing distance is zoomed in, making it difficult to meet the needs of detecting or displaying the defects of fine structures such as road cracks and bridge supports. Difficulties in updating and maintaining: Once the scenario changes (such as the progress of construction), it is usually necessary to re-collect field data and process data in the entire area, resulting in long model update cycles, high costs, and data that is seriously lagging behind the actual situation on site.

[0004] The implicit reconstruction technique based on neural radiation fields (NeRF) uses neural networks such as multilayer perceptrons (MLP) to encode the scene as an implicit radiation field, and then synthesizes a new perspective image through volume rendering technology to generate a high-quality 3D model.

[0005] However, this technology also has significant shortcomings: Training and rendering are extremely costly. Implicit representations require massive computing resources for training, and rendering requires dense point sampling and querying along each ray, resulting in extremely low rendering frame rates and making it difficult to achieve smooth real-time interaction on ordinary hardware. It is difficult to adapt to large-scale scenarios. Neural radiation fields have extremely high requirements for graphics card memory and storage space, and lack effective representation of a large number of blank areas in the scene. When directly applied to long-distance, large-scale highway engineering scenarios, it faces huge challenges in terms of computational complexity and resource consumption.

[0006] Therefore, existing 3D reconstruction technologies for highway engineering cannot simultaneously balance reconstruction efficiency, cost, precision, real-time interaction, and scene adaptability. There is an urgent need for a new method that can quickly, cost-effectively, and with high fidelity achieve 3D reconstruction of the entire highway scene (including bridges and tunnels), and support real-time interaction and dynamic updates. Summary of the Invention

[0007] The purpose of this invention is to address the aforementioned problems by providing a rapid reconstruction method for full-scene highway construction based on multi-source parallax control and adaptive explicit radiative field. Based on the explicit radiative field construction strategy, anisotropic three-dimensional Gaussian spheres are used to represent continuous highway, bridge, and tunnel scenes. Combined with targeted data acquisition strategies and differentiable rasterization rendering technology, this method achieves low-cost, high-efficiency, and real-time interactive engineering-grade 3D real-scene reconstruction.

[0008] The technical solution of the present invention is as follows: A method for rapid reconstruction of a full-scene highway project based on multi-source parallax control and adaptive explicit radiance field includes the following steps: Data acquisition and pose calculation: Different acquisition strategies are adopted for different scenarios in highway engineering to acquire multi-source image data, and the camera pose of each frame image is calculated. 3D Gaussian scene model construction: The scene is represented by a set of anisotropic 3D Gaussian spheres, each Gaussian sphere is defined by position, covariance matrix, opacity and spherical harmonic coefficients; Model training and optimization: Based on image data and camera pose, the property parameters of a 3D Gaussian sphere are iteratively optimized until convergence through differentiable rendering and adaptive density control strategies. Real-time rendering: Employs a tile-based differential rasterization rendering pipeline to project an optimized set of 3D Gaussian spheres onto a 2D plane, generating a real-time interactive 3D scene.

[0009] By organically combining four steps—differential data acquisition, explicit 3D Gaussian representation, adaptive optimization, and real-time rendering—a complete rapid reconstruction technology system for all highway scenes has been constructed. Compared with existing technologies, this method achieves a closed-loop process from data acquisition to real-time interaction, providing a basic framework for the implementation of subsequent innovative features, and significantly improving reconstruction efficiency and lowering the implementation threshold overall.

[0010] Furthermore, in the data acquisition and pose calculation, the internal and external fusion acquisition strategy for tunnel scenarios includes: Acquire the sequence of images of the tunnel entrance collected by a drone outside the tunnel and their coordinates in the world coordinate system. First camera pose ,in Let be the rotation matrix from the world coordinate system to the camera coordinate system. For the corresponding translation vector, This is a special Euclidean group; Acquire tunnel image sequences and their local coordinate system from vehicle-mounted or handheld devices inside the tunnel. The second camera pose is below; Extract the same feature point pairs in the public area of ​​the cave entrance. , The three-dimensional coordinates of the feature point in the world coordinate system. Given the 3D coordinates of feature points in the local coordinate system, mismatched point pairs with a reprojection error greater than 2 pixels are removed. It is the number of high-precision matching points retained after removing false matches; Based on the remaining feature point pairs, the Euclidean transformation matrix from the local coordinate system to the world coordinate system is solved by singular value decomposition. And calculate the scale correction factor: , in These are the centroids of the two sets of feature points; According to the mapping formula The pose of the second camera is uniformly mapped to the world coordinate system, achieving seamless fusion of scenes inside and outside the tunnel; among which, For rotation matrix, It is a translation vector. The first tunnel in the world coordinate system The pose of the frame camera. Local coordinate system The first tunnel inside The pose of the frame camera.

[0011] The transformation matrix is ​​solved by SVD decomposition. And a scale correction factor is introduced. This solution addresses the issue of inconsistent spatial references caused by the lack of GNSS signals inside and outside the tunnel. Compared to conventional visual stitching methods, this technical solution uses rigid transformation and scale correction of common feature areas to accurately map the pose of the video stream inside the tunnel without GPS signals to a unified geographic coordinate system, achieving seamless integration of models inside and outside the tunnel and eliminating the industry pain points of model disconnect and coordinate misalignment in traditional methods.

[0012] Furthermore, for long tunnels with a length of 1 km or more, the internal and external data fusion acquisition strategy further includes: At the tunnel exit, the internal and external fusion strategy of the entrance is reused, and the common feature area of ​​the exit is extracted to construct a closed-loop constraint; Add the closed-loop constraint term to the objective function of the joint bundle adjustment optimization:

[0013] in, The first tunnel in the world coordinate system The pose of the frame camera, The camera pose is captured from outside the exit. The coordinate system transformation matrix to be solved at the exit point. This is a partial pose of the camera captured inside the exit area. It is the Frobenius norm. The weighting coefficient for the closed-loop constraint has a value range of 10 to 100 to eliminate the cumulative drift of long-mileage trajectories. For all keyframe camera poses to be optimized, Let the world coordinates of all 3D feature points to be optimized be... For the first The number of feature points observed in each keyframe This is a camera pinhole projection model, used to project 3D world coordinates onto the image pixel plane. For feature points at the th Pixel observation coordinates on the frame image For the first The spatial coordinates of a three-dimensional feature point in the world coordinate system.

[0014] By introducing closed-loop constraints and constructing a joint bundle adjustment optimization objective function in long tunnel scenarios, the cumulative drift problem of long-mileage acquisition trajectories is effectively solved. Compared with methods without closed-loop correction, this technical solution controls pose drift errors within the pixel-level range through closed-loop detection and weighted optimization of the common feature region at the exit, ensuring the geometric consistency and global accuracy of the reconstruction model of ultra-long tunnels exceeding 1 kilometer in length.

[0015] Furthermore, in the data acquisition and pose calculation, the acquisition strategy for the bridge scenario includes: The drone is controlled to orbit around the key structural nodes of the bridge, following a parametric spiral trajectory defined by the following parametric equations: , , , in, , , These are the geometric center coordinates of the key structural nodes. For the observation radius, The vertical lifting speed, Angular velocity; Real-time calculation of UAV attitude, ensuring accuracy at each sampling point. Camera optical axis and vector Overlapping is used to achieve full-dimensional data coverage of the blind spots at the bottom and sides of the bridge.

[0016] By controlling the drone's attitude using a parameterized helical equation, full-dimensional data coverage of visual blind spots such as the bridge's bottom and sides is achieved. Compared to conventional linear aerial photography, this technical solution ensures that the camera is always pointed at the main structure through centripetal constraints, enabling the complete acquisition of multi-view features of complex structural components such as piers, towers, and cables, filling data blind spots that traditional orthophotos cannot cover.

[0017] Furthermore, in the data acquisition and pose calculation, the acquisition strategy for linear highway scenarios includes: Control the drone to fly back and forth at low and high altitudes above the center line of the road, and dynamically adjust the camera pitch angle. To make it satisfy the constraints: , in, Let the principal optical axis vector of the camera be _____. , This is the local normal vector of the road surface, used to obtain the optimal anisotropic texture information.

[0018] By dynamically adjusting the camera's pitch angle Constraints were established to create optimal acquisition conditions for anisotropic texture features of the road surface. Compared with acquisition methods using a fixed pitch angle, this technical solution ensures optimal cross-parallax between the principal optical axis and the road surface through real-time calculation and feedback control of geometric relationships, significantly improving the observability and reconstruction accuracy of linear texture features such as road cracks and markings.

[0019] Furthermore, the data acquisition and pose calculation, specifically the acquisition of data inside the tunnel, includes: The data acquisition vehicle is controlled to travel at a constant speed along the centerline of the tunnel lane, and the speed meets the following constraints: , in, For camera sampling frame rate, The maximum baseline distance between adjacent frames is used to ensure that the overlap rate of adjacent frames is ≥80%, and the ratio of inter-frame baseline to scene depth is controlled in the range of 0.1~0.5.

[0020] Speed ​​constraints By controlling the ratio of inter-frame baseline to scene depth acquired inside the tunnel, the overlap rate of adjacent frames is guaranteed to be ≥80%. Compared with methods that arbitrarily control the acquisition speed, this technical solution provides sufficient longitudinal parallax for subsequent visual feature matching and 3D reconstruction through quantitative motion constraints, avoiding the loss of reconstruction accuracy due to excessively large inter-frame baseline or the data redundancy due to excessively small inter-frame baseline.

[0021] Furthermore, in the construction of the 3D Gaussian scene model, the process of projecting a 3D Gaussian sphere onto a 2D image plane is described by the covariance projection formula: , in, It is the covariance matrix of a three-dimensional Gaussian sphere. It is the view transformation matrix from the world coordinate system to the camera coordinate system. It is the affine approximation Jacobian matrix of the projective transformation. It is the projected two-dimensional covariance matrix. This is the transpose of the matrix.

[0022] Using the covariance projection formula This method achieves precise mapping from a 3D anisotropic Gaussian sphere to a 2D pixel plane. Compared to traditional point cloud projection or voxel rendering, this solution utilizes the Jacobian matrix... The affine approximation accurately describes the shape, orientation, and coverage of a three-dimensional ellipsoid on the imaging plane. It is the core mathematical foundation connecting three-dimensional geometric properties with two-dimensional visual representation, ensuring the geometric fidelity of the rendered image.

[0023] Furthermore, the initialization in the model training and optimization includes: The structure-of-motion algorithm is used to sparsely reconstruct the acquired image data, and the resulting sparse point cloud coordinates are directly used as the initial positions of the 3D Gaussian sphere. ; The scaling factor is initialized based on the comparison between the local density of the point cloud and the preset density threshold. When the local density of the point cloud is higher than the first density threshold, the scaling factor of the corresponding Gaussian sphere is initialized to the first scaling value. When the local density of the point cloud is lower than the second density threshold, the scaling factor of the corresponding Gaussian sphere is initialized to the second scaling value. The first scaling value is less than the second scaling value, so as to accelerate training convergence and avoid local optima.

[0024] By directly using the sparse point cloud coordinates generated by the motion reconstructed structure as the initial position of the Gaussian sphere. The scaling factor is initialized based on the local density of the point cloud, enabling efficient startup of model training. Compared with random initialization, this technique utilizes the prior geometric information provided by SfM to make the initial Gaussian distribution closer to the structure of the real scene, significantly accelerating the training convergence speed and effectively avoiding the problem of local optima caused by improper initial values.

[0025] Furthermore, the loss function used in the model training and optimization... pixel-level L1 norm loss and structural similarity loss The weighted composition is expressed as follows: , in, , To render the image With real images Structural similarity index between them The weighting coefficients for balancing the two types of losses are set between 0.2 and 0.8.

[0026] Through the composite loss function The model training is constrained, taking into account both pixel-level color accuracy and macroscopic structural consistency. Compared to single L1 or L2 loss, this technical solution introduces structural similarity loss. This allows the reconstructed model to perform excellently in maintaining structural features such as the sharpness of road markings and the clarity of guardrail edges, meeting the dual requirements of geometric accuracy and visual quality for engineering applications.

[0027] Furthermore, the real-time rendering specifically includes: Divide the screen into multiple A pixel-based patch, excluding Gaussian spheres outside the view frustum; For each Gaussian sphere covering a tile, its floating-point depth value is... Convert to 32-bit ordered integer key value ; Use GPU thread blocks to analyze the key-value pairs within each tile. Perform parallel radix sort to determine the depth order of Gaussian spheres within the same tile; For each pixel, the color contribution of the Gaussian sphere is accumulated using the alpha blending formula, following the order of depth from near to far, until the accumulated opacity approaches 1. The alpha blending formula is as follows: , in, It is the first The color of the Gaussian sphere Its opacity, The set of Gaussian spheres covered by the current pixel.

[0028] Through tile-based parallel radix sorting and Alpha hybrid formula This technology constructs a highly efficient real-time rendering pipeline. Compared to traditional ray-stepping rendering, this solution utilizes frustum culling and depth keying... Transformation and parallel sorting reduce the rendering computation of tens of millions of Gaussian spheres to a level that can be processed in real time, achieving smooth interaction of over 60FPS on Nvidia RTX 3060 level graphics cards, and supporting engineers to perform lag-free scene roaming on the web and desktop.

[0029] Compared with existing technologies, the advantages of this invention are: 1. Significantly improves reconstruction efficiency and greatly shortens project cycle. This application abandons the complex manual processing flow of "point cloud → mesh → texture mapping" in traditional photogrammetry technology, and also avoids the time-consuming implicit training process of neural radiation field technology. By adopting an end-to-end explicit radiation field reconstruction algorithm based on 3D Gaussian splashing, it realizes the rapid generation from data acquisition to interactive 3D model. Experiments show that for the same scale of highway scene, the reconstruction time of this application can be shortened to less than one-third of the traditional method, and it has the potential to complete the rapid modeling of tens of kilometers of highway scene within 24 hours, which greatly improves the implementation efficiency of engineering projects. 2. Effectively reduces hardware and labor costs and improves economic efficiency. This application has more flexible requirements for data acquisition equipment, not only compatible with professional acquisition equipment, but also able to make full use of image data collected by consumer-grade drones, reducing reliance on expensive professional scanning equipment (such as high-precision LiDAR), thus reducing the cost of data acquisition equipment by more than 20%. At the same time, due to the high degree of automation of the reconstruction process, the workload of manual internal processing is greatly reduced, and labor costs can be saved by more than 50%, making it possible to model highway engineering with high frequency and low cost throughout the entire life cycle. 3. To achieve high-fidelity close-up detail reproduction and improve model quality, this application uses anisotropic three-dimensional Gaussian spheres as the basic representation unit and combines them with an adaptive density control strategy. By automatically "cloning" or "splitting" Gaussian spheres in high-frequency detail regions with large gradients (such as road surface cracks and bridge textures), adaptive fitting of fine geometric structures is achieved. At the same time, spherical harmonic functions are introduced to express view-related colors, which can realistically reproduce complex material effects such as road surface reflections and metal guardrail luster. Compared with traditional oblique photogrammetry models, the method of this application can improve texture clarity by more than 30% under close-up observation, meeting the stringent requirements of engineering for microscopic defect detection and refined display. 4. Supports incremental dynamic updates, enhancing model timeliness. Addressing the industry pain point of dynamic changes in highway engineering (such as construction progress), this application utilizes the discrete explicit representation characteristics of a three-dimensional Gaussian sphere to establish an efficient incremental update mechanism. When a local scene changes, only rapid supplementary data collection and incremental training of the changed area are needed to achieve a local update of the original model, without the need for a full-process reconstruction of the entire scene. This solves the problem of traditional models being "fixed as soon as they are built," shortening the model update cycle from days to hours, effectively supporting the dynamic needs of construction progress management and disaster emergency response. 5. Achieving seamless integration of tunnel interior and exterior, enhancing full-scene reconstruction capabilities; This application, specifically addressing the unique scenario of tunnels, designs a dual-source acquisition and fusion scheme combining "external UAV + internal vehicle / handheld devices." Through visual matching and SVD decomposition algorithms based on common feature regions, the coordinate system transformation matrix and scale factor are accurately calculated, resolving the issues of inconsistent spatial references and cumulative drift caused by the lack of GNSS signals inside the tunnel. Ultimately, seamless integration and integrated high-precision reconstruction of tunnel entrances, tunnel bodies, internal auxiliary structures, external power sources, and bridge scenes under a unified geographic coordinate system are achieved, filling the application gap of existing technologies in complex tunnel scenarios. 6. Achieve real-time interaction in ultra-large-scale scenes and enhance user experience. This application designs a tile-based differential rasterization rendering pipeline. Through optimization strategies such as view frustum culling and GPU parallel cardinality sorting, tens of millions of 2D Gaussian projections are efficiently sorted and blended, replacing the time-consuming ray stepping in traditional volume rendering. This innovation enables models to achieve smooth real-time rendering and interaction at over 60 frames per second (FPS) on ordinary desktop graphics cards (such as Nvidia RTX 3060), supporting lag-free scene roaming, viewpoint rotation, and zooming on both web and desktop platforms, providing engineers with an excellent visualization experience. Attached Figure Description

[0030] Figure 1 This is a flowchart of the overall process of the method described in this application.

[0031] Figure 2 This is a schematic diagram illustrating drone data collection strategies for different scenarios such as highways, bridges, and tunnels.

[0032] Figure 3 This is a schematic diagram illustrating the geometric transformation principle of projecting a three-dimensional Gaussian sphere onto a two-dimensional plane.

[0033] Figure 4 This is a rendering diagram of the prototype system. Detailed Implementation

[0034] It should be noted that relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0035] The features and performance of the present invention will be further described in detail below with reference to embodiments.

[0036] Please see Figure 1-4 A fast reconstruction method for full-scene highway reconstruction based on multi-source parallax control and adaptive explicit radiation field, such as... Figure 1 As shown, it includes the following steps: Multi-source data acquisition and camera attitude control in highway engineering: In view of the characteristics of highway engineering scenarios, such as large span and complex structure (including bridges and tunnels), this application designs a differentiated acquisition strategy to ensure the parallax and feature coverage required for 3D reconstruction.

[0037] like Figure 2 As shown, the data acquisition strategy for linear highway scenes is as follows: For long-distance highways, a "multi-altitude linear flight" strategy is employed. The drone is controlled to fly back and forth above the road centerline, at both low altitudes (to capture texture details such as road surface cracks and markings) and high altitudes (to capture the overall road alignment and surrounding environment). By adjusting the camera's pitch angle, a specific angle is maintained between the principal optical axis and the road surface to obtain sufficient geometric texture features. The camera is set at... The pose at time t is: , in, Let the rotation matrix describe the camera's orientation. for The pose of the camera at all times. This is a translation vector describing the camera's spatial position. It is actually a special Euclidean group. A concise expression, definition The pose of the camera in the world coordinate system is the core foundation for mapping pixel coordinates to world coordinates in 3D reconstruction, and the camera pose calculation in the entire acquisition process is based on this expression.

[0038] To ensure the observability of pavement cracks, a sampling constraint formula is defined: , in, Let the principal optical axis vector of the camera be _____. This is the local normal vector of the road surface. This application dynamically adjusts... ,make sure In order to obtain the optimal anisotropic texture information.

[0039] A surround-view acquisition strategy for bridge scenes: For complex structures such as beam bridges, arch bridges, and suspension bridges, a "circling" acquisition trajectory is employed. The drone orbits the bridge structure in multiple layers, centered on the geometric center or key nodes (such as towers and piers). During acquisition, algorithms calculate the camera's attitude in real time to ensure the camera always points towards the main structure, covering blind spots such as the bridge's sides and underside that cannot be captured by conventional orthophotos. The geometric center is determined by the tower, pier, or span center. Perform multi-radius It ascends in a circular or spiral motion. The algorithm calculates the camera attitude in real time to ensure that the camera's optical axis always points towards the main structure. This covers the "visual blind spots" of traditional orthophotos, such as the area under the bridge and its sides. The drone trajectory equation can be defined as a parameterized helix: , , , in, , , These are the geometric center coordinates of the key structural nodes. For the observation radius, The vertical lifting speed, Angular velocity. At each sampling point Camera pose The centripetal constraint must be satisfied, i.e., the camera... Axis and Vector Overlapping is used to achieve full-dimensional sampling of complex structural components.

[0040] Tunnel Scene Internal and External Fusion Acquisition Strategy: To address industry pain points in tunnel scenarios, such as complete GNSS signal obstruction, uneven internal lighting, weak texture in the lining area, easy drift of long-distance acquisition trajectories, and inconsistent spatiotemporal references between internal and external scenes, this application adopts a dual-source data fusion scheme of "differentiated acquisition for different scenarios + rigid alignment of spatiotemporal references + global joint optimization" to achieve full coverage of parallax and features without blind spots in the entire scene from tunnel entrance to tunnel body to exit, ensuring geometric consistency and coordinate uniformity between the reconstructed model and the external highway and bridge scenes.

[0041] External data acquisition: For the connection section between the tunnel entrance and the linear highway, the data acquisition benchmark and feature integration with the main scene are unified. The specific strategies are as follows: Data acquisition for the main line connection section: Utilizing a multi-altitude linear flight strategy, complete full-coverage data acquisition of the main road, slopes, inclines, tunnel portal structure, and surrounding terrain within a 200-500m range outside the tunnel entrance, while maintaining camera pitch angle constraints. Consistent with the disparity control rules for linear highway acquisition, this ensures the continuity of texture features and the robustness of matching corresponding points. Specialized data collection of the tunnel entrance structure: Using the geometric center of the tunnel entrance as the center, set up 3 to 5 circular flight paths at different heights to cover the tunnel entrance facade, end walls, drainage facilities, tunnel entrance side ditches and other structures, to ensure full coverage of the multi-parallax features of the tunnel entrance structure, and to provide sufficient common feature points for subsequent fusion of internal and external coordinate systems.

[0042] The poses of all cameras collected outside the tunnel are unified in the world coordinate system. (Unified geographic coordinate system for the project, consistent with the coordinate system used for data collection on the main highway and bridges) under the following conditions, the first The frame camera pose is defined as: , in, Let be the rotation matrix from the world coordinate system to the camera coordinate system. For the corresponding translation vector, It is a special Euclidean group.

[0043] Internal data acquisition: Addressing the challenges of no GNSS signal, weak texture, and insufficient lighting inside the tunnel, a data acquisition scheme combining "vehicle-mounted multi-sensor fusion acquisition + local parallax constraint + closed-loop control" is adopted, as detailed below: The data acquisition vehicle is controlled to travel at a constant speed along the centerline of the tunnel lane, and the speed meets the following constraints: , in, For camera sampling frame rate, The maximum baseline distance between adjacent frames is used to ensure that the overlap rate of adjacent frames is ≥80%, and the ratio of the inter-frame baseline to the scene depth is controlled within the range of 0.1~0.5, ensuring sufficient longitudinal parallax and avoiding loss of reconstruction accuracy. For auxiliary structures such as emergency parking lanes, cross passages, and substations in tunnels, supplementary local surround acquisition is used to achieve full coverage of structural blind spots.

[0044] Fusion Algorithm: Extract the tunnel entrance as a common feature area, use a visual feature point matching algorithm to calculate the transformation matrix between the internal trajectory coordinate system and the external geographic coordinate system, and uniformly map the pose of the video frames inside the tunnel to the external UAV coordinate system to form a unified scene representation.

[0045] Keyframe sets for external openings ( (Number of keyframes for external openings) and the set of keyframes for internal openings ( (Number of keyframes for internal openings), a scale-invariant feature extraction algorithm is used to extract feature points and perform bidirectional matching. After removing mismatches using the RANSAC algorithm, the following is obtained: For high-precision matching of feature point pairs with the same name: , in, The three-dimensional coordinates of the feature point in the world coordinate system. The feature points are given three-dimensional coordinates in the local coordinate system. Mismatched point pairs with reprojection errors greater than 2 pixels are eliminated to ensure matching accuracy.

[0046] Solving from the local coordinate system inside the tunnel To the world coordinate system Euclidean transformation matrix This matrix contains rotation matrices. Translation vector Satisfies the rigid coordinate transformation relationship: , The optimal transformation matrix is ​​solved using the Singular Value Decomposition (SVD) method. The specific calculation process is as follows: Calculate the centroids of the two sets of feature points: .

[0047] in, Let be the centroid of the feature point set in the world coordinate system. Let be the centroid of the feature point set in the local coordinate system. Calculating the centroids of the two sets of matching feature points in their respective coordinate systems is a preliminary step in SVD for solving the optimal rigid transformation matrix, used to eliminate the interference of translation on the solution of the rotation matrix.

[0048] Decentralized processing of feature points: .

[0049] in, The decentralized first world coordinate system Coordinates of feature points The decentered first (in local coordinate system) The coordinates of the feature points are determined. After decentering, the centroids of both feature point sets become the origin (0,0,0). At this point, the transformation between the two sets of points involves only rotation and no translation. The optimal rotation matrix can be solved independently, completely eliminating the interference of translation.

[0050] Construct the covariance matrix: .

[0051] Construct covariance matrices for two sets of decentralized feature points to describe the linear correlation between the two sets of points. Let be the covariance matrix to be decomposed.

[0052] Perform SVD decomposition on the covariance matrix: .

[0053] in, yes Left singular orthogonal matrices satisfying ; yes A diagonal matrix, where the elements on the diagonal are the singular values ​​of matrix H, arranged in descending order, and are all non-negative real numbers; yes Right singular orthogonal matrices satisfying .

[0054] Solve for the optimal rotation matrix: .

[0055] Based on the SVD decomposition results, the optimal rotation matrix between the two sets of point sets is solved, while the differences in the rotation matrices are corrected to avoid the appearance of reflection matrices. Construct a diagonal matrix, with the first two diagonal elements fixed at 1, and the third element being the matrix itself. The determinant value, for The transpose of .

[0056] Solving for the optimal translation vector: .

[0057] After solving for the optimal rotation matrix, the optimal translation vector between the two sets of points is calculated to complete the solution of the rigid transformation matrix.

[0058] Based on the initial transformation matrix, the camera poses in all local coordinate systems within the tunnel are uniformly mapped to the world coordinate system: .

[0059] in, The first tunnel in the world coordinate system The pose of the frame camera is made consistent with the pose acquired by the external drone. Local coordinate system The first tunnel inside The pose of the frame camera. It is a Euclidean transformation matrix.

[0060] To address the scale drift problem that easily occurs in visual inertial odometry in long tunnel scenarios, a scale correction factor is calculated using common feature points. Eliminate scale bias between internal and external scenes: .

[0061] By incorporating the scale factor into the transformation matrix, scale correction of the pose within the tunnel is achieved. The corrected coordinate transformation and pose mapping formula is then updated as follows: , .

[0062] To further eliminate the cumulative error in pose calculation, a joint bundle adjustment (BA) optimization model is constructed, which includes external UAV keyframes, internal vehicle-mounted keyframes, and all 3D feature points. The objective function is to minimize the sum of reprojection errors of all keyframes. , in, For all keyframe camera poses to be optimized (including external ones) Frame, Internal frame), Let the world coordinates of all 3D feature points to be optimized be... For the first The number of feature points observed in each keyframe This is a camera pinhole projection model, used to project 3D world coordinates onto the image pixel plane. For feature points at the th Pixel observation coordinates on the frame image For the first The spatial coordinates of a three-dimensional feature point in the world coordinate system.

[0063] For long / extra-long tunnels with a length of 1 km or more, a global closed-loop constraint is added at the tunnel exit. The internal and external fusion strategy of the entrance is reused at the tunnel exit, and a closed-loop constraint term is constructed by extracting the common feature region at the exit. This term is then added to the optimization objective function to eliminate long-mileage trajectory cumulative drift.

[0064] in, The first tunnel in the world coordinate system The pose of the frame camera. The camera pose was captured from outside the tunnel exit. The coordinate system transformation matrix to be solved at the exit point. This is a partial pose of the camera captured inside the exit area. The Frobenius norm is used to measure the pose transformation residuals. The weighting coefficients for the closed-loop constraints, with a default value of 10 to 100.

[0065] The above fusion scheme achieves complete unification of pose references for scenes inside and outside the tunnel, solving industry pain points such as lack of GNSS signals, long-distance drift, and disconnect between internal and external models in tunnel scenes. It provides high-precision, full-coverage pose and image data support for the initialization and optimization of subsequent 3D Gaussian scene models.

[0066] Construction of 3D scene models: This application abandons the traditional triangular mesh or voxel representation and instead uses a discrete, differentiable set of three-dimensional Gaussian spheres to represent highway engineering scenarios.

[0067] Set the properties of a 3D Gaussian sphere: Each 3D Gaussian sphere in the scene is determined by the following parameters: Position ): Three-dimensional vector , where represents the center of the Gaussian sphere in the world coordinate system.

[0068] Covariance matrix ): A matrix that controls the shape and orientation of a Gaussian sphere (an anisotropic ellipsoid).

[0069] Opacity ): Scalar, range of values This indicates the ability of the Gaussian sphere to block light.

[0070] Spherical Harmonics : Used to fit view-dependent color, enabling the reconstructed model to represent anisotropic material effects such as road surface reflection and metal guardrail gloss.

[0071] like Figure 3 As shown, the projection transformation (Splatting) from three dimensions to two dimensions: To render a 3D scene onto a 2D screen, this application employs a splashing technique. Given the view transformation matrix of the current camera... (View Matrix) The process of projecting a three-dimensional Gaussian sphere onto a two-dimensional image plane is described by the transformation of its covariance matrix. The projected two-dimensional covariance matrix... The calculation formula is: , in: It is the three-dimensional covariance matrix obtained through training; It is the affine approximation Jacobian matrix of the projective transformation; It is the transformation matrix from the world coordinate system to the camera coordinate system. This is the transpose of the matrix. This formula is the mathematical foundation for the correct mapping from three-dimensional space to two-dimensional pixel space in this application.

[0072] Model training based on adaptive density control: The purpose of model training is to optimize the Gaussian parameters mentioned above so that the rendered image is as consistent as possible with the acquired real image.

[0073] Initialization (SfM Initialization): Using the Structure from Motion (SfM) algorithm, sparse point cloud reconstruction is performed on the acquired images using COLMAP. The coordinates of the generated sparse point cloud are directly used as the initial positions of the 3D Gaussian sphere. The scaling factor is initialized based on the point cloud density, thereby accelerating training convergence and avoiding local optima.

[0074] Loss function calculation and backpropagation: In each iteration, the rendered image is compared with the real image, and a loss function is calculated. The loss function is weighted by the L1 norm and D-SSIM (structural similarity). The mathematical expression is as follows: , in, It is primarily responsible for pixel-level color alignment. It calculates the average absolute error of the color value of each pixel between the rendered image and the real image, ensuring accurate color reproduction. Its primary responsibility is to maintain the structural features of the image (such as the continuity of road surface texture and the sharpness of road markings). It is typically defined as... Similarity is evaluated by comparing the brightness, contrast, and structure of images. Hyperparameters are used to balance the weights of both. Automatic differentiation is performed using frameworks such as PyTorch to calculate the gradient of the loss function with respect to position, rotation, scaling, opacity, and spherical harmonics, and the parameters are updated using gradient descent (Adam Optimizer).

[0075] Adaptive Density Control: To address the challenges of complex road surface textures and large distances between near and far views, this application introduces a dynamic Gaussian sphere splitting and pruning mechanism: Cloning: In regions with large gradients (indicating large reconstruction errors) and small Gaussian sphere volumes (typically regions rich in detail, such as road surface cracks), a new Gaussian sphere is copied to increase the sampling density.

[0076] Split: In regions with large gradients and excessively large Gaussian sphere volumes (indicating that the Gaussian sphere over-covers multiple features, such as roadbed edges), the large Gaussian sphere is split into two smaller Gaussian spheres to improve geometric accuracy.

[0077] Pruning: Regularly detect and remove opacity. Below the threshold (e.g.) Alternatively, use a Gaussian sphere that is too large (over-expanding) to eliminate floating noise in the scene and reduce the model's memory usage.

[0078] Tile-based differentiable rasterization rendering: To achieve real-time rendering on both desktop and web platforms, this application designs an efficient rasterization rendering pipeline.

[0079] Cone removal and tiling: First, remove the Gaussian sphere except for the view frustum. Then divide the screen into... Tiles of pixels.

[0080] Depth sorting: For each Gaussian sphere covering a tile, sort them cardinally based on their depth value from the camera. First, calculate the depth value of the center point of each feature sphere in the camera coordinate system. (Floating-point numbers) are converted to 32-bit unsigned integers to ensure efficient bitwise operations during sorting while maintaining depth monotonicity. This is achieved through parallel radix sorting based on GPU thread blocks. Unlike traditional comparison sorting, radix sort has a time complexity of O(log n). ,in With a bit depth, it is ideal for parallel processing of large-scale feature balls.

[0081] Pixel-level color accumulation (Alpha Blending): For each pixel, accumulate the color contribution of the Gaussian sphere in order of depth from near to far. Final pixel color. The calculation formula follows the discrete form of the volume rendering equation: , in, It is the first The color of the Gaussian sphere It refers to its opacity. When the accumulated opacity reaches saturation (close to 1), the calculation of that pixel stops to reduce unnecessary calculations.

[0082] Engineering application components: To enhance the practicality of the technical solution, this application also includes the following optional engineering features: Lightweight Web Loader: A rendering viewer developed based on the WebGL / WebGPU standards, supporting direct loading of compressed 3D Gaussian model files (.ply or .splat format) in the browser without the need to install professional software.

[0083] Interactive roaming control: Integrates a roaming controller customized for highway engineering, supporting automatic flight mode along the route, fixed-point observation mode, and switching between first-person perspective.

[0084] In another specific embodiment, the "K15+000 to K17+000" section of a highway reconstruction and expansion project, which is 2 kilometers long and includes a complex interchange and a tunnel entrance about 300 meters long, is taken as the specific application background.

[0085] Contextualized data acquisition and pose calculation: During the data acquisition phase, the system automatically executes differentiated strategies based on the scenario type.

[0086] 1. Parallax control calculation for linear highways: For the straight road section K15, "multi-altitude linear flight" will be implemented.

[0087] Input parameters: Set low-altitude flight altitude Target pitch angle Local normal vector of the road surface .

[0088] Calculation process: According to the formula Calculate the principal optical axis vector of the camera. .when hour, This determines the camera attitude control commands, creating a cross parallax between the principal optical axis and the ground, ensuring that the anisotropic features of road markings and cracks are fully captured.

[0089] 2. Example of rigid fusion of internal and external coordinate systems in tunnel: For the tunnel entrance area, the video tracks inside the tunnel without GNSS need to be fused into the world coordinate system. .

[0090] Input parameters: Extract pairs of feature points with the same name from the common area of ​​the cave entrance, where one of the points has coordinates in the world coordinate system. Coordinates in the local coordinate system of the tunnel .

[0091] Calculation process: The transformation matrix is ​​calculated through SVD decomposition. Calculate the scaling correction factor. To eliminate drift: Applying the mapping formula This unifies the poses of all images inside the cave to the engineering geographic coordinate system, achieving seamless integration of internal and external models.

[0092] Construction and training of explicit radiation field model: 1. The process of transforming a 3D projection into a 2D projection: Take the 3D Gaussian sphere projection in the bridge scene training process as an example.

[0093] Input parameter: Position of the Gaussian sphere 3D covariance matrix (Defined as a long ellipsoid in the cable direction), camera view transformation matrix .

[0094] Calculation process: Applying the formula .

[0095] in, This is the Jacobian matrix. The calculated value is... This represents the projection range of the ellipsoid onto the image plane. If the calculation shows that its screen coverage radius exceeds a threshold, then the feature sphere will participate in the color contribution calculation for that pixel.

[0096] 2. Adaptive density control decision: At iteration 3000, the system detects a cracked area in the road surface. Input value: the average gradient value of this area. (Exceeds the threshold of 0.0002), and the Gaussian sphere scaling factor s is extremely small.

[0097] Operation: Trigger the "cloning" mechanism to replicate the Gaussian sphere at that location to increase the feature sampling density until the loss function... The crack details are reduced to the target range, thus restoring them with high fidelity.

[0098] Rendering pipeline order: Radix sort is performed when rendering the complex, interwoven structure of a bridge.

[0099] Example data: A tile contains 3 Gaussian spheres with depth values ​​of... .

[0100] Calculation process: Convert the floating-point depth to a 32-bit ordered integer. Using GPU parallel radix sort, the resulting sequence is as follows: Pixel color calculation: based on the formula First, accumulate the depth of the nearest The color rendering stops when the accumulated transparency reaches saturation. The training process and rendering results are as follows: Figure 4 As shown, the algorithm achieves real-time rendering of large scenes at over 60 FPS on an RTX 3060 graphics card.

[0101] Figure 4 The demonstration showcases the performance of the rapid reconstruction prototype system built based on this invention on a desktop environment. The interface displays the entire rendering application process. By uploading multi-angle photos or videos to the platform and placing them in the training queue to initialize point cloud information, iterative training of a 3D Gaussian sphere is performed based on the reference images. After 10 minutes of training on an Nvidia RTX 4090 graphics card, the training results are viewed. Real-time rendering at 60fps on an Nvidia RTX 3060 graphics card shows the reconstructed high-precision 3D road scene, verifying the texture preservation capability of the method. The demonstration also showcases the interactive functions of viewpoint movement, rotation, and zoom via mouse or touch operation, demonstrating the high frame rate characteristics of real-time rendering.

[0102] The embodiments described above merely illustrate specific implementation methods of this application, and while the descriptions are detailed and specific, they should not be construed as limiting the scope of protection of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the technical solution of this application, and these modifications and improvements all fall within the scope of protection of this application.

Claims

1. A method for rapid reconstruction of a full-scene highway project based on multi-source parallax control and adaptive explicit radiation field, characterized in that, Includes the following steps: Data acquisition and pose calculation: Different acquisition strategies are adopted for different scenarios in highway engineering to acquire multi-source image data, and the camera pose of each frame of image is calculated. Different scenarios include tunnel scenarios, bridge scenarios, and linear highway scenarios. 3D Gaussian scene model construction: The scene is represented by a set of anisotropic 3D Gaussian spheres, each Gaussian sphere is defined by position, covariance matrix, opacity and spherical harmonic coefficients; Model training and optimization: Based on image data and camera pose, the property parameters of a 3D Gaussian sphere are iteratively optimized until convergence through differentiable rendering and adaptive density control strategies. Real-time rendering: Employs a tile-based, differentiable rasterization rendering pipeline to project an optimized set of 3D Gaussian spheres onto a 2D plane, generating a real-time interactive 3D scene; In data acquisition and pose calculation, the internal and external fusion acquisition strategies for tunnel scenarios include: Acquire the sequence of images of the tunnel entrance collected by a drone and their coordinates in the world coordinate system. Next First camera pose of the frame ,in Let be the rotation matrix from the world coordinate system to the camera coordinate system. For the corresponding translation vector, This is a special Euclidean group; Acquire tunnel image sequences and their local coordinate system from vehicle-mounted or handheld devices inside the tunnel. The second camera pose is below; Extract the same feature point pairs in the public area of ​​the cave entrance. , The three-dimensional coordinates of the feature point in the world coordinate system. Given the 3D coordinates of feature points in the local coordinate system, mismatched point pairs with a reprojection error greater than 2 pixels are removed. It is the number of matching points retained after removing false matches; Based on the remaining feature point pairs, the Euclidean transformation matrix from the local coordinate system to the world coordinate system is solved by singular value decomposition. And calculate the scale correction factor: ， in Let be the centroid of the feature point set in the world coordinate system. Let be the centroid of the feature point set in the local coordinate system; According to the mapping formula The pose of the second camera is uniformly mapped to the world coordinate system; among which, For rotation matrix, It is a translation vector. The first tunnel in the world coordinate system The pose of the frame camera. The pose of the second camera; For long tunnels with a length of 1 km or more, the internal and external data fusion acquisition strategy further includes: At the tunnel exit, the internal and external fusion strategy of the entrance is reused, and the common feature area of ​​the exit is extracted to construct a closed-loop constraint; Add the closed-loop constraint term to the objective function of the joint bundle adjustment optimization: in, For the world coordinate system The pose of the frame camera, The camera pose is captured from outside the exit. The coordinate system transformation matrix to be solved at the exit point. This is a partial pose of the camera captured inside the exit area. It is the Frobenius norm. The weighting coefficients for the closed-loop constraints range from 10 to 100. For all keyframe camera poses to be optimized, Let the world coordinates of all 3D feature points to be optimized be... For the first The number of feature points observed in each keyframe This is a camera pinhole projection model, used to project 3D world coordinates onto the image pixel plane. For feature points at the th Pixel observation coordinates on the frame image For the first The spatial coordinates of a three-dimensional feature point in the world coordinate system.

2. The method for rapid reconstruction of a full-scene highway network based on multi-source parallax control and adaptive explicit radiation field as described in claim 1, characterized in that, The data acquisition and pose calculation strategies for bridge scenarios include: The drone is controlled to orbit around the key structural nodes of the bridge, following a parametric spiral trajectory defined by the following parametric equations: ， ， ， in, , , These are the geometric center coordinates of the key structural nodes. For the observation radius, The vertical lifting speed, Angular velocity; Real-time calculation of UAV attitude, ensuring accuracy at each sampling point. Camera optical axis and vector Overlapping is used to achieve full-dimensional data coverage of the blind spots at the bottom and sides of the bridge.

3. The method for rapid reconstruction of a full-scene highway network based on multi-source parallax control and adaptive explicit radiation field as described in claim 1, characterized in that, In the data acquisition and pose calculation, the acquisition strategy for linear highway scenarios includes: Control the drone to fly back and forth at low and high altitudes above the center line of the road, and dynamically adjust the camera pitch angle. To make it satisfy the constraints: ， in, Let the principal optical axis vector of the camera be _____. , This is the local normal vector of the road surface, used to obtain the optimal anisotropic texture information.

4. The method for rapid reconstruction of a full-scene highway network based on multi-source parallax control and adaptive explicit radiation field as described in claim 1, characterized in that, The data acquisition and pose calculation process includes the following for data acquisition inside the tunnel: The data acquisition vehicle is controlled to travel at a constant speed along the centerline of the tunnel lane, and the speed meets the following constraints: ， in, For camera sampling frame rate, The maximum baseline distance between adjacent frames is used to ensure that the overlap rate of adjacent frames is ≥80%, and the ratio of inter-frame baseline to scene depth is controlled in the range of 0.1~0.

5.

5. The method for rapid reconstruction of a full-scene highway network based on multi-source parallax control and adaptive explicit radiation field as described in claim 1, characterized in that, In the construction of the 3D Gaussian scene model, the process of projecting a 3D Gaussian sphere onto a 2D image plane is described by the covariance projection formula: ， in, It is the covariance matrix of a three-dimensional Gaussian sphere. It is the view transformation matrix from the world coordinate system to the camera coordinate system. It is the affine approximation Jacobian matrix of the projective transformation. It is the projected two-dimensional covariance matrix. This is the transpose of the matrix.

6. The method for rapid reconstruction of a full-scene highway network based on multi-source parallax control and adaptive explicit radiation field as described in claim 1, characterized in that, The initialization in the model training and optimization includes: The structure-of-motion algorithm is used to sparsely reconstruct the acquired image data, and the resulting sparse point cloud coordinates are directly used as the initial positions of the 3D Gaussian sphere. ; The scaling factor is initialized based on the comparison between the local density of the point cloud and the preset density threshold. When the local density of the point cloud is higher than the first density threshold, the scaling factor of the corresponding Gaussian sphere is initialized to the first scaling value. When the local density of the point cloud is lower than the second density threshold, the scaling factor of the corresponding Gaussian sphere is initialized to the second scaling value. The first scaling value is less than the second scaling value, so as to accelerate training convergence and avoid local optima.

7. The method for rapid reconstruction of a full-scene highway network based on multi-source parallax control and adaptive explicit radiation field as described in claim 1, characterized in that, The loss function used in the model training and optimization pixel-level L1 norm loss and structural similarity loss The weighted composition is expressed as follows: ， in, , To render the image With real images Structural similarity index between them The weighting coefficients for balancing the two types of losses are set between 0.2 and 0.

8.

8. The method for rapid reconstruction of a full-scene highway network based on multi-source parallax control and adaptive explicit radiation field as described in claim 1, characterized in that, The real-time rendering specifically includes: Divide the screen into multiple A pixel-based patch, excluding Gaussian spheres outside the view frustum; For each Gaussian sphere covering a tile, its floating-point depth value is... Convert to 32-bit ordered integer key value ; Use GPU thread blocks to analyze the key-value pairs within each tile. Perform parallel radix sort to determine the depth order of Gaussian spheres within the same tile; For each pixel, the color contribution of the Gaussian sphere is accumulated using the alpha blending formula, following the order of depth from near to far, until the accumulated opacity reaches 1. The alpha blending formula is as follows: ， in, It is the first The color of the Gaussian sphere Its opacity, The set of Gaussian spheres covered by the current pixel.

Images