Highway full-scene rapid reconstruction method based on multi-source parallax control and adaptive explicit radiation field

Abstract

The invention discloses a road full-scene rapid reconstruction method based on multi-source parallax control and an adaptive explicit radiation field, and relates to the technical field of three-dimensional reconstruction. Comprising the following steps: collecting multi-source image data by adopting a differentiation strategy aiming at scenes such as roads, bridges and tunnels, and resolving camera poses; the scene is represented by an anisotropic three-dimensional Gaussian ball set, and accurate mapping from three dimensions to two dimensions is realized through a covariance projection formula; gaussian ball attributes are iteratively optimized based on an adaptive density control strategy of a gradient threshold and volume constraint, and automatic cloning / splitting is carried out in a detail area to improve precision; and real-time rendering of more than 60FPS is realized by adopting parallel cardinal number sorting based on image blocks and an Alpha mixed formula. The problems that a traditional method is high in cost, the close shot is fuzzy, the interior and exterior of the tunnel are disjointed, and real-time interaction is difficult are solved, rapid modeling and incremental updating within 24 hours are achieved, and the reconstruction efficiency, the detail fidelity and the user experience are remarkably improved.

Description

A Fast Reconstruction Method for Full-Scene Highway Reconstruction Based on Multi-Source Parallax Control and Adaptive Explicit Radiation Field Technical Field

[0001] This invention relates to the field of 3D reconstruction technology, specifically to a method for rapid 3D scene reconstruction for large infrastructure scenarios such as highways, bridges, and tunnels. This method utilizes multi-view fusion technology and spatial explicit radiation field technology for data acquisition, model training, and real-time rendering. Background Technology

[0002] In the planning, construction, management, and maintenance of highway engineering throughout its entire lifecycle, high-precision digital reality models are a key foundation for achieving visualized management and intelligent decision-making. Currently, the reconstruction of 3D scenes for large-scale infrastructure mainly relies on the following existing technologies, but they all have significant limitations in practical applications: Traditional oblique photogrammetry and 3D laser scanning technology. This technology mainly uses UAV oblique photogrammetry or ground / airborne LiDAR for field data acquisition, followed by industry data processing, including complex steps such as point cloud generation and registration, surface mesh construction, texture mapping and texturing, to finally generate a 3D reality model.

[0003] However, this technology has the following drawbacks: high cost and low efficiency. It relies on specialized data acquisition equipment (such as LiDAR) and a large amount of manual data processing. For example, modeling a 30-kilometer-long road usually requires two professionals to spend 10 to 20 days to complete, resulting in extremely high equipment and labor costs. It also lacks close-up details. Although the technology can restore the overall outline of a large-scale scene, when the viewing distance is zoomed in, the surface texture of the model is often blurry and lacks high-frequency details, making it difficult to meet the needs for detecting or finely displaying defects in fine structures such as road surface cracks and bridge supports. Furthermore, it is difficult to update and maintain the model. Once the scene changes (such as the progress of construction), it is usually necessary to re-acquire data from the field and process the data in the office, resulting in long model update cycles, high costs, and data that is seriously lagging behind the actual situation on site.

[0004] The implicit reconstruction technique based on neural radiation fields (NeRF) uses neural networks such as multilayer perceptrons (MLP) to encode the scene as an implicit radiation field, and then synthesizes a new perspective image through volume rendering technology to generate a high-quality 3D model.

[0005] However, this technology also has significant shortcomings: training and rendering overhead is huge. The implicit representation requires massive computing resources for training, and dense point sampling and querying are required along each ray during rendering, resulting in extremely low rendering frame rates and making it difficult to achieve smooth real-time interaction on ordinary hardware. It is also difficult to adapt to large-scale scenes. The neural radiation field has extremely high requirements for graphics card memory and storage space, and it lacks effective representation of a large number of blank areas in the scene. When directly applied to long-distance, large-scale highway engineering scenes, it faces huge challenges in terms of computational complexity and resource consumption.

[0006] Therefore, existing 3D reconstruction technologies for highway engineering cannot simultaneously balance reconstruction efficiency, cost, precision, real-time interaction, and scene adaptability. There is an urgent need for a new method that can quickly, cost-effectively, and with high fidelity achieve 3D reconstruction of the entire highway scene (including bridges and tunnels), and support real-time interaction and dynamic updates. Summary of the Invention

[0007] The purpose of this invention is to address the aforementioned problems by providing a rapid reconstruction method for full-scene highway construction based on multi-source parallax control and adaptive explicit radiative field. Based on the explicit radiative field construction strategy, anisotropic three-dimensional Gaussian spheres are used to represent continuous highway, bridge, and tunnel scenes. Combined with targeted data acquisition strategies and differentiable rasterization rendering technology, this method achieves low-cost, high-efficiency, and real-time interactive engineering-grade 3D real-scene reconstruction.

[0008] The technical solution of this invention is as follows: A method for rapid reconstruction of a full-scene highway project based on multi-source parallax control and adaptive explicit radiation field, comprising the following steps: Data acquisition and pose calculation: Differentiated acquisition strategies are adopted for different scenes in highway engineering to acquire multi-source image data, and the camera pose of each frame image is calculated; 3D Gaussian scene model construction: The scene is represented by an anisotropic set of 3D Gaussian spheres, each Gaussian sphere being defined by position, covariance matrix, opacity, and spherical harmonic coefficients; Model training and optimization: Based on image data and camera pose, the attribute parameters of the 3D Gaussian spheres are iteratively optimized through differentiable rendering and adaptive density control strategies until convergence; Real-time rendering: A tile-based differentiable rasterization rendering pipeline is used to project the optimized 3D Gaussian sphere set onto a 2D plane to generate a real-time interactive 3D scene.

[0009] By organically combining four steps—differential data acquisition, explicit 3D Gaussian representation, adaptive optimization, and real-time rendering—a complete rapid reconstruction technology system for all highway scenes has been constructed. Compared with existing technologies, this method achieves a closed-loop process from data acquisition to real-time interaction, providing a basic framework for the implementation of subsequent innovative features, and significantly improving reconstruction efficiency and lowering the implementation threshold overall.

[0010] Furthermore, in the data acquisition and pose calculation, the internal and external fusion acquisition strategy for the tunnel scene includes: acquiring the tunnel entrance image sequence collected by the UAV outside the tunnel and its coordinates in the world coordinate system. First camera pose ,in Let be the rotation matrix from the world coordinate system to the camera coordinate system. For the corresponding translation vector, For special Euclidean groups; to obtain tunnel image sequences acquired by vehicle-mounted or handheld devices inside the tunnel and their local coordinate system. The second camera pose is below; extract the same feature points of the public area at the entrance of the tunnel. , The three-dimensional coordinates of the feature point in the world coordinate system. Given the 3D coordinates of feature points in the local coordinate system, mismatched point pairs with a reprojection error greater than 2 pixels are removed. This represents the number of high-precision matching points retained after removing mismatches; based on the remaining feature point pairs, the Euclidean transformation matrix from the local coordinate system to the world coordinate system is solved through singular value decomposition. And calculate the scale correction factor: ,in These are the centroids of the two sets of feature points; according to the mapping formula The pose of the second camera is uniformly mapped to the world coordinate system, achieving seamless fusion of scenes inside and outside the tunnel; among which, Let be a rotation matrix. It is a translation vector. The first tunnel in the world coordinate system The pose of the frame camera. Local coordinate system The first tunnel inside The pose of the frame camera.

[0011] The transformation matrix is ​​solved by SVD decomposition. And a scale correction factor is introduced. This solution addresses the issue of inconsistent spatial references caused by the lack of GNSS signals inside and outside the tunnel. Compared to conventional visual stitching methods, this technical solution uses rigid transformation and scale correction of common feature areas to accurately map the pose of the video stream inside the tunnel without GPS signals to a unified geographic coordinate system, achieving seamless integration of models inside and outside the tunnel and eliminating the industry pain points of model disconnect and coordinate misalignment in traditional methods.

[0012] Furthermore, for long tunnels with a length of 1 km or more, the internal and external fusion acquisition strategy further includes: reusing the internal and external fusion strategy at the tunnel exit to extract the common feature region at the exit and construct a closed-loop constraint; and adding the closed-loop constraint term to the objective function of the joint bundle adjustment optimization.

[0013] in, The first tunnel in the world coordinate system The pose of the frame camera, The camera pose is captured from outside the exit. The coordinate system transformation matrix to be solved at the exit point. This is a partial pose of the camera captured inside the exit area. It is the Frobenius norm. The weighting coefficients for closed-loop constraints range from 10 to 100 to eliminate the cumulative drift of long-mileage trajectories. For all keyframe camera poses to be optimized, Let the world coordinates of all 3D feature points to be optimized be... For the first The number of feature points observed in each keyframe This is a camera pinhole projection model that projects 3D world coordinates onto the image pixel plane. For feature points at the th Pixel observation coordinates on the frame image For the first The spatial coordinates of a three-dimensional feature point in the world coordinate system.

[0014] By introducing closed-loop constraints and constructing a joint bundle adjustment optimization objective function in long tunnel scenarios, the cumulative drift problem of long-mileage acquisition trajectories is effectively solved. Compared with methods without closed-loop correction, this technical solution controls pose drift errors within the pixel-level range through closed-loop detection and weighted optimization of the common feature region at the exit, ensuring the geometric consistency and global accuracy of the reconstruction model of ultra-long tunnels exceeding 1 kilometer in length.

[0015] Furthermore, in the data acquisition and pose calculation, the acquisition strategy for the bridge scenario includes: controlling the UAV to fly around the key structural nodes of the bridge along a parameterized spiral trajectory, which is defined by the following parametric equations: , , ,in, , , These are the geometric center coordinates of the key structural nodes. For the observation radius, The vertical lifting speed, Angular velocity; real-time calculation of UAV attitude to ensure accuracy at each sampling point. Camera optical axis and vector Overlapping is used to achieve full-dimensional data coverage of the blind spots at the bottom and sides of the bridge.

[0016] By controlling the drone's attitude using a parameterized helical equation, full-dimensional data coverage of visual blind spots such as the bridge's bottom and sides is achieved. Compared to conventional linear aerial photography, this technical solution ensures that the camera is always pointed at the main structure through centripetal constraints, enabling the complete acquisition of multi-view features of complex structural components such as piers, towers, and cables, filling data blind spots that traditional orthophotos cannot cover.

[0017] Furthermore, in the data acquisition and pose calculation, the acquisition strategy for linear highway scenarios includes: controlling the UAV to fly back and forth at low and high altitudes above the centerline of the road, and dynamically adjusting the camera pitch angle. To make it satisfy the constraints: ,in, Let the principal optical axis vector of the camera be _____. , This is the local normal vector of the road surface, used to obtain the optimal anisotropic texture information.

[0018] By dynamically adjusting the camera's pitch angle Constraints were established to create optimal acquisition conditions for anisotropic texture features of the road surface. Compared with acquisition methods using a fixed pitch angle, this technical solution ensures optimal cross-parallax between the principal optical axis and the road surface through real-time calculation and feedback control of geometric relationships, significantly improving the observability and reconstruction accuracy of linear texture features such as road cracks and markings.

[0019] Furthermore, in the data acquisition and pose calculation, the acquisition inside the tunnel includes: controlling the acquisition vehicle to travel at a constant speed along the centerline of the tunnel lane, with the speed satisfying the following constraints: ,in, For camera sampling frame rate, The maximum baseline distance between adjacent frames is used to ensure that the overlap rate of adjacent frames is ≥80%, and the ratio of inter-frame baseline to scene depth is controlled in the range of 0.1~0.5.

[0020] Speed ​​constraints By controlling the ratio of inter-frame baseline to scene depth acquired inside the tunnel, the overlap rate of adjacent frames is guaranteed to be ≥80%. Compared with methods that arbitrarily control the acquisition speed, this technical solution provides sufficient longitudinal parallax for subsequent visual feature matching and 3D reconstruction through quantitative motion constraints, avoiding the loss of reconstruction accuracy due to excessively large inter-frame baseline or the data redundancy due to excessively small inter-frame baseline.

[0021] Furthermore, in the construction of the 3D Gaussian scene model, the process of projecting a 3D Gaussian sphere onto a 2D image plane is described by the covariance projection formula: ,in, It is the covariance matrix of a three-dimensional Gaussian sphere. It is the view transformation matrix from the world coordinate system to the camera coordinate system. It is the affine approximation Jacobian matrix of the projective transformation. It is the projected two-dimensional covariance matrix. This is the transpose of the matrix.

[0022] Using the covariance projection formula This method achieves precise mapping from a 3D anisotropic Gaussian sphere to a 2D pixel plane. Compared to traditional point cloud projection or voxel rendering, this solution utilizes the Jacobian matrix... The affine approximation accurately describes the shape, orientation, and coverage of a three-dimensional ellipsoid on the imaging plane. It is the core mathematical foundation connecting three-dimensional geometric properties with two-dimensional visual representation, ensuring the geometric fidelity of the rendered image.

[0023] Furthermore, the initialization in the model training and optimization includes: using the structure-of-motion reconstruction algorithm to perform sparse reconstruction on the acquired image data, and directly using the generated sparse point cloud coordinates as the initial position of the three-dimensional Gaussian sphere. The scaling factor is initialized based on the comparison between the local density of the point cloud and the preset density threshold. When the local density of the point cloud is higher than the first density threshold, the scaling factor of the corresponding Gaussian sphere is initialized to the first scaling value. When the local density of the point cloud is lower than the second density threshold, the scaling factor of the corresponding Gaussian sphere is initialized to the second scaling value. The first scaling value is less than the second scaling value, so as to accelerate training convergence and avoid local optima.

[0024] The sparse point cloud coordinates generated by the motion reconstructed structure are directly used as the initial positions of the Gaussian sphere. The scaling factor is initialized based on the local density of the point cloud, enabling efficient startup of model training. Compared with random initialization, this technique utilizes the prior geometric information provided by SfM to make the initial Gaussian distribution closer to the structure of the real scene, significantly accelerating the training convergence speed and effectively avoiding the problem of local optima caused by improper initial values.

[0025] Furthermore, the loss function used in the model training and optimization... pixel-level L1 norm loss and structural similarity loss The weighted composition is expressed as follows: ,in, , To render the image With real images Structural similarity index between them The weighting coefficients for balancing the two types of losses are set between 0.2 and 0.8.

[0026] Through the composite loss function The model training is constrained, taking into account both pixel-level color accuracy and macroscopic structural consistency. Compared to single L1 or L2 loss, this technical solution introduces structural similarity loss. This allows the reconstructed model to perform excellently in maintaining structural features such as the sharpness of road markings and the clarity of guardrail edges, meeting the dual requirements of geometric accuracy and visual quality for engineering applications.

[0027] Furthermore, the real-time rendering specifically includes: dividing the screen into multiple... The pixel-based patch is processed, and Gaussian spheres outside the view frustum are removed; for each patch covering a Gaussian sphere, its floating-point depth value is calculated. Convert to 32-bit ordered integer key value ;Utilize GPU thread blocks to analyze the key-value pairs within each tile Parallel radix sorting is performed to determine the depth order of Gaussian spheres within the same patch. For each pixel, the color contribution of the Gaussian spheres is accumulated using an alpha blending formula, in order of increasing depth, until the accumulated opacity approaches 1. The alpha blending formula is as follows: ,in, It is the first The color of the Gaussian sphere Its opacity, The set of Gaussian spheres covered by the current pixel.

[0028] Through tile-based parallel radix sorting and Alpha hybrid formula This technology constructs a highly efficient real-time rendering pipeline. Compared to traditional ray-stepping rendering, this solution utilizes frustum culling and depth keying... Transformation and parallel sorting reduce the rendering computation of tens of millions of Gaussian spheres to a level that can be processed in real time, achieving smooth interaction of over 60FPS on Nvidia RTX 3060 level graphics cards, and supporting engineers to perform lag-free scene roaming on the web and desktop.

[0029] Compared with existing technologies, the beneficial effects of this invention are: 1. Significantly improves reconstruction efficiency and greatly shortens project cycle. This application abandons the complex manual processing flow of "point cloud → mesh → texture mapping" in traditional photogrammetry technology, and also avoids the time-consuming implicit training process of neural radiation field technology; by adopting an end-to-end explicit radiation field reconstruction algorithm based on 3D Gaussian splashing, it realizes the rapid generation from collected data to an interactive 3D model; experiments show that for highway scenes of the same scale, the reconstruction time of this application can be shortened to less than one-third of the traditional method, and it has the potential to complete the rapid modeling of tens of kilometers of highway scenes within 24 hours, greatly improving the efficiency of engineering projects. 1. Improved efficiency; 2. Reduced hardware and labor costs, enhancing economic efficiency. This application offers greater flexibility in data acquisition equipment requirements, not only being compatible with professional acquisition equipment but also fully utilizing image data collected by consumer-grade drones, reducing reliance on expensive professional scanning equipment (such as high-precision LiDAR), resulting in a reduction of over 20% in data acquisition equipment costs; 3. Enhanced high-fidelity close-up detail restoration, improving model quality. This application employs anisotropic three-dimensional Gaussian spheres as the basic... The method employs an expression unit and incorporates an adaptive density control strategy. It automatically "clones" or "splits" Gaussian spheres in high-frequency detail regions with large gradients (such as road cracks and bridge textures), achieving adaptive fitting of fine geometric structures. Simultaneously, it introduces spherical harmonic functions to express view-dependent colors, realistically reproducing complex material effects such as road surface reflections and metal guardrail luster. Compared to traditional oblique photogrammetry models, this method improves texture clarity by over 30% in close-up observation, meeting the stringent requirements of engineering for microscopic defect detection and refined display. Furthermore, it supports incremental dynamic updates, enhancing model timeliness and addressing industry pain points related to dynamic changes in highway engineering (such as construction progress). Utilizing the discrete explicit representation characteristics of a 3D Gaussian sphere, an efficient incremental update mechanism was established. When a local scene changes, only rapid supplementary acquisition and incremental training of the changed area are needed to achieve a local update of the original model, without the need for a full-process reconstruction of the entire scene. This solves the problem of the traditional model being "fixed as soon as it is built," shortening the model update cycle from days to hours, effectively supporting the dynamic needs of construction progress management and disaster emergency response. 5. Achieving seamless integration inside and outside the tunnel, enhancing the full-scene reconstruction capability. This application designs a dual-source acquisition and fusion scheme of "external UAV + internal vehicle / handheld device" for the special scenario of tunnels.By employing visual matching and SVD decomposition algorithms based on common feature regions, the coordinate system transformation matrix and scale factor are accurately solved, resolving the issues of inconsistent spatial references and cumulative drift caused by the lack of GNSS signals inside the tunnel. Ultimately, seamless integration and high-precision reconstruction of the tunnel entrance, tunnel body, internal auxiliary structures, external power, and bridge scenes under a unified geographic coordinate system are achieved, filling the application gap of existing technologies in complex tunnel scenarios. 6. Real-time interaction in ultra-large-scale scenes is realized, enhancing user experience. This application designs a tile-based differential rasterization rendering pipeline. Through optimization strategies such as view frustum culling and GPU parallel cardinality sorting, tens of millions of 2D Gaussian projections are efficiently sorted and blended, replacing the time-consuming ray stepping in traditional volume rendering. This innovation enables the model to achieve smooth real-time rendering and interaction at over 60 frames per second (FPS) on ordinary desktop graphics cards (such as Nvidia RTX 3060), supporting lag-free scene roaming, viewpoint rotation, and zooming on both web and desktop platforms, providing engineers with an excellent visualization experience. Attached Figure Description

[0030] Figure 1 is a flowchart of the overall process of the method of this application.

[0031] Figure 2 is a schematic diagram of drone data collection strategies for different scenarios such as highways, bridges, and tunnels.

[0032] Figure 3 is a schematic diagram of the geometric transformation principle of projecting a three-dimensional Gaussian sphere onto a two-dimensional plane.

[0033] Figure 4 is a schematic diagram of the prototype system rendering effect. Detailed Implementation

[0034] It should be noted that relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0035] The features and performance of the present invention will be further described in detail below with reference to embodiments.

[0036] Please refer to Figures 1-4. A method for rapid reconstruction of a full-scene highway based on multi-source parallax control and adaptive explicit radiation field is shown in Figure 1. It includes the following steps: Multi-source data acquisition and camera attitude control for highway engineering: In view of the characteristics of highway engineering scenes with large span and complex structure (including bridges and tunnels), this application designs a differentiated acquisition strategy to ensure the parallax and feature coverage required for 3D reconstruction.

[0037] As shown in Figure 2, the linear highway scene acquisition strategy employs a "multi-altitude linear flight" strategy for long-mileage highways. The drone is controlled to fly back and forth above the road centerline, at both low altitude (for capturing texture details such as road surface cracks and markings) and high altitude (for capturing the overall road alignment and surrounding environment). By adjusting the camera's pitch angle, a specific angle is maintained between the principal optical axis and the road surface to obtain sufficient geometric texture features. The camera is set at... The pose at time t is: ,in, Let the rotation matrix describe the camera's orientation. for The pose of the camera at all times. This is a translation vector describing the camera's spatial position. It is actually a special Euclidean group. A concise expression, definition The pose of the camera in the world coordinate system is the core foundation for mapping pixel coordinates to world coordinates in 3D reconstruction, and the camera pose calculation in the entire acquisition process is based on this expression.

[0038] To ensure the observability of pavement cracks, a sampling constraint formula is defined: ,in, Let the principal optical axis vector of the camera be _____. This is the local normal vector of the road surface. This application dynamically adjusts... ,make sure In order to obtain the optimal anisotropic texture information.

[0039] A surround-style acquisition strategy for bridge scenes: For complex structures such as beam bridges, arch bridges, and suspension bridges, a "surround-style" acquisition trajectory is adopted. The drone flies in multiple layers around the geometric center or key nodes (such as bridge towers and piers) of the bridge structure. During the acquisition process, the camera attitude is calculated in real time through algorithms to ensure that the camera is always pointed at the main structure, covering blind spots such as the sides and bottom of the bridge that cannot be obtained by conventional orthophotos. The geometric center is the bridge tower, pier, or span center. Perform multi-radius It ascends in a circular or spiral motion. The algorithm calculates the camera attitude in real time to ensure that the camera's optical axis always points towards the main structure. This covers the "visual blind spots" of traditional orthophotos, such as the area under the bridge and its sides. The drone trajectory equation can be defined as a parameterized helix: , , ,in, , , These are the geometric center coordinates of the key structural nodes. For the observation radius, The vertical lifting speed, Angular velocity. At each sampling point Camera pose The centripetal constraint must be satisfied, i.e., the camera... Axis and Vector Overlapping is used to achieve full-dimensional sampling of complex structural components.

[0040] Internal and external fusion acquisition strategy for tunnel scenes: In response to industry pain points such as complete GNSS signal blockage, uneven internal lighting, weak texture in the lining area, easy drift of long-distance acquisition trajectory, and inconsistent spatiotemporal reference between internal and external scenes, this application adopts a dual-source data fusion scheme of "differentiated acquisition for different scenes + rigid alignment of spatiotemporal reference + global joint optimization" to achieve blind-spot-free, high-precision parallax and feature full coverage of the entire scene from tunnel entrance to tunnel body to exit, ensuring geometric consistency and coordinate uniformity between the reconstructed model and the external highway and bridge scenes.

[0041] External data acquisition: For the connection section between the tunnel entrance and the linear highway, the data acquisition benchmark and feature integration with the main line scene were unified. The specific strategy is as follows: Main line connection section acquisition: Using the multi-altitude linear flight strategy, full coverage acquisition of the main highway line, slopes, inclines, tunnel portal structure, and surrounding terrain within a range of 200-500m outside the tunnel entrance was completed, while maintaining camera pitch angle constraints. Consistent with the disparity control rules for linear highway acquisition, this ensures the continuity of texture features and the robustness of matching corresponding points. For specialized acquisition of tunnel entrance structures: using the geometric center of the tunnel entrance as the center, 3-5 circular flight paths at different heights are set to cover the tunnel entrance facade, end walls, drainage facilities, and tunnel entrance side ditches, ensuring full coverage of the multi-disparity features of the tunnel entrance structure and providing sufficient common feature points for subsequent fusion of internal and external coordinate systems.

[0042] The poses of all cameras collected outside the tunnel are unified in the world coordinate system. (Unified geographic coordinate system for the project, consistent with the coordinate system used for data collection on the main highway and bridges) under the following conditions, the first The frame camera pose is defined as: ,in, Let be the rotation matrix from the world coordinate system to the camera coordinate system. For the corresponding translation vector, It is a special Euclidean group.

[0043] Internal data acquisition: Addressing the challenges of no GNSS signal, weak texture, and insufficient lighting inside the tunnel, a data acquisition scheme of "vehicle-mounted multi-sensor fusion acquisition + local parallax constraint + closed-loop control" is adopted. Specifically, the acquisition vehicle is controlled to travel at a constant speed along the centerline of the tunnel lane, with the speed meeting the following constraints: ,in, For camera sampling frame rate, The maximum baseline distance between adjacent frames is used to ensure that the overlap rate of adjacent frames is ≥80%, and the ratio of the inter-frame baseline to the scene depth is controlled within the range of 0.1~0.5, ensuring sufficient longitudinal parallax and avoiding loss of reconstruction accuracy. For auxiliary structures such as emergency parking lanes, cross passages, and substations in tunnels, supplementary local surround acquisition is used to achieve full coverage of structural blind spots.

[0044] Fusion Algorithm: Extract the tunnel entrance as a common feature area, use a visual feature point matching algorithm to calculate the transformation matrix between the internal trajectory coordinate system and the external geographic coordinate system, and uniformly map the pose of the video frames inside the tunnel to the external UAV coordinate system to form a unified scene representation.

[0045] Keyframe sets for external openings ( (Number of keyframes for external openings) and the set of keyframes for internal openings ( (Number of keyframes for internal openings), a scale-invariant feature extraction algorithm is used to extract feature points and perform bidirectional matching. After removing mismatches using the RANSAC algorithm, the following is obtained: For high-precision matching of feature point pairs with the same name: ,in, The three-dimensional coordinates of the feature point in the world coordinate system. The feature points are given three-dimensional coordinates in the local coordinate system. Mismatched point pairs with reprojection errors greater than 2 pixels are removed to ensure matching accuracy.

[0046] Solving from the local coordinate system inside the tunnel To the world coordinate system Euclidean transformation matrix This matrix contains rotation matrices. Translation vector Satisfies the rigid coordinate transformation relationship: The optimal transformation matrix is ​​solved using the Singular Value Decomposition (SVD) method. The specific calculation process is as follows: Calculate the centroids of the two sets of feature points: .

[0047] in, Let be the centroid of the feature point set in the world coordinate system. Let be the centroid of the feature point set in the local coordinate system. Calculating the centroids of the two sets of matching feature points in their respective coordinate systems is a preliminary step in SVD for solving the optimal rigid transformation matrix, used to eliminate the interference of translation on the solution of the rotation matrix.

[0048] Decentralized processing of feature points: .

[0049] in, The decentralized first world coordinate system Coordinates of feature points The decentered first (in local coordinate system) The coordinates of the feature points are determined. After decentering, the centroids of both feature point sets become the origin (0,0,0). At this point, the transformation between the two sets of points involves only rotation and no translation. The optimal rotation matrix can be solved independently, completely eliminating the interference of translation.

[0050] Construct the covariance matrix: .

[0051] Construct covariance matrices for two sets of decentralized feature points to describe the linear correlation between the two sets of points. Let be the covariance matrix to be decomposed.

[0052] Perform SVD decomposition on the covariance matrix: .

[0053] in, yes Left singular orthogonal matrices satisfying ; yes A diagonal matrix, where the elements on the diagonal are the singular values ​​of matrix H, arranged in descending order, and are all non-negative real numbers; yes Right singular orthogonal matrices satisfying .

[0054] Solve for the optimal rotation matrix: .

[0055] Based on the SVD decomposition results, the optimal rotation matrix between the two sets of point sets is solved, while the differences in the rotation matrices are corrected to avoid the appearance of reflection matrices. Construct a diagonal matrix, with the first two diagonal elements fixed at 1, and the third element being the matrix itself. The determinant value, for The transpose of .

[0056] Solving for the optimal translation vector: .

[0057] After solving for the optimal rotation matrix, the optimal translation vector between the two sets of points is calculated to complete the solution of the rigid transformation matrix.

[0058] Based on the initial transformation matrix, the camera poses in all local coordinate systems within the tunnel are uniformly mapped to the world coordinate system: .

[0059] in, The first tunnel in the world coordinate system The pose of the frame camera is made consistent with the pose acquired by the external drone. Local coordinate system The first tunnel inside The pose of the frame camera. It is a Euclidean transformation matrix.

[0060] To address the scale drift problem that easily occurs in visual inertial odometry in long tunnel scenarios, a scale correction factor is calculated using common feature points. Eliminate scale bias between internal and external scenes: .

[0061] By incorporating the scale factor into the transformation matrix, scale correction of the pose within the tunnel is achieved. The corrected coordinate transformation and pose mapping formula is then updated as follows: , .

[0062] To further eliminate the cumulative error in pose calculation, a joint bundle adjustment (BA) optimization model is constructed, which includes external UAV keyframes, internal vehicle-mounted keyframes, and all 3D feature points. The objective function is to minimize the sum of reprojection errors of all keyframes. ,in, For all keyframe camera poses to be optimized (including external ones) Frame, Internal frame), Let the world coordinates of all 3D feature points to be optimized be... For the first The number of feature points observed in each keyframe This is a camera pinhole projection model that projects 3D world coordinates onto the image pixel plane. For feature points at the th Pixel observation coordinates on the frame image For the first The spatial coordinates of a three-dimensional feature point in the world coordinate system.

[0063] For long / extra-long tunnels with a length of 1 km or more, a global closed-loop constraint is added at the tunnel exit. The internal and external fusion strategy of the entrance is reused at the tunnel exit, and a closed-loop constraint term is constructed by extracting the common feature region at the exit. This term is then added to the optimization objective function to eliminate long-mileage trajectory cumulative drift.

[0064] in, The first tunnel in the world coordinate system The pose of the frame camera. The camera pose was captured from outside the tunnel exit. The coordinate system transformation matrix to be solved at the exit point. This is a partial pose of the camera captured inside the exit area. The Frobenius norm is used to measure the pose transformation residuals. The weighting coefficients for the closed-loop constraints, with a default value of 10 to 100.

[0065] The above fusion scheme achieves complete unification of pose references for scenes inside and outside the tunnel, solving industry pain points such as lack of GNSS signals, long-distance drift, and disconnect between internal and external models in tunnel scenes. It provides high-precision, full-coverage pose and image data support for the initialization and optimization of subsequent 3D Gaussian scene models.

[0066] Construction of 3D scene model: This application abandons the traditional triangular mesh or voxel representation and instead uses a discrete, differentiable set of 3D Gaussian spheres to represent the highway engineering scene.

[0067] Setting the properties of a 3D Gaussian sphere: Each 3D Gaussian sphere in the scene is determined by the following parameters: Position, ): Three-dimensional vector , where represents the center of the Gaussian sphere in the world coordinate system.

[0068] Covariance matrix ): A matrix that controls the shape and orientation of a Gaussian sphere (an anisotropic ellipsoid).

[0069] Opacity ): Scalar, range of values This indicates the ability of the Gaussian sphere to block light.

[0070] Spherical Harmonics : Used to fit view-dependent color, enabling the reconstructed model to represent anisotropic material effects such as road surface reflection and metal guardrail gloss.

[0071] As shown in Figure 3, the 3D-to-2D projection transformation (Splatting) employs a splatting technique to render a 3D scene onto a 2D screen. Given the view transformation matrix of the current camera... (View Matrix) The process of projecting a three-dimensional Gaussian sphere onto a two-dimensional image plane is described by the transformation of its covariance matrix. The projected two-dimensional covariance matrix... The calculation formula is: ,in: It is the three-dimensional covariance matrix obtained through training; It is the affine approximation Jacobian matrix of the projective transformation; It is the transformation matrix from the world coordinate system to the camera coordinate system. This is the transpose of the matrix. This formula is the mathematical foundation for the correct mapping from three-dimensional space to two-dimensional pixel space in this application.

[0072] Model training based on adaptive density control: The purpose of model training is to optimize the above Gaussian parameters so that the rendered image is as consistent as possible with the acquired real image.

[0073] Initialization (SfM Initialization): Using the Structure from Motion (SfM) algorithm, sparse point cloud reconstruction is performed on the acquired image using COLMAP. The coordinates of the generated sparse point cloud are directly used as the initial positions of the 3D Gaussian sphere. The scaling factor is initialized based on the point cloud density, thereby accelerating training convergence and avoiding local optima.

[0074] Loss Function Calculation and Backpropagation: In each iteration, the rendered image is compared with the real image, and the loss function is calculated. The loss function is composed of L1 norm and D-SSIM (structural similarity) weights. The mathematical expression is as follows: ,in, It is primarily responsible for pixel-level color alignment. It calculates the average absolute error of the color value of each pixel between the rendered image and the real image, ensuring accurate color reproduction. Its primary responsibility is to maintain the structural features of the image (such as the continuity of road surface texture and the sharpness of road markings). It is typically defined as... Similarity is evaluated by comparing the brightness, contrast, and structure of images. Hyperparameters are used to balance the weights of both. Automatic differentiation is performed using frameworks such as PyTorch to calculate the gradient of the loss function with respect to position, rotation, scaling, opacity, and spherical harmonics, and the parameters are updated using gradient descent (Adam Optimizer).

[0075] Adaptive Density Control: To address the challenges of complex road surface textures and large distances between near and far views, this application introduces a dynamic Gaussian sphere splitting and trimming mechanism: Clone: ​​In regions with large gradients (indicating large reconstruction errors) and small Gaussian sphere volumes (typically regions rich in detail, such as road surface cracks), a new Gaussian sphere is copied to increase the sampling density.

[0076] Split: In regions with large gradients and excessively large Gaussian sphere volumes (indicating that the Gaussian sphere over-covers multiple features, such as roadbed edges), the large Gaussian sphere is split into two smaller Gaussian spheres to improve geometric accuracy.

[0077] Pruning: Regularly detect and remove opacity. Below the threshold (e.g.) Alternatively, use a Gaussian sphere that is too large (over-expanding) to eliminate floating noise in the scene and reduce the model's memory usage.

[0078] Tile-based Differentiable Rasterization Rendering: To achieve real-time rendering on both desktop and web platforms, this application designs an efficient rasterization rendering pipeline.

[0079] Frust culling and Tiling: First, the Gaussian sphere outside the view frustum is culled. The screen is then divided into sections. Tiles of pixels.

[0080] Depth Sorting: For each Gaussian sphere covering a tile, sort it cardinally based on its depth value from the camera. First, sort the depth values ​​of the center point of each feature sphere in the camera coordinate system. (Floating-point numbers) are converted to 32-bit unsigned integers to ensure efficient bitwise operations during sorting while maintaining depth monotonicity. This is achieved through parallel radix sorting based on GPU thread blocks. Unlike traditional comparison sorting, radix sort has a time complexity of O(log n). ,in With a bit depth, it is ideal for parallel processing of large-scale feature balls.

[0081] Pixel-level color accumulation (Alpha Blending): For each pixel, the color contribution of the Gaussian sphere is accumulated in order of depth from near to far. Final pixel color. The calculation formula follows the discrete form of the volume rendering equation: ,in, It is the first The color of the Gaussian sphere It refers to its opacity. When the accumulated opacity reaches saturation (close to 1), the calculation of that pixel stops to reduce unnecessary calculations.

[0082] Engineering application components: To enhance the practicality of the technical solution, this application also includes the following optional engineering features: Lightweight Web-based loader: A rendering viewer developed based on the WebGL / WebGPU standard, which supports directly loading compressed 3D Gaussian model files (.ply or .splat format) in the browser without the need to install professional software.

[0083] Interactive roaming control: Integrates a roaming controller customized for highway engineering, supporting automatic flight mode along the route, fixed-point observation mode, and switching between first-person perspective.

[0084] In another specific embodiment, the "K15+000 to K17+000" section of a highway reconstruction and expansion project, which is 2 kilometers long and includes a complex interchange and a tunnel entrance about 300 meters long, is taken as the specific application background.

[0085] Contextualized data acquisition and pose calculation: During the data acquisition phase, the system automatically executes differentiated strategies based on the scenario type.

[0086] 1. Parallax control calculation for linear highways: For the straight road surface of section K15, "multi-altitude linear flight" is performed.

[0087] Input parameters: Set low-altitude flight altitude Target pitch angle Local normal vector of the road surface .

[0088] Calculation process: According to the formula Calculate the principal optical axis vector of the camera. .when hour, This determines the camera attitude control commands, creating a cross parallax between the principal optical axis and the ground, ensuring that the anisotropic features of road markings and cracks are fully captured.

[0089] 2. Example of rigid fusion of tunnel internal and external coordinate systems: For the tunnel entrance area, it is necessary to fuse the video trajectory inside the tunnel without GNSS into the world coordinate system. .

[0090] Input parameters: Extract pairs of feature points with the same name from the common area of ​​the cave entrance, where one of the points has coordinates in the world coordinate system. Coordinates in the local coordinate system of the tunnel .

[0091] Calculation process: The transformation matrix is ​​calculated through SVD decomposition. Calculate the scaling correction factor. To eliminate drift: Applying the mapping formula This unifies the poses of all images inside the cave to the engineering geographic coordinate system, achieving seamless integration of internal and external models.

[0092] Explicit radiation field model construction and training: 1. 3D to 2D projection transformation process: taking the 3D Gaussian sphere projection in the bridge scene training process as an example.

[0093] Input parameter: Position of the Gaussian sphere 3D covariance matrix (Defined as a long ellipsoid in the cable direction), camera view transformation matrix .

[0094] Calculation process: Applying the formula .

[0095] in, This is the Jacobian matrix. The calculated value is... This represents the projection range of the ellipsoid onto the image plane. If the calculation shows that its screen coverage radius exceeds a threshold, then the feature sphere will participate in the color contribution calculation for that pixel.

[0096] 2. Adaptive Density Control Decision: After 3000 iterations, the system detects a cracked area in the road surface. Input value: The average gradient value of this area. (Exceeds the threshold of 0.0002), and the Gaussian sphere scaling factor s is extremely small.

[0097] Operation: Trigger the "cloning" mechanism to replicate the Gaussian sphere at that location to increase the feature sampling density until the loss function... The crack details are reduced to the target range, thus restoring them with high fidelity.

[0098] Rendering pipeline sorting: When rendering the complex, interwoven structure of a bridge, radix sorting is performed.

[0099] Example data: A tile contains 3 Gaussian spheres with depth values ​​of... .

[0100] Calculation process: Convert the floating-point depth to a 32-bit ordered integer. Using GPU parallel radix sort, the resulting sequence is as follows: Pixel color calculation: based on the formula First, accumulate the depth of the nearest The color rendering process stops when the cumulative transparency reaches saturation. The training process and rendering results are shown in Figure 4. This algorithm achieves real-time rendering of large scenes at over 60 FPS on an RTX 3060 graphics card.

[0101] Figure 4 illustrates the desktop execution of the rapid reconstruction prototype system built based on this invention. The interface displays the entire rendering application process. By uploading multi-angle photos or videos to the platform and placing them in the training queue to initialize point cloud information, iterative training of a three-dimensional Gaussian sphere is performed based on the reference images. After 10 minutes of training on an Nvidia RTX 4090 graphics card, the training results are viewed. The reconstructed high-precision 3D road scene can be seen in 60fps real-time rendering on an Nvidia RTX 3060 graphics card, verifying the texture preservation capability of the method. It also demonstrates the interactive functions of user-controlled viewpoint movement, rotation, and zoom via mouse or touch operation, showcasing the high frame rate characteristics of real-time rendering.

[0102] The embodiments described above merely illustrate specific implementation methods of this application, and while the descriptions are detailed and specific, they should not be construed as limiting the scope of protection of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the technical solution of this application, and these modifications and improvements all fall within the scope of protection of this application.

Claims

1. A method for rapid reconstruction of a full-scene highway project based on multi-source parallax control and adaptive explicit radiation field, characterized in that, Includes the following steps: Data Acquisition and Pose Calculation: Differentiated acquisition strategies are used to acquire multi-source image data for different scenarios in highway engineering, and the camera pose of each frame is calculated; 3D Gaussian Scene Model Construction: The scene is represented by an anisotropic set of 3D Gaussian spheres, each Gaussian sphere being defined by position, covariance matrix, opacity, and spherical harmonic coefficients; Model Training and Optimization: Based on image data and camera pose, the attribute parameters of the 3D Gaussian spheres are iteratively optimized until convergence through differentiable rendering and adaptive density control strategies; Real-time Rendering: A tile-based differentiable rasterization rendering pipeline is used to project the optimized 3D Gaussian sphere set onto a 2D plane to generate a real-time interactive 3D scene.

2. The method for rapid reconstruction of a full-scene highway network based on multi-source parallax control and adaptive explicit radiation field as described in claim 1, characterized in that, In the data acquisition and pose calculation, the internal and external fusion acquisition strategy for the tunnel scene includes: acquiring the tunnel entrance image sequence captured by the UAV outside the tunnel and its coordinates in the world coordinate system. First camera pose ,in Let be the rotation matrix from the world coordinate system to the camera coordinate system. For the corresponding translation vector, For special Euclidean groups; to obtain tunnel image sequences acquired by vehicle-mounted or handheld devices inside the tunnel and their local coordinate system. The second camera pose is below; extract the same feature points in the public area of ​​the cave entrance. ， The three-dimensional coordinates of the feature point in the world coordinate system. Given the 3D coordinates of feature points in the local coordinate system, mismatched point pairs with a reprojection error greater than 2 pixels are removed. This represents the number of high-precision matching points retained after removing mismatches; based on the remaining feature point pairs, the Euclidean transformation matrix from the local coordinate system to the world coordinate system is solved through singular value decomposition. And calculate the scale correction factor: ,in These are the centroids of the two sets of feature points; according to the mapping formula The pose of the second camera is uniformly mapped to the world coordinate system, achieving seamless fusion of scenes inside and outside the tunnel; among which, Let be a rotation matrix. It is a translation vector. The first tunnel in the world coordinate system The pose of the frame camera. Local coordinate system The first tunnel inside The pose of the frame camera.

3. The method for rapid reconstruction of a full-scene highway network based on multi-source parallax control and adaptive explicit radiation field as described in claim 2, characterized in that, For long tunnels with a length of 1 km or more, the internal and external fusion acquisition strategy further includes: reusing the internal and external fusion strategy at the tunnel exit to extract common feature regions at the exit and construct closed-loop constraints; and adding the closed-loop constraint term to the objective function of the joint bundle adjustment optimization. in, The first tunnel in the world coordinate system The pose of the frame camera, The camera pose is captured from outside the exit. The coordinate system transformation matrix to be solved at the exit point. This is a partial pose of the camera captured inside the exit area. It is the Frobenius norm. The weighting coefficient for the closed-loop constraint has a value range of 10 to 100 to eliminate the cumulative drift of long-mileage trajectories. For all keyframe camera poses to be optimized, Let the world coordinates of all 3D feature points to be optimized be... For the first The number of feature points observed in each keyframe This is a camera pinhole projection model that projects 3D world coordinates onto the image pixel plane. For feature points at the th Pixel observation coordinates on the frame image For the first The spatial coordinates of a three-dimensional feature point in the world coordinate system.

4. The method for rapid reconstruction of a full-scene highway network based on multi-source parallax control and adaptive explicit radiation field as described in claim 1, characterized in that, In the data acquisition and pose calculation, the acquisition strategy for the bridge scenario includes: controlling the UAV to fly around the key structural nodes of the bridge along a parameterized spiral trajectory, which is defined by the following parametric equations: ， ， ,in, 、 、 These are the geometric center coordinates of the key structural nodes. For the observation radius, The vertical lifting speed, Angular velocity; real-time calculation of UAV attitude to ensure accuracy at each sampling point. Camera optical axis and vector Overlapping is used to achieve full-dimensional data coverage of the blind spots at the bottom and sides of the bridge.

5. The method for rapid reconstruction of a full-scene highway network based on multi-source parallax control and adaptive explicit radiation field as described in claim 1, characterized in that, In the data acquisition and pose calculation, the acquisition strategy for linear highway scenarios includes: controlling the UAV to fly back and forth at low and high altitudes above the centerline of the road, and dynamically adjusting the camera pitch angle. To make it satisfy the constraints: ,in, Let the principal optical axis vector of the camera be _____. ， This is the local normal vector of the road surface, used to obtain the optimal anisotropic texture information.

6. The method for rapid reconstruction of a full-scene highway network based on multi-source parallax control and adaptive explicit radiation field as described in claim 2, characterized in that, In the data acquisition and pose calculation, the data acquisition inside the tunnel includes: controlling the acquisition vehicle to travel at a constant speed along the centerline of the tunnel lane, with the speed satisfying the following constraints: ,in, For camera sampling frame rate, The maximum baseline distance between adjacent frames is used to ensure that the overlap rate of adjacent frames is ≥80%, and the ratio of inter-frame baseline to scene depth is controlled in the range of 0.1~0.

5.

7. The method for rapid reconstruction of a full-scene highway network based on multi-source parallax control and adaptive explicit radiation field as described in claim 1, characterized in that, In the construction of the 3D Gaussian scene model, the process of projecting a 3D Gaussian sphere onto a 2D image plane is described by the covariance projection formula: ,in, It is the covariance matrix of a three-dimensional Gaussian sphere. It is the view transformation matrix from the world coordinate system to the camera coordinate system. It is the affine approximation Jacobian matrix of the projective transformation. It is the projected two-dimensional covariance matrix. This is the transpose of the matrix.

8. The method for rapid reconstruction of a full-scene highway network based on multi-source parallax control and adaptive explicit radiation field as described in claim 1, characterized in that, The initialization in the model training and optimization includes: using the structure-of-motion reconstruction algorithm to perform sparse reconstruction on the acquired image data, and directly using the generated sparse point cloud coordinates as the initial position of the three-dimensional Gaussian sphere. The scaling factor is initialized based on the comparison between the local density of the point cloud and the preset density threshold. When the local density of the point cloud is higher than the first density threshold, the scaling factor of the corresponding Gaussian sphere is initialized to the first scaling value. When the local density of the point cloud is lower than the second density threshold, the scaling factor of the corresponding Gaussian sphere is initialized to the second scaling value. The first scaling value is less than the second scaling value, so as to accelerate training convergence and avoid local optima.

9. The method for rapid reconstruction of a full-scene highway network based on multi-source parallax control and adaptive explicit radiation field as described in claim 1, characterized in that, The loss function used in the model training and optimization pixel-level L1 norm loss and structural similarity loss The weighted composition is expressed as follows: ,in, ， To render the image With real images Structural similarity index between them The weighting coefficients for balancing the two types of losses are set between 0.2 and 0.

8.

10. The method for rapid reconstruction of a full-scene highway network based on multi-source parallax control and adaptive explicit radiation field as described in claim 1, characterized in that, The real-time rendering specifically includes: dividing the screen into multiple... The pixel-based patch is processed, and Gaussian spheres outside the view frustum are removed; for each patch covering a Gaussian sphere, its floating-point depth value is calculated. Convert to 32-bit ordered integer key value ;Utilize GPU thread blocks to analyze the key-value pairs within each tile Parallel radix sorting is performed to determine the depth order of Gaussian spheres within the same patch. For each pixel, the color contribution of the Gaussian spheres is accumulated using an alpha blending formula, in order of increasing depth, until the accumulated opacity reaches 1. The alpha blending formula is as follows: ,in, It is the first The color of the Gaussian sphere It is its opacity, The set of Gaussian spheres covered by the current pixel.

Images