A structural semantic guided bridge point cloud gaussian splatting modeling and rendering method

Abstract

The application discloses a kind of structural semantic guided bridge point cloud Gaussian splat modeling and rendering method, belong to computer vision and bridge engineering digitization field.The method includes: collecting bridge multi-view image, obtains sparse point cloud and camera pose by motion recovery structure technology;Generate dense point cloud and carry out deep learning semantic segmentation, identify bridge pier, bridge deck, cable and other key components;The semantic label is mapped back to sparse point cloud;Based on semantic label, the difference of sparse point cloud is initialized Gaussian element, and the initial shape of different components is given to meet its geometric characteristics;Introduce geometric constraint loss function in optimization process to carry out joint optimization;Finally output high-fidelity and accurate geometric structure Gaussian splat model.The application solves the model redundancy and geometric distortion problem when traditional Gaussian splat technology processes bridge scene, realizes the lightweight, high-precision reconstruction and real-time rendering of bridge model.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and digital bridge engineering technology, specifically to a structural semantic-guided Gaussian splash modeling and rendering method for bridge point clouds. Background Technology

[0002] Currently, technologies such as terrestrial laser scanning, airborne LiDAR, and UAV photography are increasingly used to acquire bridge point clouds. However, traditional technologies suffer from problems such as scanning blind spots, insufficient accuracy and stability, and difficulty in lightweighting massive point clouds. Furthermore, traditional triangular mesh models are bloated, prone to geometric distortion, and unable to dynamically adjust detail levels, hindering the demand for refined 3D modeling and real-time visualization of bridges. With the advancement of digital operation and maintenance of infrastructure, there is an urgent need to solve the core challenge of balancing model accuracy and lightweight design.

[0003] Current 3D Gaussian splashing technology has achieved a breakthrough in real-time rendering of large-scale scenes, and point cloud semantic segmentation technology can also automatically identify bridge components such as piers. Both provide new directions for bridge modeling. However, the former is prone to geometric distortion and model redundancy due to the lack of structural priors, and the latter's semantic information does not provide in-depth guidance for reconstruction and optimization. Therefore, it still has obvious limitations in the application of bridge structured scenarios.

[0004] In summary, traditional modeling techniques have significant shortcomings, while emerging technologies have application limitations. To meet the demands for high-precision, lightweight modeling and real-time visualization of bridges, there is an urgent need for a new solution that deeply integrates structural semantics into the reconstruction process, so as to retain the advantages of 3D Gaussian splash rendering while ensuring the geometric accuracy and engineering practicality of the model. Summary of the Invention

[0005] The purpose of this invention is to propose a structural semantic-guided Gaussian splash modeling and rendering method for bridge point clouds to solve the problems in the background technology.

[0006] The objective of this invention is achieved through the following technical solution: A structural semantic-guided Gaussian splash modeling and rendering method for bridge point clouds includes the following steps: S1: Bridge Scene Data Acquisition and Sparse 3D Reconstruction Multi-view sequence images of the bridge are acquired by cameras mounted on drones or ground equipment; sparse point clouds of the bridge and camera poses of each image are obtained from the sequence images using structure-reconstruction-motion technique. S2: Dense Point Cloud Generation and Structural Semantic Segmentation Based on the sparse point cloud obtained from S1 and the camera pose, a dense point cloud of the bridge is generated through a multi-view stereo vision algorithm; the dense point cloud is segmented using a pre-trained bridge point cloud semantic segmentation model, and key structural components such as piers, bridge deck, and cables are identified and labeled. S3: Mapping of semantic labels to sparse point clouds The structural semantic labels obtained from the dense point cloud in S2 are associated with the sparse point cloud obtained in S1 through spatial mapping relationship, thereby obtaining a sparse point cloud with semantic labels. S4: Structural semantic-guided differential Gaussian initialization Based on the semantic tags obtained in S3, the points in the sparse point cloud are differentiated, and the Gaussian elements of points of different structure categories are initialized to the shape that conforms to the geometric characteristics of that category.

[0007] S5: Joint Optimization of Gaussian Splash Model with Geometric Constraints Based on the Gaussian units initialized by S4, and combined with the camera pose obtained by S1, differentiable rendering is performed, and the difference between the rendered image and the original image is minimized by an optimization algorithm. In this optimization process, corresponding geometric constraint loss functions are introduced for Gaussian units with different semantic labels, and optimization is performed together. S6: High-quality Gaussian splash model output and rendering Output an optimized Gaussian splash model, which includes the position, covariance, color, and opacity parameters of each Gaussian element; use a graphics renderer to perform real-time, multi-view immersive rendering of the model.

[0008] Preferably, S1 specifically includes: S11. Plan the flight path of the UAV to collect images of the bridge deck, bottom, sides and piers from multiple angles with high overlap, and construct a sequence of images covering the complete structure of the bridge. S12. Extract feature points from the sequence of images using the scale-invariant feature transform algorithm; S13. Use a feature matching algorithm to match feature points of adjacent images to obtain matching point pairs; S14. The incremental motion recovery structure method is adopted. The initial image pair is selected to initialize the 3D model, and the PnP algorithm is used to calculate the camera pose of the new image one by one. S15. Calculate the three-dimensional coordinates of the matching feature points using triangulation, and simultaneously optimize the camera pose and three-dimensional point coordinates using bundle adjustment to obtain the optimized sparse point cloud and accurate camera pose.

[0009] Preferably, S2 specifically includes: S21. Based on the sparse point cloud obtained from S1 and the camera pose, a dense point cloud on the bridge surface is generated by stereo matching and depth map fusion in a multi-view stereo vision algorithm. S22. Construct a point cloud dataset containing semantic labels for bridge components such as piers, bridge decks, and cables; S23. The RandLA-Net point cloud deep learning network is trained on the dataset to obtain the pre-trained bridge point cloud semantic segmentation model. S24. Input the dense point cloud generated in S21 into the trained semantic segmentation model and output the semantic label of the structural component corresponding to each 3D point.

[0010] Preferably, S3 specifically includes: S31. For each point in the sparse point cloud, use the spatial indexing algorithm to find its K nearest neighbors in the dense point cloud. S32. Calculate the semantic label distribution of the K nearest neighbors; S33. Assign the most frequently occurring semantic label to the sparse point to complete the mapping of semantic labels from dense point cloud to sparse point cloud.

[0011] Preferably, S4 specifically includes: S41. For the set of points labeled as planar structural components, principal component analysis is used to fit the plane in which the points are located to obtain the plane normal vector. S42. Adjust the covariance matrix of the Gaussian element corresponding to each point in the component according to the plane normal vector, so that it is flattened and perpendicular to the direction of the normal vector, so as to conform to the plane geometric characteristics. S43. For the set of points labeled as linear structural components, the centerline is fitted using the random sampling consensus algorithm to obtain the tangent direction of the centerline; S44. Adjust the covariance matrix of the Gaussian element corresponding to each point in the component according to the tangent direction, so that it is elongated and parallel to the tangent direction to conform to linear geometric characteristics. S45. For the point set labeled as a regular columnar structural component, the dominant axis is obtained by using an axial fitting method based on principal component analysis. S46. Adjust the covariance matrix of the Gaussian element corresponding to each point in the component according to the dominant axis described in S45, so that its shape conforms to the geometric characteristics of the regular cylindrical component.

[0012] Preferably, S5 specifically includes: S51. Construct the total loss function, which is a weighted sum of photometric reconstruction loss, coplanar constraint loss, collinear constraint loss, and regular cylinder constraint loss, and its expression is: L total =λ photo L photo +λ plane L plane +λ line L line +λ reg L reg , Among them, L photo For photometric reconstruction loss, L plane With L line These represent the coplanar and collinear constraint losses, respectively, L reg λ represents the loss due to the constraint of the regular column, and λ is the weight coefficient of each loss term. S52. Using a differentiable renderer, render and generate an image based on the current Gaussian meta-parameters and camera pose, and calculate the photometric reconstruction loss L between the generated image and the original input image. photo ; S53. For all Gaussian elements belonging to the same planar structural component, calculate the sum of squared distances from their center points to the fitted reference plane, and use this as the coplanar constraint loss L. plane ; S54. For all Gaussian elements belonging to the same linear structural component, calculate the sum of squared distances from their center points to the fitted center line, and use this as the collinearity constraint loss L. line ; S55. For all Gaussian elements belonging to the same regular columnar structural component, calculate the sum of the squares of their distances from the geometric properties of the regular column, and use this sum as the constraint loss L of the regular column. reg ; S56. Using the Adam optimizer, minimize the total loss function L. total Iteratively update the position, covariance, color, and opacity parameters of all Gaussian elements; S57. During the optimization process, adaptive density control is performed periodically: Gaussian elements with large positional gradients and large scales are cloned to fill in details, while Gaussian elements with opacity below a set threshold are removed to eliminate redundancy.

[0013] Preferably, S6 specifically includes: S61. Save the optimized Gaussian splash model. The model data includes the three-dimensional position of each Gaussian element, a 3×3 covariance matrix, RGB color values, opacity alpha values, and optional semantic label information. S62. Load the model data using a tile-based rasterization renderer; S63. For a given rendering viewpoint, sort the depths according to the distance between the Gaussian units and the viewpoint. S64. Project each Gaussian element onto the two-dimensional image plane in sequence, perform Alpha blending calculation, and finally output a high-fidelity bridge rendering image.

[0014] Compared with the prior art, the present invention has the following significant advantages: First, this invention creatively integrates point cloud semantic segmentation results with the dynamic reconstruction process of 3D Gaussian splashing, overcoming the inherent defects of standard 3D Gaussian splashing in highly structured bridge scenarios by utilizing a "structural semantic guidance" mechanism. By assigning initial shapes to Gaussian elements of different structural components that conform to their geometric characteristics, and applying strict geometric constraints from the beginning of the optimization process, the "geometric distortion" phenomenon is fundamentally and effectively suppressed, significantly improving the geometric accuracy and structural rationality of the generated model.

[0015] Secondly, through the aforementioned semantically guided differential initialization and constraint optimization, this invention makes the distribution of Gaussian units in the scene more consistent with the physical structure of the bridge, avoiding the generation of unnecessary redundant Gaussian units in flat or regular areas. At the same time, it guides resources to be more accurately focused on the depiction of realistic details, thereby achieving "lightweighting" of the model while ensuring high-fidelity visual effects. This significantly reduces the amount of model data and improves the real-time rendering and interaction efficiency on lightweight platforms such as Web and mobile devices.

[0016] Furthermore, the method described in this invention has broad applicability and scalability. Its proposed "semantic segmentation-label mapping-differential initialization-constraint optimization" framework is not only applicable to bridges but can also be transferred to the 3D modeling of other buildings or infrastructures with distinct structural features. The Gaussian splash model generated by this method, rich in semantic information, lays a solid foundation for further advanced applications such as component-level querying and structural deformation analysis.

[0017] Finally, this invention has significant value in engineering practice. It provides a complete technical path from ordinary UAV imagery to high-precision, real-time interactive digital models of bridges, effectively resolving the contradictions between cost, efficiency, and effectiveness in traditional methods, and providing a powerful technical tool for rapid bridge inspection, health monitoring, and the construction of digital twin systems. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings involved in the embodiments are now briefly described. Obviously, the drawings in the following description are merely illustrative of some embodiments of the present invention. For those skilled in the art, other forms of drawings can be constructed based on these drawings without creative effort.

[0019] Figure 1 This is the overall flowchart of the structural semantic-guided Gaussian splash modeling and rendering method for bridge point clouds proposed in this invention; Figure 2 This is the original image of the bridge captured by the UAV in this embodiment of the invention; Figure 3 This is a schematic diagram of sparse point cloud of a bridge in an embodiment of the present invention; Figure 4 This is a schematic diagram of the Gaussian ellipsoid representation of the sparse point cloud of a bridge in an embodiment of the present invention; Figure 5 This is the rendering result of the bridge in an embodiment of the present invention. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this application clearer, the application is described and illustrated below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application. All other embodiments obtained by those skilled in the art based on the embodiments provided in this application without inventive effort are within the scope of protection of this application.

[0021] It should be understood that the step numbers used in the text are for ease of description only and are not intended to limit the order in which the steps are performed.

[0022] It should be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0023] The terms “comprising” and “including” indicate the presence of the described feature, whole, step, operation, element and / or component, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or collections thereof.

[0024] This invention addresses the problems of data bloat and loss of detail in traditional mesh models for bridge scenarios, as well as the model redundancy and geometric distortion caused by the lack of prior structural knowledge in standard 3D Gaussian splashing technology. By deeply coupling point cloud semantic segmentation technology with 3D Gaussian splashing, it innovatively guides the initialization and optimization process of Gaussian elements through structural semantics. This method retains the advantages of high fidelity and real-time rendering of 3D Gaussian splashing while generating geometrically accurate and structurally lightweight bridge models, providing an efficient technical means for the digital twinning and intelligent operation and maintenance of bridges. The following description, with reference to accompanying figures and specific examples, illustrates the structural semantic-guided bridge point cloud Gaussian splashing modeling and rendering method proposed in this invention.

[0025] Example 1: Please see Figure 1 This example proposes a Gaussian splash modeling and rendering method for point clouds of cable-stayed bridges guided by structural semantics, including the following steps: S1. Data Acquisition and Sparse 3D Reconstruction of Cable-Stayed Bridge Scene Multi-view image sequences of a cable-stayed bridge are acquired using cameras mounted on drones or ground equipment; sparse point clouds of the cable-stayed bridge and camera poses for each image are obtained from the image sequences using structure-of-motion (SOR) techniques; specifically including: S11. Plan the drone flight path to acquire multi-angle, high-overlap images of the cable-stayed bridge deck, cable system, and main tower, and construct a sequence of images covering the complete structure of the cable-stayed bridge (original images of the cable-stayed bridge are shown below). Figure 2 (as shown) S12. Extract feature points from the sequence of images using the scale-invariant feature transform algorithm; S13. Use a feature matching algorithm to match feature points of adjacent images to obtain matching point pairs; S14. The incremental motion recovery structure method is adopted. The initial image pair is selected to initialize the 3D model, and the PnP algorithm is used to calculate the camera pose of the new image one by one. S15. Calculate the 3D coordinates of the matching feature points using triangulation, and simultaneously optimize the camera pose and 3D point coordinates using bundle adjustment to obtain the optimized sparse point cloud of the cable-stayed bridge (e.g., ...). Figure 3 (as shown) and precise camera pose; S2, Dense Point Cloud Generation and Structural Semantic Segmentation Based on the sparse point cloud obtained from S1 and the camera pose, a dense point cloud of the cable-stayed bridge is generated using a multi-view stereo vision algorithm. A pre-trained semantic segmentation model for cable-stayed bridge point clouds is then used to segment the dense point cloud, identifying and labeling key structural components such as the main tower, bridge deck, and cables. Specifically, this includes... S21. Based on the sparse point cloud obtained from S1 and the camera pose, a dense point cloud on the surface of the cable-stayed bridge is generated by stereo matching and depth map fusion in a multi-view stereo vision algorithm. S22. Construct a point cloud dataset containing semantic labels for cable-stayed bridge components such as the main tower, bridge deck, and cables; S23. The RandLA-Net point cloud deep learning network is trained on the dataset to obtain the pre-trained cable-stayed bridge point cloud semantic segmentation model. S24. Input the dense point cloud generated in S21 into the trained semantic segmentation model and output the semantic label of the structural component corresponding to each 3D point. S3. Mapping of semantic labels to sparse point clouds The structural semantic labels obtained from the dense point cloud in S2 are associated with the sparse point cloud obtained in S1 through spatial mapping relationships, thereby obtaining a sparse point cloud with semantic labels; specifically including: S31. For each point in the sparse point cloud, use the spatial indexing algorithm to find its K nearest neighbors in the dense point cloud. S32. Calculate the semantic label distribution of the K nearest neighbors; S33. Assign the most frequently occurring semantic label to the sparse point to complete the mapping of semantic labels from dense point cloud to sparse point cloud; S4. Structural semantic-guided differential Gaussian initialization Based on the semantic tags obtained in S3, the points in the sparse point cloud are differentiated by initializing their Gaussian elements to conform to the geometric characteristics of the category for points of different structure categories. Specifically, this includes: S41. For the set of points labeled as the bridge surface, use principal component analysis to fit the plane in which it lies and obtain the plane normal vector. S42. Adjust the covariance matrix of the Gaussian element corresponding to each point in the component according to the plane normal vector. Specifically, significantly reduce the eigenvalues ​​of the covariance matrix in the direction of the normal vector so that it is flattened into an ellipsoid and laid parallel to the bridge surface to conform to the plane geometric characteristics. S43. For the set of points labeled as cables, use the random sampling consensus algorithm to fit its centerline and obtain the tangent direction of the centerline; S44. Adjust the covariance matrix of the Gaussian element corresponding to each point in the component according to the tangent direction. Specifically, significantly increase the eigenvalues ​​of the covariance matrix in the tangent direction and shrink them in the vertical direction to make it into a slender ellipsoid and extend along the cable direction to conform to linear geometric characteristics. S45. For the point set labeled as the main tower, the directional bounding box fitting algorithm based on principal component analysis is used to obtain its three mutually orthogonal principal axis directions v1, v2, v3. S46. Adjust the covariance matrix of the Gaussian element corresponding to each point in the component according to the three principal axis directions v1, v2, v3, so that its shape is consistent with the direction of the fitted cuboid bounding box, that is, the three characteristic axes of the Gaussian element are parallel to v1, v2, v3 respectively.

[0026] S5, Joint Optimization of Gaussian Splash Model with Incorporated Geometric Constraints Based on the Gaussian units initialized in S4, differentiable rendering is performed using the camera pose obtained in S1, and the difference between the rendered image and the original image is minimized through an optimization algorithm. During this optimization process, corresponding geometric constraint loss functions are introduced for Gaussian units with different semantic labels to jointly optimize the rendering. Specifically, this includes: S51. Construct the total loss function, which is a weighted sum of photometric reconstruction loss, coplanar constraint loss, collinear constraint loss, and cuboid constraint loss, and its expression is: L total = λ photo L photo + λ plane Lplane + λ line L line + λ cuboid L cuboid , Among them, L photo For photometric reconstruction loss, L plane With L line These represent the coplanar and collinear constraint losses, respectively, L cuboid Let λ represent the cuboid constraint loss, and λ be the weighting coefficient of each loss term. S52. Using a differentiable renderer, render and generate an image based on the current Gaussian meta-parameters and camera pose, and calculate the photometric reconstruction loss L between the generated image and the original input image. photo The photometric reconstruction loss L photo A weighted combination of L1 loss and structural similarity loss is used, and its calculation formula is as follows: L photo = (1 - α) L1(I gt , I r ) + α L D-SSIM (I gt , I r ), Among them, I gt Represents the original input real image; I r L1(I) represents the synthesized image generated by the current Gaussian model rendering. gt , I r ) represents I gt with I r The L1 norm loss per pixel is used to measure the absolute difference; L D-SSIM (I gt , I r ) represents a loss function based on structural similarity, used to perceive differences in structural information of images; α is the fusion weight of the two loss terms, which is an adjustable hyperparameter, typically ranging from 0 to 1.

[0027] S53. For all Gaussian elements belonging to the same bridge deck, calculate the sum of squared distances from their center points to the fitted reference plane, and use this sum as the coplanar constraint loss L. plane The specific calculation formula is as follows: L plane =Σ i (n plane ·(p i - c plane )) 2 , Where p i For the center of the Gaussian unit, c plane Assign the cloud centroid to the bridge surface, n plane This is the bridge surface normal vector; S54. For all Gaussian elements belonging to the same cable, calculate the sum of squared distances from their center points to the fitted center line, and use this as the collinearity constraint loss L. line The specific calculation formula is as follows: L line = Σ j || (p j - c line ) - ((p j - c line ) v line ) v line || 2 , Where p j For the center of the Gaussian unit, c line Let v be the centroid of the point cloud of the cable. line This is the direction vector of the cable; S55. For all Gaussian elements belonging to the same main tower, calculate their regular prism constraint loss. In this embodiment, the regular prism is specifically a cuboid, therefore, calculate its cuboid constraint loss L. cuboid This loss function encourages Gaussian elements to be distributed near the surface of a cuboid aligned with the principal axis. The specific calculation formula is as follows: calculate the sum of the squared distances from the center point of each Gaussian element to its nearest 1-2 side planes, using the formula: L cuboid = Σ k [ min{ d(p k Plane1) 2 , d(p k Plane2) 2 , ..., d(p k Plane6) 2} ], Where k iterates through all Gaussian elements belonging to the main tower; p k It is the center coordinate of the k-th Gaussian element; d(p k Plane m () is point p k The directed distance to the m-th side plane is calculated using the following formula: d(p k Plane m ) = n m · (p k - c m ), Where n m c is the unit normal vector of the m-th side plane; mIt is a given point on the m-th lateral plane; the min{...} operator means taking the minimum of the six squared distance values ​​within the parentheses.

[0028] S56. Using the Adam optimizer, with an initial learning rate of 0.001, minimize the total loss function L. total Iteratively update the position, covariance, color, and opacity parameters of all Gaussian elements; S57. During the optimization process, adaptive density control is performed periodically: once every 100 iterations, Gaussian elements with large positional gradients and large scales are cloned to fill in details, while Gaussian elements with opacity below a set threshold are removed to eliminate redundancy (Gaussian ellipsoid representation of sparse point cloud of cable-stayed bridge as follows). Figure 4 (as shown) S6, High-quality Gaussian splash model output and rendering Output an optimized Gaussian splash model, which includes the position, covariance, color, and opacity parameters of each Gaussian element; use a graphics renderer to perform real-time, multi-view immersive rendering and display of the model, specifically including: S61: Save the optimized Gaussian splash model. The model data includes the 3D position of each Gaussian element, a 3×3 covariance matrix, RGB color values, opacity alpha values, and optional semantic label information. S62: Load the model data using a tile-based rasterization renderer; S63: For a given rendering viewpoint, sort the depths according to the distance of the Gaussian units from the viewpoint; S64: Project each Gaussian element sequentially onto the two-dimensional image plane, perform Alpha blending calculations, and finally output a high-fidelity rendered image of the cable-stayed bridge (e.g., ...). Figure 5 (As shown).

[0029] The above specific embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Those skilled in the art should understand that the above embodiments do not limit the present invention in any way, and all similar technical solutions obtained by equivalent substitution or equivalent transformation are within the protection scope of the present invention.

Claims

1. A structural semantic-guided Gaussian splash modeling and rendering method for bridge point clouds, characterized in that, Includes the following steps: S1. Bridge Scene Data Acquisition and Sparse 3D Reconstruction: Multi-view sequence images of the bridge are acquired by cameras mounted on drones or ground equipment; the sequence images are processed using structure-reconstruction-motion technique to obtain sparse point clouds of the bridge and camera poses of each image; S2. Dense Point Cloud Generation and Structural Semantic Segmentation: Based on the sparse point cloud obtained from S1 and the camera pose, a dense point cloud of the bridge is generated through a multi-view stereo vision algorithm; the dense point cloud is segmented using a pre-trained bridge point cloud semantic segmentation model, and key structural components such as piers, bridge deck, and cables are identified and labeled to obtain corresponding structural semantic labels. S3. Mapping of semantic labels to sparse point clouds: The structural semantic labels obtained from the dense point cloud in S2 are associated with the sparse point cloud obtained in S1 through spatial mapping relationship, thereby obtaining a sparse point cloud with semantic labels. S4. Differentiated Gaussian initialization guided by structural semantics: Based on the sparse point cloud with semantic labels obtained by S3, the points in the sparse point cloud are differentiated, and the Gaussian elements of points of different structure categories are initialized to the shape that conforms to the geometric characteristics of the category. S5. Joint optimization of Gaussian splash model incorporating geometric constraints: Based on the Gaussian units initialized by S4, differentiable rendering is performed using the camera pose obtained by S1, and the difference between the rendered image and the original image is minimized through an optimization algorithm. During the optimization process, corresponding geometric constraint loss functions are introduced for Gaussian units with different semantic labels to optimize them together. S6, High-quality Gaussian splash model output and rendering: Output an optimized Gaussian splash model, which includes the position, covariance, color, and opacity parameters of each Gaussian element; use a graphics renderer to perform real-time, multi-view immersive rendering of the model.

2. The structural semantic-guided Gaussian splash modeling and rendering method for bridge point clouds according to claim 1, characterized in that, S1 specifically includes: S11. Plan the flight path of the UAV to collect images of the bridge deck, bottom, sides and piers from multiple angles with high overlap, and construct a sequence of images covering the complete structure of the bridge. S12. Use the scale-invariant feature transform algorithm to extract feature points from the image sequence; S13. Use a feature matching algorithm to match feature points of adjacent images to obtain matching point pairs; S14. The incremental motion recovery structure method is adopted. The initial image pair is selected to initialize the 3D model, and the PnP algorithm is used to calculate the camera pose of the new image one by one. S15. Calculate the three-dimensional coordinates of the matching feature points using triangulation, and simultaneously optimize the camera pose and three-dimensional point coordinates using bundle adjustment to obtain the optimized sparse point cloud and accurate camera pose.

3. A structural semantic-guided Gaussian splash modeling and rendering method for bridge point clouds according to claim 1 or 2, characterized in that, S2 specifically includes: S21. Based on the sparse point cloud obtained from S1 and the camera pose, a dense point cloud on the bridge surface is generated by stereo matching and depth map fusion in a multi-view stereo vision algorithm. S22. Construct a point cloud dataset containing semantic labels for each bridge component, including piers, deck, and cables; S23. Use the RandLA-Net point cloud deep learning network to train on the point cloud dataset obtained in S22 to obtain a pre-trained bridge point cloud semantic segmentation model. S24. Input the dense point cloud generated in S21 into the trained semantic segmentation model and output the semantic label of the structural component corresponding to each 3D point.

4. The structural semantic-guided Gaussian splash modeling and rendering method for bridge point clouds according to claim 3, characterized in that, S3 specifically includes: S31. For each point in the sparse point cloud, use the spatial indexing algorithm to find its K nearest neighbors in the dense point cloud. S32. Calculate the semantic label distribution of the K nearest neighbors; S33. Assign the most frequently occurring semantic label to the sparse point to complete the mapping of semantic labels from dense point cloud to sparse point cloud.

5. The structural semantic-guided Gaussian splash modeling and rendering method for bridge point clouds according to claim 4, characterized in that, S4 specifically includes: S41. For the set of points labeled as planar structural components, principal component analysis is used to fit the plane in which the points are located to obtain the plane normal vector. S42. Adjust the covariance matrix of the Gaussian element corresponding to each point in the component according to the plane normal vector described in S41, so that it is flattened and perpendicular to the direction of the normal vector, so as to conform to the plane geometric characteristics. S43. For the set of points labeled as linear structural components, the centerline is fitted using the random sampling consensus algorithm to obtain the tangent direction of the centerline; S44. Adjust the covariance matrix of the Gaussian element corresponding to each point in the component according to the tangent direction described in S43, so that it is elongated and parallel to the tangent direction to conform to the linear geometric characteristics. S45. For the point set labeled as a regular columnar structural component, the dominant axis is obtained by using an axial fitting method based on principal component analysis. S46. Adjust the covariance matrix of the Gaussian element corresponding to each point in the component according to the dominant axis described in S45, so that its shape conforms to the geometric characteristics of the regular cylindrical component.

6. The structural semantic-guided Gaussian splash modeling and rendering method for bridge point clouds according to claim 5, characterized in that, S5 specifically includes: S51. Construct the total loss function, which is a weighted sum of photometric reconstruction loss, coplanar constraint loss, collinear constraint loss, and regular cylinder constraint loss, and its expression is: L total =λ photo L photo +λ plane L plane +λ line L line +λ reg L reg , Among them, L photo For photometric reconstruction loss, L plane With L line These represent the coplanar and collinear constraint losses, respectively, L reg λ represents the loss due to the constraint of the regular column, and λ is the weight coefficient of each loss term. S52. Using a differentiable renderer, render and generate an image based on the current Gaussian meta-parameters and camera pose, and calculate the photometric reconstruction loss L between the generated image and the original input image. photo ; S53. For all Gaussian elements belonging to the same planar structural component, calculate the sum of squared distances from their center points to the fitted reference plane, and use this as the coplanar constraint loss L. plane ; S54. For all Gaussian elements belonging to the same linear structural component, calculate the sum of squared distances from their center points to the fitted center line, and use this as the collinearity constraint loss L. line ; S55. For all Gaussian elements belonging to the same regular columnar structural component, calculate the degree to which they deviate from the geometric properties of the regular column, and use this as the constraint loss L of the regular column. reg ; S56. Using the Adam optimizer, minimize the total loss function L total Iteratively update the position, covariance, color, and opacity parameters of all Gaussian elements; S57. During the optimization process, adaptive density control is performed periodically: Gaussian elements with large positional gradients and large scales are cloned to fill in details, while Gaussian elements with opacity below a set threshold are removed to eliminate redundancy.

7. The structural semantic-guided Gaussian splash modeling and rendering method for bridge point clouds according to claim 1, characterized in that, S6 specifically includes: S61. Save the optimized Gaussian splash model. The model data includes the three-dimensional position of each Gaussian element, a 3×3 covariance matrix, RGB color values, opacity alpha values, and optional semantic label information. S62. Load the model data using a tile-based rasterization renderer; S63. For a given rendering viewpoint, sort the depths according to the distance between the Gaussian units and the viewpoint. S64. Project each Gaussian element onto the two-dimensional image plane in sequence, perform Alpha blending calculation, and finally output a high-fidelity bridge rendering image.

Images