Gaussian spatter map construction method and system for urban rail scenarios

Abstract

The embodiment of the specification provides a Gaussian splatting map construction method and system for an urban rail scene, wherein the method comprises: representing a continuous time trajectory of rail vehicle operation by a non-uniform B-spline in a front end; outputting a continuous time trajectory and a globally consistent LiDAR point cloud map; accurately fusing radar point clouds of a current frame into the globally consistent LiDAR point cloud map, and removing map points with a large re-projection error in an optimization process; based on an optimized trajectory pose and the globally consistent LiDAR point cloud map, generating a three-dimensional spatial point set, initializing a Gaussian splatting map according to the three-dimensional spatial point set, separating a foreground from a background through an explicit sky Gaussian model, executing an incremental Gaussian expansion mechanism, dynamically expanding a map coverage area according to rendering feedback, performing global appearance consistency optimization and rendering error minimization on the Gaussian splatting map, and outputting a high-fidelity, photo-level three-dimensional digital twin map.

Description

Technical Field

[0001] This document relates to the field of rail transit technology, and in particular to a method and system for constructing Gaussian splash maps for urban rail scenarios. Background Technology

[0002] With the implementation of the national strategy for building a strong transportation network, the digital operation and maintenance and intelligent sensing of urban rail transit facilities have become crucial. Achieving high-precision and highly realistic reconstruction of the environment along urban rail lines is the foundation for building digital twins of urban rail transit. Traditional surveying and SLAM technologies mainly rely on point cloud or mesh representations, which are difficult to balance high geometric accuracy with photorealistic rendering effects.

[0003] In recent years, 3D Gaussian splashing technology has attracted attention due to its excellent real-time rendering capabilities, but it still has significant shortcomings when applied to urban rail transit scenarios. First, rail vehicles exhibit violent, non-uniform motion during startup, braking, and crossing switches. Traditional discrete-time trajectories or uniform B-spline trajectories struggle to accurately capture these high-frequency dynamic changes while maintaining computational efficiency. Second, the geometry of rail scenes is often monotonous, with highly repetitive track and sleeper textures. Traditional visual odometry relies on feature point triangulation, which is prone to system drift or even positioning failure in such scenarios due to mismatches or inaccurate depth estimation. Finally, urban rail transit typically features boundless, open scenes with a large proportion of sky. Existing Gaussian mapping methods often incorrectly model the sky as a floating foreground Gaussian sphere, resulting in blurred edges and distorted colors for foreground objects in the rendering.

[0004] Therefore, there is an urgent need to develop a real-time Gaussian map construction system that is specifically designed for urban rail environments, can integrate the advantages of multi-source sensors, and solve the aforementioned pain points. Summary of the Invention

[0005] The purpose of this invention is to provide a method and system for constructing Gaussian splash maps for urban rail scenarios, aiming to solve the above-mentioned problems in the prior art.

[0006] This invention provides a method for constructing Gaussian splash maps for urban rail scenarios, including: A continuous time trajectory representation of the rail vehicle's operation is constructed at the front end using non-uniform B-splines, wherein the control points of the time trajectory representation are adaptively adjusted by the IMU motion state; Multi-sensor data are uniformly constructed into a whole factor map, and nonlinear least squares optimization is performed on the trajectory pose and IMU bias in the continuous time trajectory representation. Based on the optimized trajectory pose and IMU bias, the LiDAR point cloud of the current frame is fused into the global voxel map, and the tracked visual association points are managed to output a LiDAR point cloud map that is consistent with the continuous time trajectory and the global map. Based on the globally consistent LiDAR point cloud map, prior depth is provided for visual features. The LiDAR map points in the globally consistent LiDAR point cloud map are projected onto the camera image. The LiDAR map points are associated with image pixels, and the associated pixels are tracked in the image sequence using optical flow. The frame-to-map reprojection error between the current frame pixel and the LiDAR map point is constructed and constructed as a visual constraint factor. The visual constraint factor is tightly coupled and optimized to obtain the optimized trajectory pose and sensor state. Using the optimized pose and sensor state, the radar point cloud of the current frame is accurately fused into the globally consistent LiDAR point cloud map, while removing map points with persistently large reprojection errors during the optimization process. Based on the optimized trajectory pose and globally consistent LiDAR point cloud map, a three-dimensional spatial point set is generated. A Gaussian splatter map is initialized according to the three-dimensional spatial point set. An explicit sky Gaussian model is used to separate the foreground and background. An incremental Gaussian expansion mechanism is executed to dynamically expand the map coverage area according to the rendering feedback. The global appearance consistency of the Gaussian splatter map is optimized and the rendering error is minimized, resulting in a high-fidelity, photorealistic three-dimensional digital twin map.

[0007] This invention provides a Gaussian splash map construction system for urban rail scenarios, comprising: Based on a front-end odometer module with tight coupling of multiple sensors, a continuous-time trajectory representation of the rail vehicle's operation is constructed at the front end using non-uniform B-splines. The control points of this time trajectory representation are adaptively adjusted through IMU motion state. Multi-sensor data are unified into a single factor map, and nonlinear least-squares optimization is performed on the trajectory pose and IMU bias in the continuous-time trajectory representation. Based on the optimized trajectory pose and IMU bias, the LiDAR point cloud of the current frame is fused into a global voxel map, and the tracked visual association points are managed, outputting a continuous-time trajectory and a globally consistent LiDAR point cloud map. Based on this globally consistent LiDAR point cloud map... To provide prior depth for visual features, LiDAR map points in the globally consistent LiDAR point cloud map are projected onto the camera image. The LiDAR map points are associated with image pixels, and optical flow is used to track associated pixels in the image sequence. The frame-to-map reprojection error between the current frame pixel and the LiDAR map point is constructed and used as a visual constraint factor. The visual constraint factor is tightly coupled and optimized to obtain the optimized trajectory pose and sensor state. Using the optimized pose and sensor state, the radar point cloud of the current frame is accurately fused into the globally consistent LiDAR point cloud map, while removing map points with persistently large reprojection errors during the optimization process. The real-time mapping module based on Gaussian splashing uses an optimized trajectory pose and a globally consistent LiDAR point cloud map to generate a 3D spatial point set. The Gaussian splash map is initialized based on the 3D spatial point set. An explicit sky Gaussian model is used to separate the foreground and background. An incremental Gaussian expansion mechanism is executed to dynamically expand the map coverage area based on rendering feedback. The global appearance consistency of the Gaussian splash map is optimized and rendering errors are minimized, resulting in a high-fidelity, photorealistic 3D digital twin map.

[0008] This invention also provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, it implements the steps of the above-described method for constructing Gaussian splash maps for urban rail scenarios.

[0009] This invention also provides a computer-readable storage medium storing an information transmission implementation program, which, when executed by a processor, implements the steps of the above-described method for constructing a Gaussian splash map for urban rail scenarios.

[0010] This invention, tailored to the specific characteristics of urban rail transit scenarios, utilizes multi-source sensor fusion and 3D Gaussian splashing technology to achieve high-precision pose estimation and photorealistic real-time map construction. It addresses the problem of tracking variable-acceleration motion through non-uniform continuous-time trajectory modeling, resolves visual texture loss issues through radar depth enhancement, and improves large-scene rendering quality through explicit sky modeling. Ultimately, it provides reliable 3D visualization data support for urban rail transit infrastructure monitoring. The digital twin map generated by this invention is not only geometrically accurate but also possesses photorealistic visual effects, clearly presenting the subtle features of track components, overhead contact lines, and other facilities. This provides strong technical support for intelligent operation and maintenance, facility status monitoring, and emergency command in urban rail transit. Attached Figure Description

[0011] To more clearly illustrate the technical solutions in one or more embodiments of this specification or in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 This is a flowchart of a Gaussian splash map construction method for urban rail scenarios according to an embodiment of the present invention; Figure 2 This is a schematic diagram of continuous time factor graph tight coupling optimization based on non-uniform B-splines according to an embodiment of the present invention; Figure 3 This is a schematic diagram demonstrating the effect of building a globally consistent point cloud map according to an embodiment of the present invention; Figure 4 This is a schematic diagram of depth prior vision enhancement based on radar point cloud according to an embodiment of the present invention; Figure 5 This is a schematic diagram demonstrating the depth prior visual enhancement effect based on radar point clouds according to an embodiment of the present invention; Figure 6 This is a schematic diagram of Gaussian splash map construction according to an embodiment of the present invention; Figure 7 This is a schematic diagram demonstrating the Gaussian splash map construction effect of an embodiment of the present invention. Figure 1 ; Figure 8 This is a schematic diagram demonstrating the Gaussian splash map construction effect of an embodiment of the present invention. Figure 2 ; Figure 9 This is a schematic diagram of a Gaussian splash map construction system for urban rail scenarios according to an embodiment of the present invention; Figure 10 This is a schematic diagram of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0013] To enable those skilled in the art to better understand the technical solutions in one or more embodiments of this specification, the technical solutions in one or more embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this specification, and not all of the embodiments. Based on one or more embodiments of this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of this document.

[0014] Method Implementation Examples According to embodiments of the present invention, a method for constructing Gaussian splash maps for urban rail scenarios is provided. Figure 1 This is a flowchart of a Gaussian splash map construction method for urban rail scenarios according to an embodiment of the present invention, such as... Figure 1 As shown, the Gaussian splash map construction method for urban rail scenarios according to an embodiment of the present invention specifically includes: Step S101 involves constructing a continuous time trajectory representation of the rail vehicle's operation at the front end using non-uniform B-splines, wherein the control points of the time trajectory representation are adaptively adjusted through IMU motion state; specifically, constructing a continuous time trajectory representation of the rail vehicle's operation at the front end using non-uniform B-splines includes: To address the time-varying nature of the rail vehicle's operating speed, at the beginning of each time interval, the average angular velocity norm and average linear acceleration norm of the IMU data within that time interval are calculated. Based on the amplitudes of the average angular velocity norm and average linear acceleration norm, combined with prior mapping relationships, the number of new control points is determined, thereby achieving dynamic adjustment of the control point density. The new control points are initialized with the values ​​of the previous control points and distributed at uniform time intervals within the current time period. Two independent cubic non-uniform cumulative B-spline curves are used, and the rotation of the track vehicle's trajectory is calculated according to Formulas 1 and 2. Peaceful relocation Parameterization is performed for a given time interval. The position and posture of the rail vehicle are controlled by the control point and We obtain the following by weighting using B-spline basis functions: Formula 1; Formula 2; in, It is composed of a non-uniform cumulative mixing matrix. The calculated coefficient vector and mixing matrix Depends on non-uniformly distributed node vectors This allows the non-uniform B-spline to better fit the local shape of the track vehicle trajectory, where i represents the starting node index of the time interval in which the current moment is located, and j represents the coefficient vector. The local element index of j, where j takes values ​​from 1 to 3.

[0015] Step S102 involves unifying the multi-sensor data into a single factor map, performing nonlinear least-squares optimization on the trajectory pose and IMU bias in the continuous-time trajectory representation, fusing the LiDAR point cloud of the current frame into the global voxel map based on the optimized trajectory pose and IMU bias, managing the tracked visually related points, and outputting a LiDAR point cloud map that is consistent with both the continuous-time trajectory and the global map. Specifically, unifying the multi-sensor data into a single factor map includes: According to Formula 3, the radar point cloud surface factor is... Visual reprojection factor IMU factor and prior factors Construct a unified factor graph: Formula 3; in, This represents the optimal state variable that minimizes the objective function. , The covariance matrix represents the radar point cloud surface factor. The covariance matrix representing the visual reprojection factor. Represents the covariance matrix of the IMU factors. This represents the covariance matrix of the prior factors.

[0016] The trajectory pose and IMU bias in the continuous time trajectory representation are optimized by nonlinear least squares. Based on the optimized trajectory pose and IMU bias, the LiDAR point cloud of the current frame is fused into the global voxel map using the optimized pose to obtain visual association points. The tracked visual association points are managed, and association points with large errors are removed. New visual association points are selected from the global voxel map and added to the tracking. Finally, a LiDAR point cloud map that is consistent with the continuous time trajectory and the global map is output.

[0017] Step S103: Based on the globally consistent LiDAR point cloud map, prior depth is provided for visual features. LiDAR map points in the globally consistent LiDAR point cloud map are projected onto the camera image. LiDAR map points are associated with image pixels, and optical flow is used to track associated pixels in the image sequence. The frame-to-map reprojection error between the current frame pixel and the LiDAR map point is constructed and used as a visual constraint factor. Tight-coupled optimization is performed on the visual constraint factor to obtain the optimized trajectory pose and sensor state. Using the optimized pose and sensor state, the radar point cloud of the current frame is accurately fused into the globally consistent LiDAR point cloud map, while removing map points with persistently large reprojection errors during the optimization process. Specifically, this includes: Based on the globally consistent LiDAR point cloud map, sensor motion is estimated, motion distortion is removed from the original point cloud, and the obtained current frame point cloud is fused into the globally consistent LiDAR point cloud map, wherein the globally consistent LiDAR point cloud map is the direct depth source of visual features. Whenever a new image frame is received from the camera, based on the initial camera pose value of the current image frame, the three-dimensional points located within the camera's field of view in the globally consistent LiDAR point cloud map are selected, and the selected three-dimensional map points are projected onto the previous image frame to obtain the corresponding pixel coordinates. The sparse optical flow tracking algorithm is used to track the pixel points corresponding to the pixel coordinates in the current frame image. The outliers of the pixel points are detected and removed to obtain the final retained inliers. The final retained inliers represent the correspondence between the point cloud and the pixel points. For each valid correspondence, its reprojection error is calculated and constructed as a visual constraint factor, wherein the visual constraint factor is directly related to the camera pose, known 3D map points and image measurement information; The visual constraint factors are input into the optimizer of the factor graph for tightly coupled optimization, minimizing the residuals of all visual constraint factors to obtain the optimized trajectory pose and sensor state. By utilizing the optimized trajectory pose and sensor status, the radar point cloud of the current frame is fused into the global voxel map, while removing map points with persistently large reprojection errors during the optimization process.

[0018] Step S104: Based on the optimized trajectory pose and globally consistent LiDAR point cloud map, a 3D spatial point set is generated through fusion. A Gaussian splatter map is initialized based on this 3D spatial point set. An explicit sky Gaussian model is used to separate the foreground and background. An incremental Gaussian expansion mechanism is executed to dynamically expand the map coverage area based on rendering feedback. Global appearance consistency optimization and rendering error minimization are performed on the Gaussian splatter map, outputting a high-fidelity, photorealistic 3D digital twin map. Specifically, this includes: Based on the optimized trajectory pose and globally consistent LiDAR point cloud map, a three-dimensional spatial point set is generated. A Gaussian splash map is initialized according to the three-dimensional spatial point set. When initializing the Gaussian map, a keyframe sliding window is maintained to perform online triangulation on continuously tracked feature corner points to generate additional visual point clouds to compensate for the lack of LiDAR point clouds in distant and sparse areas of the track. From the perspective of the current keyframe, use the existing Gaussian map to render an opacity map as shown in Formula 4. The pixel values ​​in the opacity map represent the degree to which the location is covered by the existing Gaussian. Formula 4; Where n represents the total number of Gaussian volumes participating in the rendering of this pixel, sorted by depth. q represents the projection weight of the p-th Gaussian volume at the current pixel position, where p represents the current calculated index of the Gaussian volume sorted by depth from near to far, and q represents the traversal index of the Gaussian volume in the occlusion layer in front of the p-th Gaussian volume. This represents the projection weight of the q-th Gaussian volume at the current pixel position; By setting a threshold Generate a binary mask for the new region This identifies areas in the current map that are not fully modeled, where M represents a binary mask and O represents an opacity map rendered from the current view using an existing Gaussian map. Select points in the fused point cloud of the current keyframe and its neighboring frames that can be projected onto the area marked by the binary mask M, and use these points to initialize a new Gaussian to create a Gaussian in the new observation area not covered by the map; A set is randomly sampled from all historical keyframes, and the parameters of the entire Gaussian splash map are jointly optimized by minimizing the re-rendering loss to ensure the consistency of global appearance and geometry. The loss function of the re-rendering loss is shown in Equation 5. Formula 5; in, Indicates the weighting coefficient. Indicates the rendered image. Represents the truth image. The D-SSIM loss represents structural similarity. This represents the total re-rendering loss function; When initializing the Gaussian map, a batch of background Gaussians is created specifically to represent the sky, so that they are naturally placed behind scene entities during rendering. At the start of Gaussian mapping, a large number of sky Gaussians are randomly initialized on a hemisphere centered on the origin of the world coordinate system. The color of the sky Gaussians is initialized to white, the opacity is set to a predetermined threshold, and a predetermined depth is given so that they are naturally placed behind all scene Gaussians during rendering. Entity modeling of the foreground Gaussians is then performed to output a high-fidelity, photorealistic 3D digital twin map.

[0019] In summary, the Gaussian splash map construction method for urban rail transit scenarios proposed in this invention addresses the challenge of modeling non-uniform motion of rail vehicles by utilizing non-uniform B-splines to construct continuous time trajectories and adaptively adjusting control points based on IMU motion states at the front end. Simultaneously, a depth-priority visual enhancement algorithm based on radar point clouds is designed, using the radar global map to provide direct depth constraints for visual features, overcoming visual degradation caused by texture repetition in urban rail transit scenes. At the back end, this invention proposes explicit sky modeling and incremental Gaussian expansion strategies, separating foreground entities from the background sky to achieve real-time, high-fidelity, photorealistic 3D reconstruction of boundless urban rail transit scenes. This invention effectively solves the problems of positioning drift and rendering artifacts in traditional SLAM under complex motion of rail vehicles and single-texture environments, providing a high-precision digital twin foundation for intelligent operation and maintenance of urban rail transit.

[0020] The technical solutions of the embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0021] Tightly coupled optimization of continuous time factor graph based on non-uniform B-splines.

[0022] A continuous-time parameterized representation of the rail vehicle's trajectory is constructed using non-uniform B-splines. Based on this, raw data from LiDAR, IMU, and cameras are used as unified constraint factors, solved within an optimization framework, thereby achieving high-precision and robust odometry and point cloud map construction. Figure 2 As shown, the method of this embodiment of the invention specifically includes the following three processing flows, specifically: 1. Non-uniform continuous time trajectory representation. To address the time-varying nature of the rail vehicle's speed, this embodiment dynamically adjusts the control point density based on the intensity of the rail vehicle's motion. At the beginning of each time interval, the average angular velocity norm and average linear acceleration norm of the IMU data within that interval are calculated. Based on the amplitude of these two velocity norms and a priori mapping relationship, the number of new control points is determined. Finally, new control points are initialized with the values ​​of the previous control point and distributed uniformly across the current time interval. The algorithm proposes using two independent cubic non-uniform cumulative B-spline curves to represent the rotation of the rail vehicle's trajectory. Peaceful relocation Parameterize it. For a given time interval... Its position is determined by the control point and It is obtained by weighting using B-spline basis functions.

[0023]

[0024] in, It is composed of a non-uniform cumulative mixing matrix. The calculated coefficient vector. Mixture matrix. Depends on non-uniformly distributed node vectors In this way, non-uniform B-splines can better fit the local shape of the trajectory of rail vehicles.

[0025] 2. Multi-sensor tight coupling optimization. This involves optimizing the radar point cloud surface factor. Visual reprojection factor IMU factor and prior factors A unified factor graph is constructed, and nonlinear least squares optimization is performed on the trajectory pose and IMU bias.

[0026]

[0027] 3. Map Management and Loop Closure. Utilizing the optimized pose, the LiDAR point cloud of the current frame is fused into a global voxel map, providing the geometric basis for LiDAR scan matching and visual enhancement algorithms. Simultaneously, the tracked visually relevant points are managed, removing points with large errors and selecting new points from the map to add to the tracking. The final output is a high-precision, continuous-time trajectory and a globally consistent point cloud map, such as... Figure 3 As shown.

[0028] Visual enhancement based on radar point cloud depth prior.

[0029] Urban rail scenes contain numerous parallel structures and repetitive textures, even in station areas rich in feature points. Visual inertial odometry typically relies on triangulation and optimization of visual features within a sliding window. However, in urban rail scenes, the triangulation process becomes unreliable; if triangulation fails or the depth value is inaccurate, the scale estimation and pose accuracy of the entire VIO will be affected.

[0030] To address the visual feature shortage and triangulation difficulties caused by a lack of texture, this invention utilizes a pre-constructed global LiDAR point cloud map to provide prior depth for visual features. By projecting LiDAR map points onto the image, the association between LiDAR map points and image pixels is achieved. Optical flow is then used to track these pixels in the image sequence, directly constructing the frame-to-map reprojection error between the current frame's pixels and map points. This approach eliminates the time-consuming multi-view visual feature triangulation process, bypasses unreliable multi-view visual geometry calculations in degraded scenes, and directly combines high-precision LiDAR geometric information with camera angle measurements. Even when visual information is scarce, strong constraints can be formed using LiDAR geometric information.

[0031] This invention utilizes depth priors from radar point clouds for visual enhancement. Specific processing is as follows: Figure 4 As shown, leveraging the advantage of radar's ability to directly and accurately measure the three-dimensional geometric information of the environment, a global, high-precision radar point cloud map is constructed. This global radar point cloud map is then used as a direct source of depth for visual features. When the camera captures a frame, it is not necessary to wait for multiple frames to triangulate the feature points; instead, it can directly query the three-dimensional spatial coordinates of existing related points from the point cloud map, thereby immediately obtaining reliable depth values.

[0032] First, sensor motion is estimated using radar-inertial measurement unit (RIM) odometry to remove motion distortion from the original point cloud. The resulting accurate current frame point cloud is then fused into the global radar point cloud map. Visual enhancement is performed whenever a new image frame is received. Based on the initial camera pose of the current image frame, 3D points within the camera's field of view in the global map are selected, and these selected 3D map points are projected onto the previous frame to obtain their corresponding pixel coordinates. Then, a sparse optical flow tracking algorithm is used to track these pixels in the current frame. Considering that optical flow tracking can produce mismatches, outlier detection and removal are necessary. The final retained interior points represent a high-quality correspondence between the point cloud and individual pixels.

[0033] For each valid association, its reprojection error is calculated and constructed as a visual constraint factor. This factor is directly associated with the camera pose, known 3D map points, and image measurement information. Finally, this visual factor is fed into a factor graph optimizer for tightly coupled optimization, minimizing the residuals of all factors to obtain the optimized trajectory and sensor state.

[0034] By utilizing the optimized pose, the radar point cloud of the current frame is more accurately fused into the global map, while removing map points with persistently large reprojection errors during the optimization process. The depth prior visual enhancement effect based on radar point clouds is demonstrated as follows: Figure 5 As shown.

[0035] Gaussian splash map construction; such as Figure 6 As shown, to address the issues of sparse radar scan lines and missing point clouds of distant objects, this invention employs visual SFM to supplement the radar point cloud during Gaussian map initialization. By maintaining a keyframe sliding window, online triangulation is performed on continuously tracked feature corner points to generate additional visual point clouds. This compensates for the LiDAR's omissions in distant and sparse areas of the track, ensuring that the Gaussian map can cover a broad urban track scene and contain rich geometric details from the outset, thus improving the mapping effect on small targets such as distant traffic lights and utility poles.

[0036] To address the Gaussian dilation and redundancy issues in borderless urban rail scenarios, this invention employs an incremental expansion mechanism based on rendering feedback. Every N frames, a keyframe is selected to expand the Gaussian map. First, from the perspective of the current keyframe, an opacity map is rendered using the existing Gaussian map. The pixel values ​​of the opacity map represent the degree to which the location is covered by the existing Gaussian.

[0037] ; Next, through the set threshold Generate a binary mask for the new region. This identifies areas in the current map that are not fully modeled. Finally, points from the fused point cloud of the current keyframe and its neighboring frames that can be projected onto the area marked by mask M are selected. These points are used to initialize a new Gaussian map, ensuring that Gaussians are created only in newly observed areas not covered by the map. Finally, a set is randomly sampled from all historical keyframes, and the parameters of the entire Gaussian map are jointly optimized by minimizing the re-rendering loss, ensuring consistency in global appearance and geometry. The loss function combines L1 loss and structural similarity D-SSIM loss. Indicates the rendered image. Represents the truth image. This represents the weighting coefficient.

[0038]

[0039] Furthermore, addressing the issue of a large proportion of sky in urban rail scenes, this invention employs a separate modeling approach. When initializing the Gaussian map, a batch of background Gaussians specifically representing the sky is created, ensuring they naturally appear behind scene entities during rendering. At the start of Gaussian map construction, a large number of sky Gaussians are randomly initialized on a hemisphere with a very large radius centered at the world coordinate system origin. The sky Gaussians are initialized to white, with a high opacity threshold and a very large depth to ensure they naturally appear behind all scene Gaussians during rendering, allowing for accurate entity modeling of the foreground Gaussians. The Gaussian splash map construction effect is demonstrated as follows: Figure 7 and Figure 8 As shown.

[0040] As can be seen from the above technical solutions, the embodiments of the present invention employ adaptive state estimation: non-uniform B-splines are used to construct continuous-time trajectories, and the control point density is dynamically adjusted according to the intensity of motion sensed by the IMU, enabling state estimation under non-uniform motion of urban rail vehicles. The embodiments of the present invention exhibit high robustness: A global point cloud map constructed using LiDAR provides direct depth priors for visual features, demonstrating high robustness even in scenarios with repetitive urban rail textures and visual odometry degradation caused by lighting variations. The embodiments of the present invention achieve photorealistic rendering: For outdoor boundless scenes, a sky Gaussian model is introduced, separating the sky background from foreground facilities in the model, improving the rendering artifacts of traditional Gaussian splashes in open scenes, and achieving photorealistic rendering. The embodiments of the present invention comply with security certifications.

[0041] In summary, this invention, based on Gaussian splashing technology and combined with a multi-source sensor tightly coupled odometer, proposes a complete urban rail scene map construction system. Through non-uniform continuous time trajectory optimization, the positioning accuracy of rail vehicles under complex conditions such as starting and braking is significantly improved; radar-assisted visual enhancement ensures tracking stability in track areas with simple textures; and explicit sky modeling and incremental management enable efficient, real-time, and high-fidelity 3D reconstruction of long urban rail lines. The digital twin map generated by this invention is not only geometrically accurate but also possesses photorealistic visual effects, clearly presenting the subtle features of track components, overhead contact lines, and other facilities, providing strong technical support for intelligent operation and maintenance, facility status monitoring, and emergency command of urban rail transit.

[0042] Device Example 1 According to embodiments of the present invention, a Gaussian splash map construction system for urban rail scenarios is provided. Figure 9 This is a schematic diagram of a Gaussian splash map construction system for urban rail scenarios according to an embodiment of the present invention, such as... Figure 9As shown, the Gaussian splash map construction system for urban rail scenarios according to an embodiment of the present invention specifically includes: The front-end odometer module 90, based on tightly coupled multi-source sensors, constructs a continuous-time trajectory representation of the rail vehicle's operation using non-uniform B-splines. The control points of this time trajectory representation are adaptively adjusted through IMU motion state. Multi-sensor data are unified into a single factor map, and nonlinear least-squares optimization is performed on the trajectory pose and IMU bias in the continuous-time trajectory representation. Based on the optimized trajectory pose and IMU bias, the LiDAR point cloud of the current frame is fused into a global voxel map, and the tracked visual association points are managed, outputting a continuous-time trajectory and a globally consistent LiDAR point cloud map. The globally consistent LiDAR point cloud map is then used as the basis for... Visual features provide prior depth. LiDAR map points in the globally consistent LiDAR point cloud map are projected onto the camera image. LiDAR map points are associated with image pixels, and optical flow is used to track associated pixels in the image sequence. The frame-to-map reprojection error between the current frame pixel and the LiDAR map point is constructed as a visual constraint factor. This visual constraint factor is then tightly coupled and optimized to obtain the optimized trajectory pose and sensor state. Using the optimized pose and sensor state, the radar point cloud of the current frame is accurately fused into the globally consistent LiDAR point cloud map, while map points with persistently large reprojection errors during the optimization process are removed. Specifically, this is used for: To address the time-varying nature of the rail vehicle's operating speed, at the beginning of each time interval, the average angular velocity norm and average linear acceleration norm of the IMU data within that time interval are calculated. Based on the amplitudes of the average angular velocity norm and average linear acceleration norm, combined with prior mapping relationships, the number of new control points is determined, thereby achieving dynamic adjustment of the control point density. The new control points are initialized with the values ​​of the previous control points and distributed at uniform time intervals within the current time period. Two independent cubic non-uniform cumulative B-spline curves are used, and the rotation of the track vehicle's trajectory is calculated according to Formulas 1 and 2. Peaceful relocation Parameterization is performed for a given time interval. The position and posture of the rail vehicle are controlled by the control point and We obtain the following by weighting using B-spline basis functions: Formula 1; Formula 2; in, It is composed of a non-uniform cumulative mixing matrix. The calculated coefficient vector and mixing matrix Depends on non-uniformly distributed node vectors This allows the non-uniform B-spline to better fit the local shape of the track vehicle's trajectory, where i represents the starting node index of the time interval in which the current moment is located, and j represents the coefficient vector. The local element index, where j takes values ​​from 1 to 3; According to Formula 3, the radar point cloud surface factor is... Visual reprojection factor IMU factor and prior factors Construct a unified factor graph: Formula 3; in, This represents the optimal state variable that minimizes the objective function. , The covariance matrix represents the radar point cloud surface factor. The covariance matrix representing the visual reprojection factor. Represents the covariance matrix of the IMU factors. Represents the covariance matrix of the prior factors; The trajectory pose and IMU bias in the continuous time trajectory representation are optimized by nonlinear least squares. Based on the optimized trajectory pose and IMU bias, the LiDAR point cloud of the current frame is fused into the global voxel map using the optimized pose to obtain visual association points. The tracked visual association points are managed, and association points with large errors are removed. New visual association points are selected from the global voxel map to be added to the tracking. Finally, the continuous time trajectory and the global consistent LiDAR point cloud map are output. Based on the globally consistent LiDAR point cloud map, sensor motion is estimated, motion distortion is removed from the original point cloud, and the obtained current frame point cloud is fused into the globally consistent LiDAR point cloud map, wherein the globally consistent LiDAR point cloud map is the direct depth source of visual features. Whenever a new image frame is received from the camera, based on the initial camera pose value of the current image frame, the three-dimensional points located within the camera's field of view in the globally consistent LiDAR point cloud map are selected, and the selected three-dimensional map points are projected onto the previous image frame to obtain the corresponding pixel coordinates. The sparse optical flow tracking algorithm is used to track the pixel points corresponding to the pixel coordinates in the current frame image. The outliers of the pixel points are detected and removed to obtain the final retained inliers. The final retained inliers represent the correspondence between the point cloud and the pixel points. For each valid correspondence, its reprojection error is calculated and constructed as a visual constraint factor, wherein the visual constraint factor is directly related to the camera pose, known 3D map points and image measurement information; The visual constraint factors are input into the optimizer of the factor graph for tightly coupled optimization, minimizing the residuals of all visual constraint factors to obtain the optimized trajectory pose and sensor state. By utilizing the optimized trajectory pose and sensor status, the radar point cloud of the current frame is fused into the global voxel map, while removing map points with persistently large reprojection errors during the optimization process. The real-time mapping module 92 based on Gaussian splashing generates a three-dimensional spatial point set based on the optimized trajectory pose and globally consistent LiDAR point cloud map. It initializes the Gaussian splash map based on the three-dimensional spatial point set, separates the foreground and background through an explicit sky Gaussian model, executes an incremental Gaussian expansion mechanism, dynamically expands the map coverage area based on rendering feedback, optimizes the global appearance consistency of the Gaussian splash map and minimizes rendering errors, and outputs a high-fidelity, photorealistic three-dimensional digital twin map.

[0043] Specifically used for: Based on the optimized trajectory pose and globally consistent LiDAR point cloud map, a three-dimensional spatial point set is generated. A Gaussian splash map is initialized according to the three-dimensional spatial point set. When initializing the Gaussian map, a keyframe sliding window is maintained to perform online triangulation on continuously tracked feature corner points to generate additional visual point clouds to compensate for the lack of LiDAR point clouds in distant and sparse areas of the track. From the perspective of the current keyframe, render an opacity map as shown in Formula 4 using the existing Gaussian map. The pixel values ​​in the opacity map represent the degree to which the location is covered by the existing Gaussian. Formula 4; Where n represents the total number of Gaussian volumes involved in the rendering of this pixel, sorted by depth. q represents the projection weight of the p-th Gaussian volume at the current pixel position, where p represents the current calculated index of the Gaussian volume sorted by depth from near to far, and q represents the traversal index of the Gaussian volume in the occlusion layer in front of the p-th Gaussian volume. This represents the projection weight of the q-th Gaussian volume at the current pixel position; By setting a threshold Generate a binary mask for the new region This identifies areas in the current map that are not fully modeled, where M represents a binary mask and O represents an opacity map rendered from the current view using an existing Gaussian map. Select points in the fused point cloud of the current keyframe and its neighboring frames that can be projected onto the area marked by the binary mask M, and use these points to initialize a new Gaussian to create a Gaussian in the new observation area not covered by the map; A set is randomly sampled from all historical keyframes, and the parameters of the entire Gaussian splash map are jointly optimized by minimizing the re-rendering loss to ensure the consistency of global appearance and geometry. The loss function of the re-rendering loss is shown in Equation 5. Formula 5; in, Indicates the weighting coefficient. Indicates the rendered image. Represents the truth image. The D-SSIM loss represents structural similarity. This represents the total re-rendering loss function; When initializing the Gaussian map, a batch of background Gaussians is created specifically to represent the sky, so that they are naturally placed behind scene entities during rendering. At the start of Gaussian mapping, a large number of sky Gaussians are randomly initialized on a hemisphere centered on the origin of the world coordinate system. The color of the sky Gaussians is initialized to white, the opacity is set to a predetermined threshold, and a predetermined depth is given so that they are naturally placed behind all scene Gaussians during rendering. Entity modeling of the foreground Gaussians is then performed to output a high-fidelity, photorealistic 3D digital twin map.

[0044] The embodiments of the present invention are device embodiments corresponding to the above method embodiments. The specific operation of each module can be understood with reference to the description of the method embodiments, and will not be repeated here.

[0045] Device Example 2 This invention provides an electronic device, such as... Figure 10 As shown, it includes: a memory 100, a processor 102, and a computer program stored in the memory 100 and executable on the processor 102, wherein the computer program, when executed by the processor 102, performs the steps as described in the method embodiment.

[0046] Device Example 3 This invention provides a computer-readable storage medium storing an information transmission implementation program, which, when executed by a processor 102, implements the steps described in the method embodiment.

[0047] The computer-readable storage media described in this embodiment include, but are not limited to, ROM, RAM, disk, or optical disk.

[0048] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for constructing Gaussian splash maps for urban rail scenarios, characterized in that, include: A continuous time trajectory representation of the rail vehicle's operation is constructed at the front end using non-uniform B-splines, wherein the control points of the time trajectory representation are adaptively adjusted by the IMU motion state; Multi-sensor data are uniformly constructed into a whole factor map, and nonlinear least squares optimization is performed on the trajectory pose and IMU bias in the continuous time trajectory representation. Based on the optimized trajectory pose and IMU bias, the LiDAR point cloud of the current frame is fused into the global voxel map, and the tracked visual association points are managed to output a LiDAR point cloud map that is consistent with the continuous time trajectory and the global map. Based on the globally consistent LiDAR point cloud map, prior depth is provided for visual features. The LiDAR map points in the globally consistent LiDAR point cloud map are projected onto the camera image. The LiDAR map points are associated with image pixels, and the associated pixels are tracked in the image sequence using optical flow. The frame-to-map reprojection error between the current frame pixel and the LiDAR map point is constructed and constructed as a visual constraint factor. The visual constraint factor is tightly coupled and optimized to obtain the optimized trajectory pose and sensor state. Using the optimized pose and sensor state, the radar point cloud of the current frame is accurately fused into the globally consistent LiDAR point cloud map, while removing map points with persistently large reprojection errors during the optimization process. Based on the optimized trajectory pose and globally consistent LiDAR point cloud map, a three-dimensional spatial point set is generated. A Gaussian splatter map is initialized according to the three-dimensional spatial point set. An explicit sky Gaussian model is used to separate the foreground and background. An incremental Gaussian expansion mechanism is executed to dynamically expand the map coverage area according to the rendering feedback. The global appearance consistency of the Gaussian splatter map is optimized and the rendering error is minimized, resulting in a high-fidelity, photorealistic three-dimensional digital twin map.

2. The method according to claim 1, characterized in that, The construction of a continuous-time trajectory representation for the operation of the rail vehicle using non-uniform B-splines at the front end specifically includes: To address the time-varying nature of the rail vehicle's operating speed, at the beginning of each time interval, the average angular velocity norm and average linear acceleration norm of the IMU data within that time interval are calculated. Based on the amplitudes of the average angular velocity norm and average linear acceleration norm, combined with prior mapping relationships, the number of new control points is determined, thereby achieving dynamic adjustment of the control point density. The new control points are initialized with the values ​​of the previous control points and distributed at uniform time intervals within the current time period. Two independent cubic non-uniform cumulative B-spline curves are used, and the rotation of the track vehicle's trajectory is calculated according to Formulas 1 and 2. Peaceful relocation Parameterization is performed for a given time interval. The position and posture of the rail vehicle are controlled by the control point and We obtain the following by weighting using B-spline basis functions: Official 1; Official 2; in, It is composed of a non-uniform cumulative mixing matrix. The calculated coefficient vector and mixing matrix Depends on non-uniformly distributed node vectors This allows the non-uniform B-spline to better fit the local shape of the track vehicle's trajectory, where i represents the starting node index of the time interval in which the current moment is located, and j represents the coefficient vector. Local element index.

3. The method according to claim 1, characterized in that, Constructing a unified factor graph from multi-sensor data specifically includes: According to Formula 3, the radar point cloud surface factor is... Visual reprojection factor IMU factor and prior factors Construct a unified factor graph: Official 3; in, This represents the optimal state variable that minimizes the objective function. , The covariance matrix represents the radar point cloud surface factor. The covariance matrix representing the visual reprojection factor. Represents the covariance matrix of the IMU factors. This represents the covariance matrix of the prior factors.

4. The method according to claim 1, characterized in that, The trajectory pose and IMU bias in the continuous-time trajectory representation are optimized using nonlinear least squares. Based on the optimized trajectory pose and IMU bias, the LiDAR point cloud of the current frame is fused into the global voxel map. The tracked visually associated points are managed, and a LiDAR point cloud map consistent with the continuous-time trajectory and the global map is output. Specifically, this includes: The trajectory pose and IMU bias in the continuous time trajectory representation are optimized by nonlinear least squares. Based on the optimized trajectory pose and IMU bias, the LiDAR point cloud of the current frame is fused into the global voxel map using the optimized pose to obtain visual association points. The tracked visual association points are managed, and association points with large errors are removed. New visual association points are selected from the global voxel map and added to the tracking. Finally, a LiDAR point cloud map that is consistent with the continuous time trajectory and the global map is output.

5. The method according to claim 1, characterized in that, Based on the globally consistent LiDAR point cloud map, prior depth is provided for visual features. LiDAR map points in the globally consistent LiDAR point cloud map are projected onto the camera image. The LiDAR map points are associated with image pixels, and optical flow is used to track associated pixels in the image sequence. The frame-to-map reprojection error between the current frame pixel and the LiDAR map point is constructed and used as a visual constraint factor. This visual constraint factor is then tightly coupled and optimized to obtain the optimized trajectory pose and sensor state. Using the optimized pose and sensor state, the radar point cloud of the current frame is accurately fused into the globally consistent LiDAR point cloud map. Simultaneously, map points with persistently large reprojection errors during the optimization process are removed, specifically including: Based on the globally consistent LiDAR point cloud map, sensor motion is estimated, motion distortion is removed from the original point cloud, and the obtained current frame point cloud is fused into the globally consistent LiDAR point cloud map, wherein the globally consistent LiDAR point cloud map is the direct depth source of visual features. Whenever a new image frame is received from the camera, based on the initial camera pose value of the current image frame, the three-dimensional points located within the camera's field of view in the globally consistent LiDAR point cloud map are selected, and the selected three-dimensional map points are projected onto the previous image frame to obtain the corresponding pixel coordinates. The sparse optical flow tracking algorithm is used to track the pixel points corresponding to the pixel coordinates in the current frame image. The outliers of the pixel points are detected and removed to obtain the final retained inliers. The final retained inliers represent the correspondence between the point cloud and the pixel points. For each valid correspondence, its reprojection error is calculated and constructed as a visual constraint factor, wherein the visual constraint factor is directly related to the camera pose, known 3D map points and image measurement information; The visual constraint factors are input into the optimizer of the factor graph for tightly coupled optimization, minimizing the residuals of all visual constraint factors to obtain the optimized trajectory pose and sensor state. By utilizing the optimized trajectory pose and sensor status, the radar point cloud of the current frame is fused into the global voxel map, while removing map points with persistently large reprojection errors during the optimization process.

6. The method according to claim 1, characterized in that, Based on the optimized trajectory pose and globally consistent LiDAR point cloud map, a 3D spatial point set is generated. A Gaussian splatter map is initialized based on this 3D spatial point set. An explicit sky Gaussian model is used to separate the foreground and background. An incremental Gaussian expansion mechanism is executed to dynamically expand the map coverage area based on rendering feedback. Global appearance consistency optimization and rendering error minimization are performed on the Gaussian splatter map, resulting in a high-fidelity, photorealistic 3D digital twin map. Specifically, this includes: Based on the optimized trajectory pose and globally consistent LiDAR point cloud map, a three-dimensional spatial point set is generated. A Gaussian splash map is initialized according to the three-dimensional spatial point set. When initializing the Gaussian map, a keyframe sliding window is maintained to perform online triangulation on continuously tracked feature corner points to generate additional visual point clouds to compensate for the lack of LiDAR point clouds in distant and sparse areas of the track. From the perspective of the current keyframe, render an opacity map as shown in Formula 4 using the existing Gaussian map. The pixel values ​​in the opacity map represent the degree to which the location is covered by the existing Gaussian. Official 4; Where n represents the total number of Gaussian volumes participating in the rendering of this pixel, sorted by depth. q represents the projection weight of the p-th Gaussian volume at the current pixel position, where p represents the current calculated index of the Gaussian volume sorted by depth from near to far, and q represents the traversal index of the Gaussian volume in the occlusion layer in front of the p-th Gaussian volume. This represents the projection weight of the q-th Gaussian volume at the current pixel position; By setting a threshold Generate a binary mask for the new region This identifies areas in the current map that are not fully modeled, where M represents a binary mask and O represents an opacity map rendered from the current view using an existing Gaussian map. Select points in the fused point cloud of the current keyframe and its neighboring frames that can be projected onto the area marked by the binary mask M, and use these points to initialize a new Gaussian to create a Gaussian in the new observation area not covered by the map; A set is randomly sampled from all historical keyframes, and the parameters of the entire Gaussian splash map are jointly optimized by minimizing the re-rendering loss to ensure the consistency of global appearance and geometry. The loss function of the re-rendering loss is shown in Equation 5. Official 5; in, Indicates the weighting coefficient. Indicates the rendered image. Represents the truth image. The D-SSIM loss represents structural similarity. This represents the total re-rendering loss function; When initializing the Gaussian map, a batch of background Gaussians is created specifically to represent the sky, so that they are naturally placed behind scene entities during rendering. At the start of Gaussian mapping, a large number of sky Gaussians are randomly initialized on a hemisphere centered on the origin of the world coordinate system. The color of the sky Gaussians is initialized to white, the opacity is set to a predetermined threshold, and a predetermined depth is given so that they are naturally placed behind all scene Gaussians during rendering. Entity modeling of the foreground Gaussians is then performed to output a high-fidelity, photorealistic 3D digital twin map.

7. A Gaussian splash map construction system for urban rail scenarios, characterized in that, include: Based on a front-end odometer module with tight coupling of multiple sensors, a continuous-time trajectory representation of the rail vehicle's operation is constructed at the front end using non-uniform B-splines. The control points of this time trajectory representation are adaptively adjusted through IMU motion state. Multi-sensor data are unified into a single factor map, and nonlinear least-squares optimization is performed on the trajectory pose and IMU bias in the continuous-time trajectory representation. Based on the optimized trajectory pose and IMU bias, the LiDAR point cloud of the current frame is fused into a global voxel map, and the tracked visual association points are managed, outputting a continuous-time trajectory and a globally consistent LiDAR point cloud map. Based on this globally consistent LiDAR point cloud map... To provide prior depth for visual features, LiDAR map points in the globally consistent LiDAR point cloud map are projected onto the camera image. The LiDAR map points are associated with image pixels, and optical flow is used to track associated pixels in the image sequence. The frame-to-map reprojection error between the current frame pixel and the LiDAR map point is constructed and used as a visual constraint factor. The visual constraint factor is tightly coupled and optimized to obtain the optimized trajectory pose and sensor state. Using the optimized pose and sensor state, the radar point cloud of the current frame is accurately fused into the globally consistent LiDAR point cloud map, while removing map points with persistently large reprojection errors during the optimization process. The real-time mapping module based on Gaussian splashing uses an optimized trajectory pose and a globally consistent LiDAR point cloud map to generate a 3D spatial point set. The Gaussian splash map is initialized based on the 3D spatial point set. An explicit sky Gaussian model is used to separate the foreground and background. An incremental Gaussian expansion mechanism is executed to dynamically expand the map coverage area based on rendering feedback. The global appearance consistency of the Gaussian splash map is optimized and rendering errors are minimized, resulting in a high-fidelity, photorealistic 3D digital twin map.

8. The system according to claim 7, characterized in that, The front-end odometer module is specifically used for: To address the time-varying nature of the rail vehicle's operating speed, at the beginning of each time interval, the average angular velocity norm and average linear acceleration norm of the IMU data within that time interval are calculated. Based on the amplitudes of the average angular velocity norm and average linear acceleration norm, combined with prior mapping relationships, the number of new control points is determined, thereby achieving dynamic adjustment of the control point density. The new control points are initialized with the values ​​of the previous control points and distributed at uniform time intervals within the current time period. Two independent cubic non-uniform cumulative B-spline curves are used, and the rotation of the track vehicle's trajectory is calculated according to Formulas 1 and 2. Peaceful relocation Parameterization is performed for a given time interval. The position and posture of the rail vehicle are controlled by the control point and We obtain the following by weighting using B-spline basis functions: Official 1; Official 2; in, It is composed of a non-uniform cumulative mixing matrix. The calculated coefficient vector and mixing matrix Depends on non-uniformly distributed node vectors This allows the non-uniform B-spline to better fit the local shape of the track vehicle's trajectory, where i represents the starting node index of the time interval in which the current moment is located, and j represents the coefficient vector. Local element index; According to Formula 3, the radar point cloud surface factor is... Visual reprojection factor IMU factor and prior factors Construct a unified factor graph: Official 3; in, This represents the optimal state variable that minimizes the objective function. , The covariance matrix represents the radar point cloud surface factor. The covariance matrix representing the visual reprojection factor. Represents the covariance matrix of the IMU factors. This represents the covariance matrix of the prior factors. The trajectory pose and IMU bias in the continuous time trajectory representation are optimized by nonlinear least squares. Based on the optimized trajectory pose and IMU bias, the LiDAR point cloud of the current frame is fused into the global voxel map using the optimized pose to obtain visual association points. The tracked visual association points are managed, and association points with large errors are removed. New visual association points are selected from the global voxel map and added to the tracking. Finally, the continuous time trajectory and the global consistent LiDAR point cloud map are output. Based on the globally consistent LiDAR point cloud map, sensor motion is estimated, motion distortion is removed from the original point cloud, and the obtained current frame point cloud is fused into the globally consistent LiDAR point cloud map, wherein the globally consistent LiDAR point cloud map is the direct depth source of visual features. Whenever a new image frame is received from the camera, based on the initial camera pose value of the current image frame, the three-dimensional points located within the camera's field of view in the globally consistent LiDAR point cloud map are selected, and the selected three-dimensional map points are projected onto the previous image frame to obtain the corresponding pixel coordinates. The sparse optical flow tracking algorithm is used to track the pixel points corresponding to the pixel coordinates in the current frame image. The outliers of the pixel points are detected and removed to obtain the final retained inliers. The final retained inliers represent the correspondence between the point cloud and the pixel points. For each valid correspondence, its reprojection error is calculated and constructed as a visual constraint factor, wherein the visual constraint factor is directly related to the camera pose, known 3D map points and image measurement information; The visual constraint factors are input into the optimizer of the factor graph for tightly coupled optimization, minimizing the residuals of all visual constraint factors to obtain the optimized trajectory pose and sensor state. By utilizing the optimized trajectory pose and sensor status, the radar point cloud of the current frame is fused into the global voxel map, while removing map points with persistently large reprojection errors during the optimization process. The backend real-time mapping module is specifically used for: Based on the optimized trajectory pose and globally consistent LiDAR point cloud map, a three-dimensional spatial point set is generated. A Gaussian splash map is initialized according to the three-dimensional spatial point set. When initializing the Gaussian map, a keyframe sliding window is maintained to perform online triangulation on continuously tracked feature corner points to generate additional visual point clouds to compensate for the lack of LiDAR point clouds in distant and sparse areas of the track. From the perspective of the current keyframe, render an opacity map as shown in Formula 4 using the existing Gaussian map. The pixel values ​​in the opacity map represent the degree to which the location is covered by the existing Gaussian. Official 4; Where n represents the total number of Gaussian volumes involved in the rendering of this pixel, sorted by depth. q represents the projection weight of the p-th Gaussian volume at the current pixel position, where p represents the current calculated index of the Gaussian volume sorted by depth from near to far, and q represents the traversal index of the Gaussian volume in the occlusion layer in front of the p-th Gaussian volume. This represents the projection weight of the q-th Gaussian volume at the current pixel position; By setting a threshold Generate a binary mask for the new region This identifies areas in the current map that are not fully modeled, where M represents a binary mask and O represents an opacity map rendered from the current view using an existing Gaussian map. Select points in the fused point cloud of the current keyframe and its neighboring frames that can be projected onto the area marked by the binary mask M, and use these points to initialize a new Gaussian to create a Gaussian in the new observation area not covered by the map; A set is randomly sampled from all historical keyframes, and the parameters of the entire Gaussian splash map are jointly optimized by minimizing the re-rendering loss to ensure the consistency of global appearance and geometry. The loss function of the re-rendering loss is shown in Equation 5. Official 5; in, Indicates the weighting coefficient. Indicates the rendered image. Represents the truth image. The D-SSIM loss represents structural similarity. This represents the total re-rendering loss function; When initializing the Gaussian map, a batch of background Gaussians is created specifically to represent the sky, so that they are naturally placed behind scene entities during rendering. At the start of Gaussian mapping, a large number of sky Gaussians are randomly initialized on a hemisphere centered on the origin of the world coordinate system. The color of the sky Gaussians is initialized to white, the opacity is set to a predetermined threshold, and a predetermined depth is given so that they are naturally placed behind all scene Gaussians during rendering. Entity modeling of the foreground Gaussians is then performed to output a high-fidelity, photorealistic 3D digital twin map.

9. An electronic device, characterized in that, include: The memory, the processor, and the computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the Gaussian splash map construction method for urban rail scenarios as described in any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores an information transmission implementation program, which, when executed by a processor, implements the steps of the Gaussian splash map construction method for urban rail scenarios as described in any one of claims 1 to 6.

Images