House surveying and mapping data real-time correction method and system based on multi-source sensor fusion

Abstract

The invention discloses a house surveying and mapping data real-time correction method and system based on multi-source sensor fusion, and the method comprises the steps: synchronously collecting original data, including point cloud data, visual image frames, speed data and positioning data, through a multi-source sensing module integrated in a surveying and mapping terminal; constructing a system state equation including a position state vector, a speed state vector and a sensor deviation vector according to the original data; based on the system state equation, an improved extended Kalman filtering algorithm is adopted for state updating; and based on the updated state equation, calculating a deviation vector between the position coordinates before and after correction, and when the modulus length of the deviation vector exceeds a preset length and the duration exceeds a preset duration, generating an observation station adjustment suggested path. According to the method, the problems that the fusion positioning precision is reduced, the robustness is insufficient and the error correction process is low in efficiency due to the fact that the reliability of multi-source sensor data is not uniform due to interference in a complex urban environment can be solved.

Description

Technical Field

[0001] This invention relates to the field of building surveying technology, and in particular to a method and system for real-time correction of building surveying data based on multi-source sensor fusion. Background Technology

[0002] In traditional building surveying, especially in densely built-up urban areas, solutions relying on single or simple combinations of sensors often face numerous challenges. Existing technologies typically employ a combination of Global Navigation Satellite System (GNSS) and Inertial Measurement Unit (IMU) for positioning, supplemented by LiDAR or visual sensors to acquire spatial information. However, this approach has significant shortcomings in practical applications. First, in the "urban canyon" environment of towering buildings, satellite signals are easily severely blocked, resulting in significant multipath effects. This causes jumps or long-term drift in the global absolute positioning information provided by GNSS, drastically reducing the reliability of the location data. Second, during periods of satellite signal failure or limitation, the system over-relies on IMU integration calculations. The inherent sensor bias and noise accumulate rapidly over time, generating substantial position and attitude drift errors, failing to meet the requirements of long-term, high-precision surveying. Furthermore, feature matching algorithms for visual or LiDAR point clouds are prone to degradation or failure in areas with repetitive structures (such as glass curtain walls) or sparse features, leading to the failure of local observation models and consequently, divergence in state estimation. Finally, the entire data acquisition and correction process lacks sufficient intelligence. When the system detects potential errors, it typically lacks effective and intuitive guidance for on-site personnel. Error identification and subsequent selection of supplementary testing sites heavily rely on the experience of the operators, making the process cumbersome and prone to omissions, thus reducing overall operational efficiency and data integrity. Summary of the Invention

[0003] To address at least one of the aforementioned technical problems, this invention provides a method and system for real-time correction of building surveying data based on multi-source sensor fusion.

[0004] In a first aspect, the present invention provides a real-time correction method for building survey data based on multi-source sensor fusion, the method comprising:

[0005] Raw data is collected synchronously by a multi-source sensor module integrated into the surveying terminal. The raw data includes point cloud data, visual image frames, velocity data, and positioning data.

[0006] Construct a system state equation that includes position state vector, velocity state vector and sensor deviation vector based on the original data;

[0007] Based on the system state equation, an improved extended Kalman filter algorithm is used for state update;

[0008] Based on the updated state equation, the deviation vector between the position coordinates before and after correction is calculated. When the magnitude of the deviation vector exceeds the preset length and the duration exceeds the preset duration, a suggested path for station adjustment is generated.

[0009] The suggested adjustment path is a virtual guide line extending in the opposite direction of the deviation vector, used to prompt operators to move to the error convergence area for supplementary measurement.

[0010] Secondly, the present invention also provides a real-time correction system for building surveying data based on multi-source sensor fusion, the system comprising:

[0011] The data acquisition unit is used to synchronously acquire raw data through a multi-source sensor module integrated in the surveying terminal. The raw data includes point cloud data, visual image frames, velocity data, and positioning data.

[0012] The equation building unit is used to construct system state equations containing position state vectors, velocity state vectors, and sensor deviation vectors based on the raw data.

[0013] The state update unit is used to update the state based on the system state equation using an improved extended Kalman filter algorithm.

[0014] The adjustment suggestion unit is used to calculate the deviation vector between the position coordinates before and after correction based on the updated state equation. When the magnitude of the deviation vector exceeds the preset length and the duration exceeds the preset duration, a station adjustment suggestion path is generated.

[0015] The suggested adjustment path is a virtual guide line extending in the opposite direction of the deviation vector, used to prompt operators to move to the error convergence area for supplementary measurement.

[0016] Thirdly, the present invention also provides an electronic device including a processor and a memory, the memory being used to store computer program code, the computer program code including computer instructions, wherein when the processor executes the computer instructions, the electronic device performs the method as described in the first aspect above and any possible implementation thereof.

[0017] Fourthly, the present invention also provides a computer-readable storage medium storing a computer program, the computer program including program instructions that, when executed by a processor of an electronic device, cause the processor to perform a method as described in the first aspect above and any possible implementation thereof.

[0018] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0019] 1) By constructing a fusion framework of "visual-laser geometric constraint observation equation" and "global absolute position observation equation," this scheme creatively combines high-frequency, relative geometric feature observation with low-frequency, absolute satellite positioning observation. This achieves complementary advantages: the registration of visual and laser point clouds can effectively suppress short-term high-frequency drift of the IMU, while satellite positioning information can correct cumulative errors. Thus, the system uses absolute position as a reference when the signal is good, and maintains accuracy by relying on the geometric relationship between sensors when the signal is blocked, thereby solving the problem of sharp drop in accuracy caused by the failure of a single signal source.

[0020] 2) By introducing an "environmental occlusion confidence factor" and designing a dynamic adjustment formula for it and the satellite observation noise covariance matrix, this scheme enables the algorithm to perceive the intensity of external environmental interference in real time. When the satellite signal may deteriorate due to building obstruction, the system automatically reduces the trust weight of the GNSS data to avoid unreliable observations contaminating the filtering results, effectively suppressing the impact of multipath effects. Simultaneously, by monitoring the eigenvalues ​​of the residual covariance matrix of the visual-laser observation equation to determine "feature matching degradation," protective measures such as freezing the global update weights and shortening the IMU integration step size are taken when degradation occurs, preventing state estimation divergence in feature-scarce regions and greatly enhancing the system's stability under harsh observation conditions.

[0021] 3) By calculating the position deviation vector in real time and setting a duration threshold, this solution can reliably determine the occurrence of systematic errors. Once the determination is made, the system does not merely issue an alarm, but actively generates a "virtual guide line" extending in the opposite direction of the deviation. This intuitive "adjustment suggestion path" directly guides the operators to move to the "error convergence area" for supplementary measurement. This replaces the traditional process of relying on manual experience to find remeasurement points with data-driven intelligent judgment, making error correction operations precise and efficient, avoiding blind supplementary measurements and area omissions, and ensuring the integrity and reliability of the final surveying results from the workflow perspective.

[0022] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of the present invention or the background art, the accompanying drawings used in the embodiments of the present invention or the background art will be described below.

[0024] Figure 1 This is a flowchart illustrating a real-time correction method for building survey data based on multi-source sensor fusion, provided in an embodiment of the present invention.

[0025] Figure 2This is a schematic diagram of a real-time correction system for building survey data based on multi-source sensor fusion, provided in an embodiment of the present invention. Detailed Implementation

[0026] To enable those skilled in the art to better understand the present invention, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0027] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0028] Please see Figure 1 , Figure 1 This is a flowchart illustrating a real-time correction method for building survey data based on multi-source sensor fusion, provided as an embodiment of the present invention. Figure 1 As shown, the method includes:

[0029] S10. Simultaneously collect raw data through a multi-source sensor module integrated in the surveying terminal. The raw data includes point cloud data, visual image frames, velocity data, and positioning data.

[0030] S20. Construct a system state equation that includes position state vector, velocity state vector and sensor deviation vector based on the original data;

[0031] S30. Based on the system state equation, an improved extended Kalman filter algorithm is used for state update;

[0032] S40. Based on the updated state equation, calculate the deviation vector between the position coordinates before and after correction. When the magnitude of the deviation vector exceeds the preset length and the duration exceeds the preset duration, generate a suggested path for station adjustment.

[0033] The suggested adjustment path is a virtual guide line extending in the opposite direction of the deviation vector, used to prompt operators to move to the error convergence area for supplementary measurement.

[0034] In step S10, four types of data were collected, stemming from the inherent needs of mobile mapping and sensor fusion, with each type specifically addressing a particular problem. Point cloud data directly generates a 3D model of the building under test, providing rich 3D spatial structural information for the algorithm. This data is used for scan matching; by analyzing the overlap between two consecutive point cloud frames, the relative pose change of the terminal within a short time can be accurately calculated. This change is highly reliable but will accumulate and drift over time. Visual image frames can supplement the insufficient point cloud information in areas lacking features (such as long corridors or white walls). When the terminal passes the same location again, visual images are more efficient and robust than point clouds in scene recognition, detecting "loopbacks" and effectively correcting accumulated drift. Velocity data (IMU) can calculate the terminal's velocity, position, and attitude changes at any given time by integrating accelerometer and gyroscope data. Between two point cloud / visual frames, IMU data can be used for high-frequency state prediction, enabling the system to smoothly and continuously track rapid motion. Location data (GNSS / UWB) provides an absolute coordinate reference and drift correction anchor point. Systems that fuse point clouds, vision, and IMU (i.e., SLAM systems) experience cumulative drift over time. The absolute position provided by GNSS acts as an "anchor point," pulling the entire trajectory back to its true geographic coordinates, serving as the ultimate basis for correcting large-scale drift.

[0035] Specifically, data collection typically involves a handheld or backpack-mounted mobile device integrating a multi-source sensor module. The operator powers on the surveying terminal, performs a self-check, and all sensors initialize and enter standby mode. Point cloud data is generated by the terminal's onboard LiDAR, which continuously emits laser beams and receives signals reflected from the surfaces of walls, structures, furniture, and other objects. By measuring the laser's time of flight, the distance and angle from the sensor to countless points on the object's surface are calculated, generating a sparse or dense point cloud sequence representing the three-dimensional geometry of the building's interior in real time at an extremely high frequency (tens of thousands to millions of points per second). Each point carries three-dimensional coordinates (x, y, z) and reflection intensity information. Visual image frames are captured by one or more optical cameras on the terminal. The cameras capture high-resolution RGB color or grayscale images at a fixed frequency (e.g., 10-30 frames per second). These image sequences record the building's texture, color, semantic information (such as door and window types, furniture types), and rich two-dimensional visual feature points. Velocity data is measured via the terminal's built-in inertial measurement unit (IMU, typically containing a three-axis accelerometer and a three-axis gyroscope). The accelerometer measures the terminal's linear acceleration in three orthogonal directions in real time. The gyroscope measures the terminal's angular velocity (rotation rate) around the three axes in real time. Positioning data is acquired via the terminal's Global Navigation Satellite System (GNSS, such as GPS, BeiDou) receiver and positioning modules such as Ultra-Wideband (UWB) for seamless indoor and outdoor positioning. The receiver receives satellite or base station signals and calculates the terminal's absolute position (latitude, longitude, and altitude) and velocity in a global geographic coordinate system (such as WGS-84). When indoor GNSS signals fail, a relative positioning reference is provided by systems such as UWB. Its output frequency is relatively low (typically 1-10Hz).

[0036] All sensor sampling times are aligned using a unified hardware clock (such as GPS PPS pulse synchronization) or a sophisticated algorithm to ensure that each point cloud frame, each image, each set of IMU data, and each positioning data point has a unified timestamp accurate to milliseconds or even microseconds. The system packages the above four types of raw data collected within the same time window, along with their shared timestamps, into a complete data frame, which is then output to the subsequent S20 step for processing.

[0037] In one embodiment, the step S20, which involves constructing a system state equation based on the original data, including a position state vector, a velocity state vector, and a sensor bias vector, includes:

[0038] The corner points and straight line edge features of buildings are extracted using visual image frames, and ICP registration is performed in combination with point cloud data to construct a visual-laser geometric constraint observation equation; the visual-laser geometric constraint observation equation is used to solve high-frequency short-time attitude drift.

[0039] A global absolute position observation equation is constructed using positioning data, which is used to solve for low-frequency long-term position cumulative error.

[0040] The visual-laser geometric constraint observation equations are constructed to solve for high-frequency, short-term attitude drift. The system processes visual images in real time, quickly identifying key features representing the stable structure of a building, such as corner points and straight edges of door and window frames. These features are precisely labeled in the images. Simultaneously, the system processes synchronously acquired point cloud data and identifies corresponding geometric entities in 3D space. For example, the straight edges of window frames detected in the image are associated and bound to a cluster of ordered point cloud lines located in the same spatial position in the point cloud. Through this association, the system establishes a sparse but stable spatiotemporal reference network in the environment, composed of multiple "visual feature-laser point cloud feature pairs." Each "feature pair" describes the same physical location in the environment from two different sensor perspectives: a 2D image and a 3D point cloud. When the mapping terminal experiences a small, unexpected angular sway or translation (i.e., "attitude drift"), the pixel position of the same physical feature in the visual image and its 3D coordinate relationship in the laser point cloud will undergo a measurable, minute change relative to the expected model of the terminal sensor. The vision-laser geometric constraint observation equation is essentially an automated measurement and feedback mechanism for this process. It continuously monitors the discrepancies between the observed values ​​of these "feature pairs" and the calculated values ​​based on the terminal's predicted position. The system continuously calculates, through algorithms, how much adjustment is needed to the terminal's predicted pose to bring all "feature pairs" back to their optimal matching and alignment. This tiny amount of rotation and translation adjustment is the calculated high-frequency, short-term attitude drift. This observation equation is equivalent to installing an "electronic image stabilization" system on the terminal, which can detect and correct minute measurement errors caused by hand tremors and footsteps at extremely high frequencies, ensuring extremely high relative accuracy of the scan data both instantaneously and locally.

[0041] The global absolute position observation equation is constructed to solve for low-frequency, long-term position accumulation errors. The system receives absolute position coordinates (such as latitude, longitude, and altitude) from positioning modules such as GNSS (outdoor) or UWB (indoor). These coordinates may contain noise ranging from centimeters to meters, but they are a global reference value that does not drift with the terminal's own calculations. The system compares the current position, calculated by the terminal itself by fusing visual, laser, and IMU data, with the absolute position just reported by the positioning module. The global absolute position observation equation is the decision rule that manages this comparison and correction process. It defines how to trust and use this external absolute position information. When a positioning signal is available (e.g., receiving a GNSS signal when approaching a window), the system performs a comparison. If a persistent and stable deviation is found between the self-calculated position and the absolute position, and this deviation exceeds the range of short-term sensor noise, the system determines that accumulated errors are at play. At this point, the observation equation initiates the correction function. It does not rigidly "pull" the terminal trajectory to the absolute coordinate point (because the absolute coordinate itself may be noisy), but rather uses it as a strong constraint to guide the entire fusion algorithm to smoothly optimize and stretch the calculated trajectory over a period of time, so that it moves closer to the absolute coordinate benchmark in the long term, while maintaining smoothness and geometric consistency in the short term.

[0042] In one embodiment, step S20, in addition to constructing the system state equation, also includes introducing an environmental occlusion confidence factor. The calculation method is as follows:

[0043] ;

[0044] In the formula, This represents the number of currently visible satellites. This represents the theoretical maximum number of visible satellites. This is the position accuracy attenuation factor. The Shannon entropy value is the vertical projection of the laser point cloud, used to characterize the complexity of the surrounding building structure. , These are the normalized weighting coefficients, and .

[0045] It's important to note that the reliability of absolute positioning signals (such as GNSS) heavily depends on the external environment, not the sensor itself. Traditional methods often treat it as a fixed-precision input, which is a major reason for fusion failures in complex scenarios. The satellite component assesses sky obstruction. Fewer satellites and poorer geometry (high PDOP values) mean significant uncertainty in the position calculated from the GNSS signal itself. The point cloud entropy component assesses the complexity of the ground and built-up environment. Projecting the laser point cloud vertically and calculating Shannon entropy quantifies the density and regularity of surrounding buildings. "Urban canyons" of tall buildings or complex indoor structures not only obstruct satellite signals but also cause multipath effects (signal reflection) that severely pollute GNSS data. High entropy values ​​indicate a complex environment and poor expected GNSS signal quality.

[0046] Therefore, in this embodiment, an environmental occlusion confidence factor is introduced, upgrading the system from "fixed-weight fusion" to "environmentally-aware adaptive fusion." In open areas... High, the system prioritizes using GNSS to strongly correct accumulated errors; indoors or between buildings The system automatically mitigates GNSS interference, relying on visual-laser inertial odometry to maintain trajectory accuracy, and then corrects itself upon returning to an open area. This effectively prevents catastrophic drift caused by incorrect "anchor points" pulling the originally correct trajectory off course when GNSS signals are severely distorted.

[0047] In calculation At that time, the number of currently visible satellites is first obtained from the GNSS receiver in real time. and position accuracy attenuation factor A frame of point cloud data is acquired in real time from the lidar, and all its points are projected onto a horizontal plane to form a two-dimensional point density distribution map. This is the theoretical maximum number of visible satellites in the local area; this ratio reflects the degree of signal obstruction. For positioning geometric accuracy factor, The larger the value, the smaller the reciprocal, indicating worse geometric conditions. To calculate the environmental complexity entropy, the two-dimensional projection surface needs to be divided into a grid. The number of points in each grid cell is counted, and their proportion in the entire point cloud is calculated as a probability distribution. Then, the information entropy of this distribution is calculated using the Shannon entropy formula. A higher entropy value indicates a more chaotic and uneven point cloud distribution (such as complex facades, balconies, and trees), suggesting a more severe multipath effect; a lower entropy value indicates a simpler or more uniform distribution (such as open spaces and flat walls). The three indicators (all normalized to similar numerical ranges) are multiplied by preset weighting coefficients. These weights are determined during system calibration and reflect the degree of importance given to different indicators (e.g., PDOP and entropy may be given more weight). The summation yields the original confidence score, which is usually mapped to the [0, 1] interval using a Sigmoid-like function, ultimately obtaining... Finally, the calculated The observation update module is passed to the extended Kalman filter in real time. The filter will... As a moderating factor, the final effective noise of global absolute position observations = basic observation noise / .when → 1 (Excellent environment), observation noise is minimized, the filter highly trusts the observation, and strong correction is performed. When → 0 (extremely poor environment) The observation noise is set to the maximum, and the filter almost ignores the observation to avoid being misled.

[0048] In one embodiment, step S30, which involves updating the state based on the system state equation using an improved extended Kalman filter algorithm, includes:

[0049] When building obstruction is detected, causing multipath effects on satellite signals, the diagonal elements of the measurement noise covariance matrix of the global absolute position observation equation are increased in real time according to the environmental obstruction confidence factor. The adjustment formula is as follows:

[0050] ;

[0051] In the formula, The sensitivity coefficient for multipath effects. , These are the measurement noise covariance matrices before and after adjustment.

[0052] eigenvalues ​​of the residual covariance matrix of the visual-laser geometrically constrained observation equation If the conditions are met If the feature matching is degraded, then it is determined to be feature matching degradation; where, The anisotropy threshold;

[0053] When feature matching is determined to be degraded, the update weights of the global absolute position observation equation are frozen, and the time step of IMU pre-integration is shortened.

[0054] When a mapping terminal approaches tall buildings or other structures, satellite signals are not only blocked but also delayed by reflections from the building surfaces (multipath effect). This causes the GNSS receiver to calculate completely incorrect, yet seemingly "smooth," positioning points. In standard Kalman filtering, if GPS observations are mistakenly assumed to be highly reliable (by assigning them a small fixed noise covariance), this erroneous data point acts as a strong constraint, "pulling" the meticulously calculated trajectory toward the wrong location, causing catastrophic and irreversible trajectory jumps. Therefore, it is essential to proactively reduce the reliance on GPS data when environmental factors that may lead to multipath are detected.

[0055] In the above adjustment formula, It is the confidence factor for environmental occlusion. When the environment is complex and occlusion is severe... Get smaller, and As it increases, This will also increase, thus amplifying the GPS observation noise covariance. In the filter, greater observation noise means that the weight of the observation is reduced. The system will assume that "this GPS data may be very inaccurate at present," and thus rely more on the relative extrapolation results of vision-laser-IMU, becoming immune to the interference of multipath errors. The coefficient is used to control the sensitivity to this effect.

[0056] To address feature matching degradation, GPS weights are frozen and IMU step size is shortened. In feature-scarce scenarios (such as long corridors, white walls, and empty garages), visual images and laser point clouds cannot extract enough or sufficiently unique features. This leads to ill-conditioned solutions to the "visual-laser geometric constraint observation equation," where the constraints provided are very weak in one direction (e.g., movement along the corridor direction cannot be determined). The eigenvalues ​​of the covariance matrix of the observation residuals characterize the strength of the constraints in different spatial directions. If the ratio of the maximum to the minimum value exceeds a threshold, it indicates that the constraint is strong in one direction but extremely weak in another, i.e., anisotropic degradation has occurred. In this case, the solution results based on this observation equation are highly unreliable in the weak direction. Therefore, a two-pronged approach is adopted:

[0057] Freezing the update weights of the global absolute position observation equation: When features degenerate and relative constraints weaken, the system should ideally crave an external absolute reference. However, this is a dangerous period. If the GPS signal is also unreliable due to poor environmental conditions (such as indoors), blindly introducing it could lead to even greater errors. Therefore, the safest strategy is to temporarily maintain the current judgment on GPS weights (freezing), neither increasing nor decreasing trust, to avoid being misled by erroneous external information when the system's own "sensory malfunctions."

[0058] Shortening the IMU pre-integration time step: When vision-laser constraints fail, the IMU becomes the most reliable source of state estimation in the short term. However, integrating IMU data (especially low-cost IMUs) introduces errors that grow rapidly over time. Shortening the pre-integration step means performing zero-rate corrections or fusing with other sensors more frequently, thereby suppressing the rate of IMU error accumulation during feature degradation and gaining time and accuracy for the system to "weather" feature-depleted regions.

[0059] Through the above embodiments, the system can safely traverse "high-risk areas" of traditional SLAM and integrated navigation systems, such as urban canyons, indoor-outdoor transition zones, and feature-sparse corridors, without sudden positioning jumps or trajectory divergence. When GPS is contaminated by multipath interference, the system can automatically rely on its internal high-precision relative navigation to maintain the accuracy of local trajectories; when internal sensor constraints degrade, it can optimize IMU usage strategies to maximize short-term accuracy while waiting for the return of feature-rich regions. The entire adjustment process is smooth, continuous, and based on mathematical metrics, rather than abrupt switching, ensuring the continuity and stability of the state estimation output.

[0060] In one embodiment, S40, based on the updated state equation, the deviation vector between the position coordinates before and after correction is calculated. First, the position coordinates are obtained based on IMU integration and the system dynamics model (i.e., the prediction step in the state equation), which includes accumulated error and drift. Then, after updating by fusing multi-source observation information such as visual, laser, and GNSS data, the filter finally outputs the optimal estimated position coordinates, which the system considers the most accurate position at present. The system calculates the difference between these two positions in real time to obtain the deviation vector. The theoretical meaning of this vector is: the deviation distance and direction in three-dimensional space between the current purely inertial (or contaminated) trajectory and the optimal trajectory after multi-sensor fusion correction. It intuitively reflects the magnitude of the error that was not effectively corrected by the observed information.

[0061] Furthermore, the magnitude (Euclidean distance) and duration of the deviation vector are calculated and compared with a preset length threshold. This threshold is set according to the mapping accuracy requirements. If the magnitude exceeds the preset length threshold, a primary alarm is triggered, indicating that there may be a significant error at the current location. The system starts a timer or counter to continuously monitor the above-mentioned magnitude exceeding conditions. Only when the magnitude continuously exceeds the preset length threshold for a duration exceeding a preset duration threshold is the "retest required" decision finally triggered. This avoids false alarms caused by instantaneous interference (such as abnormal single-frame data). Once both of the above conditions are met simultaneously, the system determines that the current area has limited sensor information (such as long-term GNSS rejection, continuous feature degradation), resulting in poor fusion correction and a diverging error trend. Manual intervention (moving a station) is necessary to reacquire valid data to close the error.

[0062] In one embodiment, step S40 further includes generating a real-time deviation heatmap, specifically: dividing the work area into M×N grid cells, and calculating the average deviation vector magnitude within each grid cell over a historical time period; smoothing the average deviation of the grid cells using a Gaussian kernel function to generate a continuous color gradient field; the color gradient gradually changes from green (deviation < 5cm) to yellow (5cm ≤ deviation < 20cm) and then to red (deviation ≥ 20cm); a flashing "data unreliable" warning icon is superimposed on the red area, and the mapping data submission function for that area is locked, requiring the operator to complete a re-measurement and correction on-site before it can be unlocked.

[0063] Based on the aforementioned real-time deviation heatmap, when calculating the path, the system reverses the deviation vector at the current moment. This reverse vector indicates the direction and distance from the "current erroneous estimated position" to the "optimal estimated position that the system considers more reliable." The suggested adjustment path is defined as a virtual guide line extending from the current position along the direction of the reverse vector. This line can be displayed visually on the augmented reality screen or 2D map interface of the surveying terminal. The system provides clear audiovisual prompts to the operators (such as screen flashing, icon highlighting, and sound prompts). On the interactive interface, in addition to displaying the virtual guide line, a text prompt can also be displayed simultaneously: "Potential positioning drift detected. It is recommended to move approximately [L] meters in the reverse direction along the indicated path to the suggested area for supplementary measurement," where [L] is the magnitude of the module.

[0064] By promptly prompting for retesting before errors accumulate to unacceptable levels, the risk of overall modeling failure or the need for complete rework due to poor local data quality is fundamentally avoided. The system can sense its own performance degradation under specific environments and proactively guide itself to areas where sensors can regain effective information (i.e., the "error convergence region"), thereby ensuring the robustness of the entire surveying and mapping task and the reliability of the results in complex environments.

[0065] In one embodiment, step S40 further includes a cloud-based collaborative verification step:

[0066] The surveying terminal uploads the data segments marked "to be post-processed and optimized" in step S3 and the corresponding environmental context information to the cloud edge computing node via the 5G / 4G network;

[0067] The cloud node uses a factor graph optimization algorithm to construct global constraint factors by combining observation data from other surveying terminals in the same time and space, and performs batch adjustment calculations on the data segments.

[0068] The correction parameters obtained from cloud computing are sent to the surveying terminal. The terminal uses the correction parameters to retrospectively correct the locally cached "post-processing optimization" data and update the final building surveying results database.

[0069] By introducing multi-terminal, spatiotemporal observation data as global constraints, cumulative errors and systematic biases that cannot be eliminated by a single terminal can be effectively corrected, significantly improving the absolute accuracy of surveying and mapping results and ensuring seamless consistency of multi-terminal data under a unified benchmark. The computationally intensive global optimization is placed in the cloud, utilizing its powerful computing capabilities for fine-grained processing, while the terminals remain lightweight and real-time. For "difficult sections" where single-terminal signals are continuously obstructed or feature degraded, the cloud can use data from other terminals to strongly constrain and repair them, forming a "complementary verification" between terminals, greatly enhancing the robustness of the overall solution in complex environments and the reliability of the final results.

[0070] See Figure 2 In one embodiment, the present invention also provides a real-time correction system for building surveying data based on multi-source sensor fusion, the system comprising:

[0071] The data acquisition unit 100 is used to synchronously acquire raw data through a multi-source sensor module integrated in the surveying terminal. The raw data includes point cloud data, visual image frames, velocity data, and positioning data.

[0072] Equation building unit 200 is used to build system state equations containing position state vector, velocity state vector and sensor deviation vector based on the original data;

[0073] The state update unit 300 is used to update the state based on the system state equation and using an improved extended Kalman filter algorithm.

[0074] The adjustment suggestion unit 400 is used to calculate the deviation vector between the position coordinates before and after correction based on the updated state equation. When the magnitude of the deviation vector exceeds the preset length and the duration exceeds the preset duration, a station adjustment suggestion path is generated.

[0075] The suggested adjustment path is a virtual guide line extending in the opposite direction of the deviation vector, used to prompt operators to move to the error convergence area for supplementary measurement.

[0076] In one embodiment, the equation building unit 200 is further configured to:

[0077] The corner points and straight line edge features of buildings are extracted using visual image frames, and ICP registration is performed in combination with point cloud data to construct a visual-laser geometric constraint observation equation; the visual-laser geometric constraint observation equation is used to solve high-frequency short-time attitude drift.

[0078] A global absolute position observation equation is constructed using positioning data, which is used to solve for low-frequency long-term position cumulative error.

[0079] In one embodiment, the equation building unit 200 is further configured to:

[0080] Introducing an environmental occlusion confidence factor. The calculation method is as follows:

[0081] ;

[0082] In the formula, This represents the number of currently visible satellites. This represents the theoretical maximum number of visible satellites. This is the position accuracy attenuation factor. The Shannon entropy value is the vertical projection of the laser point cloud, used to characterize the complexity of the surrounding building structure. , These are the normalized weighting coefficients, and .

[0083] In one embodiment, the state update unit 300 is further configured to:

[0084] When building obstruction is detected, causing multipath effects on satellite signals, the diagonal elements of the measurement noise covariance matrix of the global absolute position observation equation are increased in real time according to the environmental obstruction confidence factor. The adjustment formula is as follows:

[0085] ;

[0086] In the formula, The sensitivity coefficient for multipath effects. , These are the measurement noise covariance matrices before and after adjustment.

[0087] eigenvalues ​​of the residual covariance matrix of the visual-laser geometrically constrained observation equation If the conditions are met If the feature matching is degraded, then it is determined to be feature matching degradation; where, The anisotropy threshold;

[0088] When feature matching is determined to be degraded, the update weights of the global absolute position observation equation are frozen, and the time step of IMU pre-integration is shortened.

[0089] It is understood that the system provided in this embodiment has functions or includes modules that can be used to execute the methods described in the above method embodiments. The specific implementation can be referred to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0090] The present invention also provides an electronic device including a processor and a memory, the memory being used to store computer program code, the computer program code including computer instructions, wherein when the processor executes the computer instructions, the electronic device performs a method as described in any of the above possible implementations.

[0091] The present invention also provides a computer-readable storage medium storing a computer program, the computer program including program instructions that, when executed by a processor of an electronic device, cause the processor to perform a method as described in any of the above possible implementations.

[0092] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

Claims

1. A real-time correction method for building survey data based on multi-source sensor fusion, characterized in that, The method includes: Raw data is collected synchronously by a multi-source sensor module integrated into the surveying terminal. The raw data includes point cloud data, visual image frames, velocity data, and positioning data. Construct a system state equation that includes position state vector, velocity state vector and sensor deviation vector based on the original data; Based on the system state equation, an improved extended Kalman filter algorithm is used for state update; Based on the updated state equation, the deviation vector between the position coordinates before and after correction is calculated. When the magnitude of the deviation vector exceeds the preset length and the duration exceeds the preset duration, a suggested path for station adjustment is generated. The suggested adjustment path is a virtual guide line extending in the opposite direction of the deviation vector, used to prompt operators to move to the error convergence area for supplementary measurement.

2. The real-time correction method for building surveying data based on multi-source sensor fusion according to claim 1, characterized in that, The process of constructing a system state equation based on the original data, including a position state vector, a velocity state vector, and a sensor deviation vector, includes: The corner points and straight line edge features of buildings are extracted using visual image frames, and ICP registration is performed in combination with point cloud data to construct a visual-laser geometric constraint observation equation; the visual-laser geometric constraint observation equation is used to solve high-frequency short-time attitude drift. A global absolute position observation equation is constructed using positioning data, which is used to solve for low-frequency long-term position cumulative error.

3. The real-time correction method for building surveying data based on multi-source sensor fusion according to claim 2, characterized in that, The method further includes: Introducing an environmental occlusion confidence factor. The calculation method is as follows: ； In the formula, This represents the number of currently visible satellites. This represents the theoretical maximum number of visible satellites. This is the position accuracy attenuation factor. The Shannon entropy value is the vertical projection of the laser point cloud, used to characterize the complexity of the surrounding building structure. , These are the normalized weighting coefficients, and .

4. The real-time correction method for building surveying data based on multi-source sensor fusion according to claim 3, characterized in that, The state update based on the system state equation and using an improved extended Kalman filter algorithm includes: When building obstruction is detected, causing multipath effects on satellite signals, the diagonal elements of the measurement noise covariance matrix of the global absolute position observation equation are increased in real time according to the environmental obstruction confidence factor. The adjustment formula is as follows: ； In the formula, The sensitivity coefficient for multipath effects. , These are the measurement noise covariance matrices before and after adjustment. eigenvalues ​​of the residual covariance matrix of the visual-laser geometrically constrained observation equation If the conditions are met If the feature matching is degraded, then it is determined to be feature matching degradation; where, The anisotropy threshold; When feature matching is determined to be degraded, the update weights of the global absolute position observation equation are frozen, and the time step of IMU pre-integration is shortened.

5. A real-time correction system for building survey data based on multi-source sensor fusion, characterized in that, The system includes: The data acquisition unit is used to synchronously acquire raw data through a multi-source sensor module integrated in the surveying terminal. The raw data includes point cloud data, visual image frames, velocity data, and positioning data. The equation building unit is used to construct system state equations containing position state vectors, velocity state vectors, and sensor deviation vectors based on the raw data. The state update unit is used to update the state based on the system state equation using an improved extended Kalman filter algorithm. The adjustment suggestion unit is used to calculate the deviation vector between the position coordinates before and after correction based on the updated state equation. When the magnitude of the deviation vector exceeds the preset length and the duration exceeds the preset duration, a station adjustment suggestion path is generated. The suggested adjustment path is a virtual guide line extending in the opposite direction of the deviation vector, used to prompt operators to move to the error convergence area for supplementary measurement.

6. The real-time correction system for building surveying data based on multi-source sensor fusion according to claim 5, characterized in that, The equation-building unit is also used for: The corner points and straight line edge features of buildings are extracted using visual image frames, and ICP registration is performed in combination with point cloud data to construct a visual-laser geometric constraint observation equation; the visual-laser geometric constraint observation equation is used to solve high-frequency short-time attitude drift. A global absolute position observation equation is constructed using positioning data, which is used to solve for low-frequency long-term position cumulative error.

7. The real-time correction system for building surveying data based on multi-source sensor fusion according to claim 6, characterized in that, The equation-building unit is also used for: Introducing an environmental occlusion confidence factor. The calculation method is as follows: ； In the formula, This represents the number of currently visible satellites. This represents the theoretical maximum number of visible satellites. This is the position accuracy attenuation factor. The Shannon entropy value is the vertical projection of the laser point cloud, used to characterize the complexity of the surrounding building structure. , These are the normalized weighting coefficients, and .

8. The real-time correction system for building surveying data based on multi-source sensor fusion according to claim 7, characterized in that, The state update unit is further configured to: When building obstruction is detected, causing multipath effects on satellite signals, the diagonal elements of the measurement noise covariance matrix of the global absolute position observation equation are increased in real time according to the environmental obstruction confidence factor. The adjustment formula is as follows: ； In the formula, The sensitivity coefficient for multipath effects. , These are the measurement noise covariance matrices before and after adjustment. eigenvalues ​​of the residual covariance matrix of the visual-laser geometrically constrained observation equation If the conditions are met If the feature matching is degraded, then it is determined to be feature matching degradation; where, The anisotropy threshold; When feature matching is determined to be degraded, the update weights of the global absolute position observation equation are frozen, and the time step of IMU pre-integration is shortened.

9. An electronic device, characterized in that, include: The electronic device includes a processor and a memory, the memory being used to store computer program code, the computer program code including computer instructions, wherein when the processor executes the computer instructions, the electronic device performs the real-time correction method for building survey data based on multi-source sensor fusion as described in any one of claims 1 to 4.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which includes program instructions that, when executed by a processor of an electronic device, cause the processor to perform the real-time correction method for building survey data based on multi-source sensor fusion as described in any one of claims 1 to 4.

Images