Supervision method for bridge maintenance operation

Abstract

The invention discloses a bridge maintenance operation supervision method, and belongs to construction site management, and the method comprises the steps: building a sensor model, a personnel model and a bridge BIM model; registering the sensor model and the bridge BIM model by using a geographic coordinate system; integrating a personnel model, the registered sensor model and the bridge BIM model through a GIS platform to obtain a digital twin scene based on BIM + GIS; the method comprises the following steps: acquiring GPS coordinate data of an operator through an intelligent safety helmet, and converting the GPS coordinate data into coordinate data under a Cartesian coordinate system; establishing a multi-source data correction mechanism, and correcting the coordinate data under the Cartesian coordinate system; and according to the corrected coordinate data, the position of the operator is displayed in real time in the digital twinborn scene through the operator model. In order to solve the problem that the positioning precision of operators is low due to GPS signal drifting in bridge maintenance, the monitoring and positioning precision is improved.

Description

Technical Field

[0001] The present application relates to the field of construction site management, and more specifically, to a method for supervising bridge maintenance operations. Background Art

[0002] The complex structure of bridges and the ever-changing environment, especially the reinforced concrete structure, bridge towers and the open environment of cross-sea bridges, make it easy for GPS signals to experience multipath effects, signal shielding and atmospheric interference in the bridge maintenance environment, causing signal drift problems. This GPS signal drift seriously affects the positioning accuracy of operators, thereby reducing the supervision efficiency and safety level of maintenance operations.

[0003] At present, the academic and industrial circles have conducted extensive research on the application of smart helmets in engineering supervision. Zhang Tao et al. (Smart Safety Helmet Construction Site Management System [J]. Internet of Things Technology, 2014, (01): 89-91.) enhanced the supervision ability of the construction site by integrating data reading and wireless transmission modules; Yang Dengjie et al. (Design of Intelligent Safety Helmet System Based on Beidou [J]. Industrial Control Computer, 2019, (07): 15-17.) proposed a new type of intelligent safety helmet system based on Beidou satellite positioning technology, which achieved more accurate positioning and environmental monitoring; Yu Zilong et al. (Design of Intelligent Safety Helmet System Based on Beidou [J]. Internet of Things Technology, 2021, (02): 63-65.) used single-chip microcomputer and sensor technology to develop an intelligent safety helmet management system that can monitor the location and status of personnel.

[0004] Although these studies have made significant progress, the existing smart helmet technology still has key deficiencies in actual bridge maintenance applications: first, the GPS signal drift problem cannot be effectively solved, and the positioning accuracy is low in the complex environment of the bridge, especially in areas where GPS signals are easily interfered with, such as piers and bridge towers; second, the position display on the two-dimensional map is not intuitive enough and cannot accurately reflect the actual position of the personnel; third, the existing system cannot display the precise relative position of the helmet and the bridge structure; finally, it is difficult to integrate data between different systems, forming data islands.

[0005] Building Information Modeling (BIM) and Geographic Information System (GIS) technologies provide new ideas for solving the above problems. Zang Zhao et al. (Research on Intelligent Operation and Maintenance Technology of Air-Ground Integrated "Digital Twin" of Beijing-Zhangjiakou High-Speed ​​Railway Based on BIM+GIS [J]. Railway Transportation and Economy, 2022, (09): 139-145.) built a "digital twin" management system based on BIM+GIS, integrating the monitoring data of the air-ground integrated system into the "digital twin" body; Hua Lutao et al. (Research on the Application of Digital Twin System of Water Conservancy Project Based on BIM+GIS Technology [J]. Zhejiang Water Conservancy Science and Technology, 2022, (06): 14-17.) proposed the application of digital twin system of water conservancy project based on "BIM+GIS" technology; Shen Wei et al. (Construction and Application of Digital Twin System of Highway Engineering Based on BIM+GIS [J]. China Transportation Informatization, 2023, (05): 130-135.) developed a three-dimensional digital twin system of highway engineering based on BIM+GIS. These systems combine the "small but precise" BIM 3D model with the "large and wide" GIS geographic data to achieve functions such as geographic positioning and spatial analysis of buildings. However, although the existing BIM+GIS digital twin system has achieved results in facility monitoring, it still lacks effective means to solve the problem of GPS positioning accuracy for bridge maintenance workers. Summary of the invention

[0006] In view of the low positioning accuracy of workers in existing bridge maintenance due to GPS signal drift, the present application provides a supervision method for bridge maintenance operations, which improves the supervision positioning accuracy of workers through a multi-source data correction mechanism.

[0007] The present application provides a supervision method for bridge maintenance operations, including: S1, respectively establishing a sensor model, a personnel model and a bridge BIM model; the sensor model includes a smart helmet model; S2, using a geographic coordinate system to align the sensor model and the bridge BIM model; S3, integrating the personnel model, the aligned sensor model and the bridge BIM model through a GIS platform to obtain a digital twin scene based on BIM+GIS; S4, obtaining the GPS coordinate data of the operator through a smart helmet, and converting the GPS coordinate data into coordinate data in a Cartesian coordinate system; S5, establishing a multi-source data correction mechanism, correcting the coordinate data in the Cartesian coordinate system in step S4, and obtaining the corrected coordinate data; S6, based on the corrected coordinate data, displaying the operator's position in real time in the digital twin scene through the personnel model, and remotely supervising the operator through the smart helmet.

[0008] Furthermore, the smart helmet is a wearable device that integrates a GPS positioning module, a video call module and an identity authentication module; the personnel model is used to simulate the workers wearing the smart helmets in the digital twin scene.

[0009] Further, S2, using a geographic coordinate system to align the sensor model and the bridge BIM model, including: collecting geographic data of the target bridge, the geographic data including GPS data of key reference points, and the key reference points including expansion joints of the bridge; mapping and converting the coordinate system of the bridge BIM model with the GPS data of the key reference points to obtain an aligned bridge BIM model; setting the installation position of the sensor model in the aligned bridge BIM model according to the component positions and sensor layout plan of the target bridge to obtain an aligned sensor model; the components include main beams, piers, abutments, pylons and arch ribs.

[0010] Furthermore, S3 integrates the personnel model, the registered sensor model and the bridge BIM model through the GIS platform to obtain a digital twin scene based on BIM+GIS, including: spatially integrating the personnel model, the registered sensor model and the bridge BIM model through the GIS platform to obtain a multi-dimensional fusion model; according to the multi-dimensional fusion model, using the coordinate transformation method, mapping and transforming the local coordinate system of the registered bridge BIM model with the global coordinate system of the GIS platform, and establishing a mapping relationship between each component and the corresponding geographical location in the bridge BIM model; according to the mapping relationship, encoding each sensor device in the sensor model, assigning a unique identifier to each sensor device, and establishing a mapping table between the identifier and the corresponding sensor model in the multi-dimensional fusion model; using the sensors installed on site to collect field data, and using the identifier to update the field data to the corresponding sensor model in the multi-dimensional fusion model through the mapping table; according to the mapping relationship and the multi-dimensional fusion model with updated data, a digital twin scene based on BIM+GIS is established.

[0011] Furthermore, S4 obtains the GPS coordinate data of the operator through the smart helmet, and converts the GPS coordinate data into coordinate data in a Cartesian coordinate system, including: collecting the GPS coordinate data of the operator through the smart helmet; using a security authentication and authorization mechanism to verify the identity of the operator; after the operator's identity authentication is passed, transmitting the GPS coordinate data through a TLS or SSL encrypted communication protocol; converting the GPS data to a Cartesian coordinate system through a coordinate conversion formula to obtain coordinate data in the Cartesian coordinate system.

[0012] Furthermore, the coordinate transformation formula is: ,in: is the coordinate value in the Cartesian coordinate system, are the latitude and longitude coordinates in radians, a is the length of the major semi-axis of the WGS84 ellipsoid, b is the length of the minor semi-axis, and h is the elevation.

[0013] Furthermore, S5, a multi-source data correction mechanism is established to correct the coordinate data in the Cartesian coordinate system in step S4 to obtain the corrected coordinate data, including: obtaining inertial sensor data of the smart helmet, wherein the inertial sensor data includes acceleration and angular velocity; correcting the collected GPS coordinate data according to the inertial sensor data to obtain a drift error E1; the GPS signal above the cross-sea bridge is susceptible to multipath effect caused by sea surface reflection, and this scheme can effectively filter out the GPS drift caused by sea surface reflection through the inertial sensor data correction mechanism (E1), thereby improving positioning stability, especially under severe weather conditions with changeable sea surface conditions.

[0014] Obtain sensor model data updated by on-site sensors in the digital twin scene; calculate the coordinate offset E2 of the operator's position based on the acquired sensor model data and using the bridge BIM model; the structural deformation of the cross-sea bridge due to temperature difference, wind load and vehicle load is more significant than that of ordinary bridges. Through the BIM model deformation field correction mechanism (E2), the system can calculate the three-dimensional deformation field of the bridge in real time and provide accurate deformation compensation for the operator's position.

[0015] Obtain several historical positioning data of the same area from the database of the digital twin scene; calculate the regional positioning error offset E3 based on the historical positioning data; the weather conditions at sea are changeable, and typhoons, dense fog, etc. will seriously interfere with the quality of GPS signals. This solution uses the historical positioning data correction mechanism (E3) to establish an error model under specific meteorological conditions, realize intelligent compensation for positioning deviations under different weather conditions, and ensure all-weather supervision capabilities.

[0016] The coordinate data in the Cartesian coordinate system in step S4 is corrected by using the drift error E1, the coordinate offset E2 and the regional positioning error offset E3 to obtain the corrected coordinate data.

[0017] Furthermore, the collected GPS coordinate data is corrected according to the inertial sensor data to obtain the drift error E1, including: establishing a motion state equation of the smart helmet according to the acceleration and angular velocity in the inertial sensor data to obtain the displacement, velocity and acceleration estimates of the operator; establishing an observation equation for the GPS coordinate data and the inertial sensor data, the observation equation reflects the relationship between the state variable and the observation variable; calculating the position prediction value and the error covariance matrix of the operator at the current moment according to the motion state equation and the state estimate at the previous moment; according to the observation equation, the GPS coordinate data and the position prediction value are integrated to obtain a corrected position estimate; and calculating the vector difference between the corrected position estimate and the GPS coordinate data as the drift error E1.

[0018] Furthermore, according to the acquired sensor model data, the coordinate offset E2 of the operator's position is calculated using the bridge BIM model, including: taking the acquired sensor model data as input, and calculating the deformation field of the target bridge using the bridge BIM model; according to the coordinate data in the Cartesian coordinate system in step S4, using the mapping relationship between the BIM model and the GIS platform in step S3, determining the relative position of the operator in the target bridge; according to the relative position of the operator in the target bridge, and the deformation field of the target bridge, calculating the coordinate offset E2 of the operator through a spatial interpolation algorithm.

[0019] Further, according to the historical positioning data, the regional positioning error offset E3 is calculated, including: constructing a spatial interpolation function according to the historical positioning data : , where P is the position to be evaluated, For known historical positioning data, is the weight coefficient; is the Gaussian radial basis function; is the three-dimensional space coordinate in the Cartesian coordinate system; according to the coordinate data in the Cartesian coordinate system in step S4, the current position P of the operator is determined, and the spatial interpolation function is used Calculate the regional positioning error offset E3.

[0020] Compared with the prior art, the advantages of this application are:

[0021] Since the cross-sea bridge is located in an open sea environment, the multipath effect of the GPS signal is verified. On the one hand, this application integrates GPS and inertial data through the motion state equation and observation equation to compensate for the short-term error of GPS signal drift, especially in the bridge shielding area; on the other hand, this application considers the influence of the deformation factor of the bridge in the actual environment on positioning, and compensates for the position deviation caused by the deformation of the bridge structure through the spatial interpolation algorithm; finally, the Gaussian radial basis function is used to construct the spatial interpolation function, and the historical experience data is fully utilized to correct the systematic error, eliminating the long-term positioning deviation in specific areas. This application can fully correct the GPS coordinates from the time dimension (historical data), space dimension (deformation field) and dynamic dimension (inertial data), fundamentally solving the problem of low positioning accuracy caused by GPS signal drift. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 An exemplary flow chart of a method for supervising bridge maintenance operations of this application;

[0023] Figure 2 A schematic diagram of a virtual operator positioning effect of this application;

[0024] Figure 3 This is a signal transmission effect diagram of a smart helmet in this application;

[0025] Figure 4 This is a diagram of the virtual reality positioning and video transmission effect of an operator in this application. DETAILED DESCRIPTION

[0026] The present application is described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0027] like Figure 1 As shown, a sensor model, a personnel model and a bridge BIM model are established respectively; the sensor model includes a smart helmet model; the sensor model and the bridge BIM model are aligned using a geographic coordinate system; the personnel model, the aligned sensor model and the bridge BIM model are integrated through a GIS platform to obtain a digital twin scene based on BIM+GIS; the GPS coordinate data of the operator is obtained through the smart helmet, and the GPS coordinate data is converted into coordinate data in a Cartesian coordinate system; a multi-source data correction mechanism is established to correct the coordinate data in the Cartesian coordinate system in step S4 to obtain the corrected coordinate data; according to the corrected coordinate data, the position of the operator is displayed in real time in the digital twin scene through the personnel model, and the operator is remotely supervised through the smart helmet.

[0028] S1, respectively establish sensor model, personnel model and bridge BIM model; the sensor model includes a smart helmet model; the smart helmet is a wearable device that integrates GPS positioning module, video call module and identity authentication module; the personnel model is used to simulate the operator wearing the smart helmet in the digital twin scene. Among them, the geometric model of the sensor can be built using 3D modeling software (such as Revit or Rhino). Based on ergonomic standards, a standardized personnel 3D model is built. Based on the bridge CAD design drawings and measurement data, a detailed bridge model is built using professional BIM modeling software (such as Revit, Bentley Bridge, etc.).

[0029] S2, using the geographic coordinate system to align the sensor model and the bridge BIM model, including: collecting geographic data of the target bridge, the geographic data includes the GPS data of key reference points, the key reference points include the expansion joints of the bridge; among them, the expansion joints are the key structural nodes of the bridge. Specifically, a unified coding rule is established: "bridge code-direction-serial number" (such as XH-N-01 represents the first expansion joint in the north direction of the new Hongqiao Bridge), using the bridge deck to accurately measure and mark the position of each expansion joint, and associate each expansion joint with the bridge mileage pile number. Each expansion joint collects multi-point data: 3 points on each side to form a measurement array.

[0030] The coordinate system of the bridge BIM model is mapped and transformed with the GPS data of key reference points to obtain the aligned bridge BIM model. The seven-parameter or Helmert transformation method can be used to calculate the optimal transformation parameter set according to the corresponding relationship of key reference points, and the calculated transformation parameters can be used to perform overall coordinate transformation on the BIM model.

[0031] According to the component positions and sensor layout plan of the target bridge, the installation position of the sensor model is set in the aligned bridge BIM model to obtain the aligned sensor model; the components include main beams, piers, abutments, pylons and arch ribs; the sensors include but are not limited to strain gauges, accelerometers, displacement meters and environmental sensors.

[0032] Specifically, the digital twin basic model is constructed. The basic three-dimensional model is an important foundation for the digital twin platform to realize three-dimensional simulation, and provides key support for the simulation of bridge maintenance and management processes. The platform model is mainly divided into personnel model, sensor model and bridge BIM model, etc., relying on the GIS big scene to realize the integrated loading of various models and the real-time simulation of the working environment.

[0033] The BIM model is georeferenced, the project benchmark is modified to the actual geographic coordinates, and the sensor model is processed according to this benchmark, and the relative position of the sensor model to the target monitoring component in the bridge BIM is ensured to be correct, which is convenient for integration and loading in the GIS scene. The personnel model obtains its own latitude and longitude coordinates based on the GPS positioning function integrated in the helmet, and is integrated into the GIS scene through coordinate conversion processing, finally realizing the integrated display of multiple models on the platform.

[0034] The multi-dimensional model is effectively integrated through GIS. The relative spatial positions of the operators and sensors and related monitoring information can be viewed in real time and intuitively inside the model. The outside of the model is an external GIS scene corresponding to the reality, realizing the complementary combination of micro and macro. Relying on the fusion scene of the model, real-time interaction between managers and the work site environment and helmet wearers is realized, effectively improving the level of bridge maintenance management and work safety monitoring. The model is deeply integrated with the scene data, and the correlation mapping between the fusion data and the digital scene is realized to construct a multi-source fusion and multi-dimensional digital twin scene. Based on user-friendly design principles such as "ease of use and visibility", the design completes the integrated implementation of the platform.

[0035] By continuously detecting and updating sensor data, maintaining the dynamic association mapping between sensor entities and digital models, and displaying on-site monitoring data in real time and historical statistics in the form of charts, the corresponding model flashing and other alarm processing are performed for real-time monitoring data that exceeds the threshold. By obtaining the location of patrol personnel wearing smart helmets in real time, the location of the personnel model is dynamically updated in the platform, and the actual images of various components on the inspection site are transmitted and displayed in real time based on the video call function integrated in the helmet.

[0036] By integrating multi-source data with models to build an integrated platform, the switching between different specialized management platforms is effectively avoided, reflecting the design concept of "integrating functions with data rather than dividing data with functions". The full process of early warning inspection is realized on a unified platform, from sensor alarms to accurate positioning of BIM components, and then to the inspection personnel quickly rushing to the target components and transmitting real-time images on site. It effectively monitors and manages multifunctional data in real time, achieves the purpose of intelligent management of bridge inspections, and is of great significance to assisting bridge maintenance work.

[0037] S3, integrate the personnel model, the registered sensor model and the bridge BIM model through the GIS platform to obtain the digital twin scene based on BIM+GIS. Select a GIS platform suitable for bridge engineering applications, preferably a professional platform that supports 3D visualization and BIM integration, such as SuperMap GIS, ArcGIS Pro or the integrated environment of QGIS and Cesium.

[0038] Through the GIS platform, the personnel model, the registered sensor model and the bridge BIM model are spatially integrated to obtain a multi-dimensional fusion model. Specifically, a data converter from BIM model to GIS format is developed to support multiple formats such as IFC, Revit, 3dsMax, etc. The personnel model is imported into the GIS platform in GLTF or 3D Tiles format. The registered sensor model is imported to retain its spatial position and attribute information.

[0039] According to the multi-dimensional fusion model, the coordinate transformation method is used to map and transform the local coordinate system of the aligned bridge BIM model with the global coordinate system of the GIS platform. Specifically, the seven-parameter coordinate transformation method can be used to process the three-dimensional coordinate system conversion. Preferably, the TIN (triangulated irregular network) interpolation method is used to process the local deformation area.

[0040] Establish the mapping relationship between each component and the corresponding geographical location in the bridge BIM model. Specifically, design the association table structure, including fields such as component ID, BIM coordinates, and geographical coordinates, calculate the geographic envelope (Bounding Box) for each component, and establish a spatial index to improve the efficiency of geographic query. Convert BIM components into three-dimensional elements in GIS, retain the attribute information and topological relationship of the components, establish a two-way link mechanism, and support GIS positioning BIM or BIM positioning GIS.

[0041] According to the mapping relationship, a hierarchical classification coding system is adopted, such as "bridge code-component type-sensor type-serial number", and the coding length is unified to 16 bits to ensure the scalability of the system. Special cases of coding for different sensor types: Fixed sensors: fixed codes are assigned according to installation location and function; smart helmets: dynamic codes are associated with personnel IDs; temporary monitoring equipment: reserved coding segments support temporary equipment access. Binding of codes and physical devices is achieved through RFID tags or QR codes, and a coding verification mechanism is established to prevent equipment from being misused. A sensor identifier mapping table is created, and the table structure is designed to include information such as identifiers, device types, installation locations, and data formats. The unique position of each sensor is marked in the multi-dimensional fusion model, and a data channel between the sensor model and the physical sensor is established.

[0042] The field data is collected by sensors installed on site, and the field data is updated to the corresponding sensor model in the multi-dimensional fusion model through the mapping table using the identifier. According to the mapping relationship and the multi-dimensional fusion model of data update, a digital twin scene based on BIM+GIS is established.

[0043] S4, in order to ensure the security of data transmission, this platform establishes an authentication and authorization mechanism to authenticate the account and password bound to the helmet. Only the helmet terminal that has passed the authentication and authorization can establish a connection with the server and transmit data. In terms of transmission, the Transport Layer Security / Secure Sockets Layer (TLS / SSL) encryption communication protocol is used to encrypt the transmitted data to prevent eavesdropping or tampering, and establish a secure transmission channel between the smart helmet terminal and the data base server.

[0044] The coordinates obtained by helmet positioning are longitude and latitude, and the underlying digital twin platform serves graphics rendering and physical simulation, using the Cartesian coordinate system. Therefore, in the process of using positioning coordinates, the acquired longitude and latitude coordinates still need to be converted into Cartesian coordinates through the conversion formula to provide support for the visualization of the platform model. The coordinate conversion formula is:

[0045]

[0046] in: is the coordinate value in the Cartesian coordinate system, are the latitude and longitude coordinates in radians, a is the length of the major semi-axis of the WGS84 ellipsoid, a = 6378137.0 meters (major semi-axis); b = 6356752.3142 meters (minor semi-axis); h is the elevation.

[0047] In particular, on the one hand, GPS signals reflect on the sea surface to form multipath interference, causing positioning jumps; on the other hand, bridge steel structures block and reflect GPS signals. This application uses precise WGS84-Cartesian coordinate conversion and multi-source data fusion correction mechanism to achieve effective filtering and correction of GPS positioning signals in complex marine environments, and the converted high-precision three-dimensional Cartesian coordinate data is seamlessly integrated with subsequent BIM+GIS-based digital twin scenes, providing a unified spatial reference framework for bridge maintenance and supervision.

[0048] On the one hand, GPS signal reception faces unique challenges in the cross-sea bridge environment. As an ideal reflector, the sea surface produces a strong multipath effect, causing the receiver to receive both direct and reflected signals. This coordinate conversion scheme establishes a mathematical foundation that is more suitable for physical constraint analysis by converting longitude and latitude into a Cartesian coordinate system. On this basis, the system uses a trajectory continuity verification algorithm based on three-dimensional space to implement "physical feasibility filtering" to effectively identify and eliminate non-physical jump points caused by sea surface reflections.

[0049] In particular, the Z-axis information is clearly separated in the Cartesian system, allowing the system to independently analyze elevation anomalies. In view of the elevation fluctuations that are particularly prone to sea surface reflection, this system has established a constraint model based on the known elevation of the bridge deck. When it is detected that the elevation data deviates from the expected height of the bridge deck by more than a threshold, the correction mechanism is automatically triggered, significantly improving the positioning stability.

[0050] On the other hand, the large steel structure of the cross-sea bridge not only blocks the GPS signal, but also produces complex reflection and diffraction phenomena, forming "signal canyons" and "false signal areas". The coordinate transformation system of this application combines the precise geometric information of the BIM model to create a "spatial visibility map" to pre-mark possible GPS signal weak and abnormal areas.

[0051] In the Cartesian coordinate system, the system can accurately calculate the relative position of the operator and the steel structure. When it detects that a person has entered a known signal abnormality area, it automatically increases the weight of the inertial navigation system and reduces the credibility of the GPS data. In addition, by analyzing a large amount of historical positioning data, the system has established an "error characteristic model" specific to each section of the bridge, and implemented differentiated corrections for systematic deviations in different structural environments.

[0052] Finally, converting GPS coordinates into Cartesian coordinates not only solves the signal problem, but also provides fundamental technical support for the digital twin platform. The converted three-dimensional coordinates are directly compatible with the spatial expression system of mainstream 3D rendering engines and can be used for scene rendering without secondary conversion, greatly improving the real-time performance of the system.

[0053] Especially in bridge deformation monitoring applications, the linear characteristics of the Cartesian coordinate system make the calculation of small deformations more direct and accurate. The system can express the three-dimensional displacement vectors of various parts of the bridge with millimeter-level accuracy, providing a high-quality data basis for structural health monitoring. At the same time, the coordinate system is easy to connect with the finite element analysis system and supports advanced engineering analysis functions such as stress fields and vibration modes. In terms of interdisciplinary collaboration, the unified Cartesian coordinate system breaks the traditional boundaries of BIM (Building Information Modeling) and GIS (Geographic Information System), and realizes a seamless scale transition from the macroscopic geographical environment to the microscopic component details. This integration enables the location of maintenance personnel, sensor data, and the state of the bridge structure to be comprehensively analyzed in the same reference framework, significantly improving data relevance and decision support capabilities.

[0054] like Figure 2 As shown in the figure, the converted model coordinates need to be re-aligned according to the environmental model to minimize the positioning accuracy and errors generated in the coordinate conversion algorithm and other links. Ultimately, the on-site personnel can stand at the expansion joint wearing smart safety helmets, and the virtual human model on the digital twin model can also be accurately positioned and displayed at the expansion joint model corresponding to the BIM model.

[0055] The helmet management system can manage the list of helmets under the account, and can perform two-way binding with the unique ID of the helmet terminal. After successful binding, the helmet terminal can perform interactive operations such as positioning, video calls, etc. after login authentication. The helmet terminal integrates the automatic login function at startup (the terminal needs to have a SIM card inserted or connected to a Wi-Fi network to ensure that its network function is normal), and automatically sends the login request carrying the terminal's unique ID to the server. After the server successfully verifies it, it will change the corresponding terminal status under the account to which the terminal belongs to online and open interactive permissions.

[0056] The platform user logs in to the smart helmet system using the "user name and password + token" dual authentication mode. The user completes the login request by entering the username and password and submitting it to the server. The server verifies the user's identity. If the verification is successful, the login is successful and the token code for subsequent command verification is returned. If the verification fails, the login is blocked. After successful login, the user can obtain the list of helmet terminal information by sending a request (the request must carry a token code, the same below). The list contains information such as the terminal ID, terminal alias, whether the terminal is online, and terminal location. The online terminals in the list are bound to the platform personnel model. The initial coordinates of the model are the system coordinates converted from the current geographic coordinates of the terminal returned by the list. For terminals that need to bind their locations in real time, a WebSocket two-way communication channel is established between the platform and the server by sending a real-time location acquisition request carrying the corresponding ID. After success, the platform can obtain the real-time location of the terminal for binding the mobile personnel model. It is worth noting that once the personnel model is bound, the user will not be able to freely control the movement of the model, but the geographic information coordinates transmitted by the helmet terminal will be converted and bound in real time.

[0057] S5, establish a multi-source data correction mechanism, correct the coordinate data in the Cartesian coordinate system in step S4, and obtain the corrected coordinate data. Obtain the inertial sensor data of the smart helmet, the inertial sensor data includes acceleration and angular velocity; the acceleration can be collected by a MEMS accelerometer, and the angular velocity can be collected by a MEMS gyroscope.

[0058] First, the collected GPS coordinate data is corrected according to the inertial sensor data to obtain the drift error E1. The inertial sensor data can be collected using a three-axis accelerometer, a three-axis gyroscope, and a three-axis magnetometer.

[0059] In this embodiment, a 15-dimensional state vector is established based on the inertial sensor data, including position (3D), velocity (3D), attitude (3D), accelerometer bias (3D) and gyroscope bias (3D)

[0060] Implementing the continuous-time nonlinear state equation: The position differential equation: ; Velocity differential equation: , Represents the acceleration measurement value; attitude quaternion differential equation: ; Accelerometer zero partial differential equation: ,in, Represents the accelerometer zero bias vector; the gyroscope zero bias differential equation: , represents the gyroscope three-axis zero bias vector; where, is the attitude rotation matrix, q represents the quaternion of the attitude, g is the gravity vector, to To account for the system noise, the fourth-order Runge–Kutta method is used to discretize the state equation to adapt to the nonlinear system characteristics.

[0061] The observation equations include GPS observation equations and IMU observation equations. To build GPS observation equations: ,in, Represents the three-dimensional position vector in the Cartesian coordinate system; is the observation noise; the observation noise covariance matrix Dynamically adjust according to GPS signal quality:

[0062] ; .in, is the horizontal precision factor, which indicates the impact of the geometric distribution of GPS satellites on the horizontal accuracy; SNR is the signal-to-noise ratio, which indicates the GPS signal strength; α and β are weight coefficients, base_var indicates the benchmark observation variance, which represents the minimum variance value of GPS measurement under ideal conditions; IMU observation equation: Establish the acceleration observation equation: , represents the observed value of the accelerometer; Indicates the accelerometer zero bias; represents the acceleration observation noise. Represents the acceleration component measured by the accelerometer; establish the angular velocity observation equation: , are the observations of GPS position, acceleration and angular velocity respectively; Represents the angular velocity component measured by the gyroscope; Indicates the gyroscope zero bias; Denote the observation noise corresponding to GPS position, acceleration and angular velocity respectively. The zero-speed update (ZUPT) observation equation is constructed to detect the stationary state and provide additional constraints.

[0063] Predict the state at the next moment based on the motion state equation: , The predicted value of the state vector at time k+1; Represents an estimate of the current state; Represents the control input at the current moment; Time step; represents the nonlinear state equation; represents the control input vector at time k; calculate the state prediction covariance: ;in is the state transfer matrix (Jacobian matrix of the state equation), Represents the predicted value of the covariance matrix at time k+1; is the covariance matrix of the current state; is the process noise covariance; the Jacobian matrix is ​​calculated using numerical differentiation method to adapt to the nonlinear system characteristics. Represents the state transfer matrix, which is the Jacobian matrix of the nonlinear state equation f(*) at the current operating point.

[0064] Specifically, in this embodiment, the fourth-order Runge-Kutta (RK4) numerical integration method is used to derive the nonlinear state equation from the continuous-time differential equation , and for the attitude quaternion integration, quaternion multiplication operation () is used and normalization is performed after each step of integration; in order to adapt to the high sampling rate of IMU data, a variable step size integration strategy is used, and the time difference between IMU and GPS data is adaptively adjusted ; Jacobian matrix Compute numerically by central difference differentiation: ;in, represents the i-th row and j-th column element of the Jacobian matrix, which represents the effect of the j-th component change of the state vector on the i-th component of the state derivative; represents the jth standard basis vector, a 15-dimensional vector with 1 at position j and 0 at other positions; δ represents the disturbance size of numerical differentiation, which is taken in this embodiment. . It represents the value of the i-th component of the state equation after the state vector is positively perturbed by δ in the j-th direction; It represents the value of the i-th component of the state equation after the state vector is negatively perturbed by δ in the j-th direction.

[0065] Calculate the Kalman gain:

[0066] , represents the Kalman gain matrix; represents the observation matrix; represents the observation noise covariance matrix; update the state estimate: ; Update the covariance estimate:

[0067] , the Joseph form is applied to update the covariance, ensuring numerical stability and positive definiteness. represents the actual observation vector at time k+1; Represents the expected observation value calculated based on the state predicted value. Represents the identity matrix, which has the same dimensions as the state vector.

[0068] Implement adaptive noise estimation based on residual sequence: ,in is the observed residual, α is the forgetting factor (0.05-0.1); and Represent the observation noise covariance matrix estimated at time k and k-1 respectively;

[0069] Calculate the drift error E1 and extract the corrected position estimate: ; Get raw GPS measurements: ; Calculate the drift error vector: ; Calculate the error modulus: .

[0070] Then, the coordinate offset E2 caused by the deformation field of the bridge is calculated. Based on the bridge BIM model, a simplified finite element calculation model is established, and the real-time sensor data is input as boundary conditions and constraints to solve the static equilibrium equation and calculate the deformation field of the entire bridge: ,in, is the stiffness matrix, is the displacement vector, is the external force vector, and the calculation results are post-processed to obtain the three-dimensional deformation field distribution.

[0071] Based on the actual deformation data of the measurement points, the spatial deformation field interpolation model is established: ,in is the deformation at any point, is the deformation of the known measuring point, is the weight coefficient; is a radial basis function, such as a Wendland function or a cubic spline function; The impact radius is determined according to the structural characteristics and sensor distribution; the finite element results are constrained and interpolated to ensure the physical rationality of the deformation field.

[0072] Using the Cartesian coordinate data from step S4 and the mapping relationship from step S3, calculate the position of the operator in the local coordinate system of the bridge. : , where M is the transformation matrix from local coordinates to global coordinates, which determines the bridge components and relative position parameters where the workers are located; Indicates the position of the operator in the global coordinate system.

[0073] According to the relative position of the operator, query the deformation field data and extract the three-dimensional deformation vector of the corresponding position: , transform the deformation vector in the local coordinate system to the global coordinate system: , calculate the effect of deformation on position and generate coordinate offset E2.

[0074] Finally, statistical correction E3 is performed based on historical data, a spatiotemporal database is established to store historical positioning data and corresponding correction results, and a spatial index structure, such as an R-tree or a quadtree, is designed to support fast spatial queries. Based on the current position, query the spatially adjacent historical positioning data: Neighbor Set = Spatial Query (Current Position, Radius).

[0075] Construct a spatial interpolation function and define a Gaussian radial basis function: , where r is the spatial distance and σ is the smoothing parameter, which is adaptively adjusted according to the data density. Construct the interpolation system equation: ;in is the function value matrix, e is the historical error vector; the weight vector w is solved by regularized least squares method: ; where λ is the regularization parameter to prevent overfitting and singularity problems.

[0076] According to the current position P, calculate the distance to each historical data point: ; Calculate the value of each basis function: ; Synthetic error offset: ; decompose into three-dimensional components: .

[0077] The coordinate data in the Cartesian coordinate system in step S4 is corrected using the drift error E1, the coordinate offset E2 and the regional positioning error offset Eßß3 to obtain the corrected coordinate data. Specifically, the fusion correction model is defined as: ,in is the adaptive weight coefficient, satisfying . Get the Cartesian coordinate data in step S4: , calculate the comprehensive correction: ; Apply the correction to get the final coordinates: , conduct a rationality check on the correction results and eliminate the obviously abnormal correction results.

[0078] S6, based on the corrected coordinate data, displays the location of the operator in real time in the digital twin scene through the personnel model, and remotely monitors the operator through the smart helmet. The platform has functions such as video calls, danger alarms, and trajectory playback, provided that the user turns on the smart helmet and is in a good network environment.

[0079] 1) Video call. The platform can make video calls to the helmet devices that are bound to the current user account and online. By clicking on the personnel model corresponding to the helmet terminal ID that needs a video call, a call request is sent to the server. After verification, the server calls the corresponding helmet terminal camera module, voice module, etc. to respond to the request. After the helmet emits a "ding dong" prompt sound to remind the wearer, it officially establishes a video call connection with the platform. This function can be used to improve the accuracy and real-time nature of managers' grasp of the situation on the job site, greatly ensuring the accuracy and timeliness of management decisions.

[0080] 2) Danger alarm. The platform has established monitoring and alarm zones for areas with possible dangers in the inspection bridge, and conducts real-time monitoring and comparison of the positions of each helmet terminal in the platform. When the terminal is within 5 meters of the danger alarm zone, the helmet alarm light flashes to issue a danger reminder. Once entering the danger alarm zone, the buzzer in the helmet terminal will sound an alarm to drive people away. This function can effectively protect the lives of workers and improve the real-time nature of inspection safety management.

[0081] 3) Trajectory playback. The server records and stores the location and time information of the online helmet terminal in real time. The platform queries the terminal coordinate list of the specified time or batch and displays the connection to realize the personnel operation trajectory retrospective display function. This function can be used for personnel operation clocking in, inspection omissions investigation, inspection mileage statistics and other scenarios.

[0082] Since the virtual personnel model on the platform has the characteristics of height and body shape of ordinary personnel, it can intuitively reflect information such as the working space and working difficulty. This information will be transmitted in real time to the comprehensive management personnel at the platform user end for more effective remote work guidance. When the data transmitted and returned by the bridge sensor is abnormal, the management personnel can quickly locate the target sensor position at the platform display end and notify the inspection personnel to confirm and handle it. After arriving at the scene, the personnel can turn on the helmet video call function to transmit the on-site situation to the management center in real time, so that the management personnel can grasp the real situation on the scene in real time to make more accurate remote guidance decisions.

[0083] like Figure 3As shown, the present application uses the WebSocket two-way communication protocol to establish a persistent connection between the smart helmet terminal and the data base server, so as to obtain the real-time transmission of the current position coordinates of the helmet terminal and the platform's instructions. As the HTTP protocol based on Web communication, most of its request time and network transmission volume are used to establish a connection between the server and the browser, while the traditional HTTP request only completes the push and pull of the request data once after the connection is completed, which shows a lot of time and network cost waste for repeated requests. The HTML5 WebSocket protocol establishes a TCP Socket connection after the first request connection, so that each request does not need to repeatedly establish a connection between the server and the browser, which basically achieves real-time communication response and greatly increases communication efficiency.

[0084] like Figure 4 As shown in the figure, the platform accelerates the network transmission of audio and video signals between the helmet terminal and the user end through real-time transmission, and adopts packet loss recovery technologies such as "forward error correction" and "retransmission mechanism" to achieve 71% anti-packet loss, correct and compensate for the audio and video signals lost during the transmission process, and greatly ensure the quality of the call; broadband management technology is used to downgrade other request data during the audio and video transmission process to ensure the priority and stability of audio and video signal transmission, and realize full-link QoS guarantee. Audio and video calls are still stable and smooth under narrowband; full-band audio processing technology is used to process the audio signal in the frequency range to achieve intelligent echo, noise elimination and other effects, ensuring the high quality of the audio, greatly improving the clarity of the audio, and more realistically transmitting and restoring real emotions.

[0085] The platform is based on BIM+GIS and uses the personnel model as a carrier. After converting the real-time transmitted safety helmet terminal position coordinate information, it drives the coordinates of the personnel model in the platform to dynamically simulate the front-line workers' work scenes and relative positions and other work site information. Relying on the video call function integrated in the safety helmet terminal, the acquired audio and video information is displayed in the video information box above the personnel model, realizing the multi-source data fusion display of simulated digital scenes and real audio and video images, providing managers with more comprehensive and real on-site information.

[0086] The invention of the present application and its implementation methods are described schematically above. The description is not restrictive. Without departing from the spirit or basic features of the present application, the present application can be implemented in other specific forms. What is shown in the accompanying drawings is only one of the implementation methods of the invention of the present application. The actual structure is not limited to this. Any figure mark in the claims should not limit the claims involved. Therefore, if a person of ordinary skill in the art is inspired by it, without departing from the purpose of the present invention, a structural method and an embodiment similar to the technical solution are designed without creativity, which should all fall within the scope of protection of this patent. In addition, the word "including" does not exclude other elements or steps, and the word "one" before the element does not exclude the inclusion of "multiple" elements. The multiple elements stated in the product claim can also be implemented by one element through software or hardware. The words first, second, etc. are used to indicate names, and do not indicate any specific order.

Claims

1. A method for supervising bridge maintenance operations, characterized in that: include: S1, respectively establishing a sensor model, a personnel model and a bridge BIM model; the sensor model includes a smart helmet model; S2, aligning the sensor model and the bridge BIM model using the geographic coordinate system; S3, integrating the personnel model, the registered sensor model and the bridge BIM model through the GIS platform to obtain a digital twin scene based on BIM+GIS; S4, obtaining GPS coordinate data of the operator through the smart safety helmet, and converting the GPS coordinate data into coordinate data in a Cartesian coordinate system; S5, establishing a multi-source data correction mechanism to correct the coordinate data in the Cartesian coordinate system in step S4 to obtain corrected coordinate data; S6, based on the corrected coordinate data, displays the location of the operators in real time in the digital twin scene through the personnel model, and remotely monitors the operators through the smart safety helmets.

2. The method for supervising bridge maintenance work according to claim 1, characterized in that: The smart helmet is a wearable device that integrates a GPS positioning module, a video call module, and an identity authentication module; The personnel model is used to simulate workers wearing smart helmets in the digital twin scenario.

3. The method for supervising bridge maintenance work according to claim 1, characterized in that: S2, aligning the sensor model and the bridge BIM model using the geographic coordinate system, including: Collecting geographic data of the target bridge, wherein the geographic data includes GPS data of key reference points, wherein the key reference points include expansion joints of the bridge; The coordinate system of the bridge BIM model is mapped and converted with the GPS data of key reference points to obtain the aligned bridge BIM model; According to the component positions and sensor layout plan of the target bridge, the installation position of the sensor model is set in the aligned bridge BIM model to obtain the aligned sensor model; the components include main beams, piers, abutments, pylons and arch ribs.

4. The method for supervising bridge maintenance work according to claim 1, characterized in that: S3, obtain the digital twin scene based on BIM+GIS, including: Through the GIS platform, the personnel model, the registered sensor model and the bridge BIM model are spatially integrated to obtain a multi-dimensional fusion model; According to the multi-dimensional fusion model, the coordinate transformation method is used to map and transform the local coordinate system of the registered bridge BIM model with the global coordinate system of the GIS platform, and establish the mapping relationship between each component in the bridge BIM model and the corresponding geographical location; According to the mapping relationship, each sensor device in the sensor model is encoded, a unique identifier is assigned to each sensor device, and a mapping table between the identifier and the corresponding sensor model in the multi-dimensional fusion model is established; Using sensors installed on site to collect field data, and using identifiers to update the field data to the corresponding sensor model in the multi-dimensional fusion model through a mapping table; A digital twin scene based on BIM+GIS is established based on the multi-dimensional fusion model of mapping relationship and data update.

5. The method for supervising bridge maintenance work according to claim 1, characterized in that: S4, converting the GPS coordinate data into coordinate data in a Cartesian coordinate system, including: Collect GPS coordinate data of workers through smart helmets; Use security authentication and authorization mechanisms to verify the identity of operators; After the operator passes the identity authentication, the GPS coordinate data is transmitted through the TLS or SSL encrypted communication protocol; The GPS data is converted into a Cartesian coordinate system using a coordinate conversion formula to obtain coordinate data in the Cartesian coordinate system.

6. The method for supervising bridge maintenance work according to claim 5, characterized in that: The coordinate transformation formula is: ； in: is the coordinate value in the Cartesian coordinate system, are the latitude and longitude coordinates in radians, a is the length of the major semi-axis of the WGS84 ellipsoid, b is the length of the minor semi-axis, and h is the elevation.

7. The method for supervising bridge maintenance work according to claim 5, characterized in that: S5, obtaining the corrected coordinate data, including: Acquire inertial sensor data of the smart helmet, wherein the inertial sensor data includes acceleration and angular velocity; According to the inertial sensor data, the collected GPS coordinate data is corrected to obtain the drift error E1; Obtain sensor model data updated by field sensors in the digital twin scene; According to the acquired sensor model data, the coordinate offset E2 of the operator's position is calculated using the bridge BIM model; Obtain some historical positioning data of the same area from the database of the digital twin scene; According to the historical positioning data, calculate the regional positioning error offset E3; The coordinate data in the Cartesian coordinate system in step S4 is corrected by using the drift error E1, the coordinate offset E2 and the regional positioning error offset E3 to obtain the corrected coordinate data.

8. The method for supervising bridge maintenance work according to claim 7, characterized in that: The drift error E1 is obtained, including: According to the acceleration and angular velocity in the inertial sensor data, the motion state equation of the smart helmet is established to obtain the estimated values ​​of the operator's displacement, velocity and acceleration; Establish observation equations for GPS coordinate data and inertial sensor data. The observation equations reflect the relationship between state variables and observation variables. According to the motion state equation and the state estimation value at the previous moment, the predicted position value and error covariance matrix of the operator at the current moment are calculated; According to the observation equation, the GPS coordinate data and the position prediction value are integrated to obtain the corrected position estimate; The vector difference between the corrected position estimate and the GPS coordinate data is calculated as the drift error E1.

9. The method for supervising bridge maintenance work according to claim 7, characterized in that: Calculate the coordinate offset E2 of the operator's position, including: The acquired sensor model data is used as input to calculate the deformation field of the target bridge using the bridge BIM model; According to the coordinate data in the Cartesian coordinate system in step S4, the relative position of the operator in the target bridge is determined by using the mapping relationship between the BIM model and the GIS platform in step S3; According to the relative position of the operator in the target bridge and the deformation field of the target bridge, the coordinate offset E2 of the operator is calculated through the spatial interpolation algorithm.

10. The method for supervising bridge maintenance work according to claim 7, characterized in that: Calculate the regional positioning error offset E3, including: Construct a spatial interpolation function based on historical positioning data : ； Where P is the position to be evaluated, For known historical positioning data, is the weight coefficient; is the Gaussian radial basis function; is the three-dimensional space coordinate in the Cartesian coordinate system; According to the coordinate data in the Cartesian coordinate system in step S4, the current position P of the operator is determined, and the spatial interpolation function is used Calculate the regional positioning error offset E3.

Images