A supervision method for bridge maintenance operations

Abstract

The present application discloses a supervision method for bridge maintenance operations, belonging to construction site management, including: respectively establishing a sensor model, a personnel model and a bridge BIM model; registering the sensor model and the bridge BIM model by using a geographic coordinate system; integrating the personnel model, the registered sensor model and the bridge BIM model through a GIS platform to obtain a digital twin scenario based on BIM+GIS; obtaining the GPS coordinate data of the operating personnel through an intelligent safety helmet and converting the GPS coordinate data into coordinate data in a Cartesian coordinate system; establishing a multi-source data correction mechanism to correct the coordinate data in the Cartesian coordinate system; and displaying the positions of the operating personnel in real time in the digital twin scenario through the personnel model according to the corrected coordinate data. Aiming at the low positioning accuracy of operating personnel caused by GPS signal drift in bridge maintenance, the present application improves the supervision positioning accuracy.

Description

Technical Field

[0001] This application relates to the field of construction site management, and more specifically, to a supervision method for bridge maintenance operations. Background Art

[0002] The bridge structure is complex and the environment is changeable. Especially factors such as reinforced concrete structures, bridge towers, and the open environment where cross-sea bridges are located cause the GPS signal to be prone to multipath effects, signal blockage, and atmospheric interference in the bridge maintenance operation environment, resulting in signal drift problems. This GPS signal drift seriously affects the positioning accuracy of operators, and further reduces the supervision efficiency and safety guarantee level of maintenance operations.

[0003] At present, the academic and industrial circles have conducted extensive research on the application of intelligent safety helmets in project supervision. Zhang Tao et al. (Intelligent Safety Helmet Construction Site Management System [J]. Internet of Things Technologies, 2014, (01): 89-91.) enhanced the supervision ability of the construction site by integrating data reading and wireless transmission modules; Yang Dengjie et al. (Design of an Intelligent Safety Helmet System Based on Beidou [J]. Industrial Control Computer, 2019, (07): 15-17.) proposed a new type of intelligent safety helmet system based on Beidou satellite positioning technology, achieving more accurate positioning and environmental monitoring; Yu Zilong et al. (Design of an Intelligent Safety Helmet System Based on Beidou [J]. Internet of Things Technologies, 2021, (02): 63-65.) developed an intelligent safety helmet management system that can monitor the position and status of personnel by using single-chip microcomputer and sensor technologies.

[0004] Although these studies have made remarkable progress, in the actual application of bridge maintenance, the existing intelligent safety helmet technology still has key deficiencies: First, the GPS signal drift problem cannot be effectively solved, and the positioning accuracy is low in the complex bridge environment, especially in areas where GPS signals are easily interfered, such as bridge piers and bridge towers; Second, the position display on the two-dimensional map is not intuitive enough to accurately reflect the actual position of personnel; Third, the existing system cannot display the accurate relative position between the safety helmet and the bridge structure; Finally, it is difficult to integrate data between different systems, forming data islands.

[0005] Building Information Modeling (BIM) and Geographic Information System (GIS) technologies provide new ideas for solving the above problems. Zang Zhao et al. (Research on the Intelligent Operation and Maintenance Technology of the Air-Ground Integrated "Digital Twin" of the Beijing-Zhangjiakou High-Speed Railway Based on BIM+GIS [J]. Railway Transport and Economy, 2022, (09): 139-145.) constructed a "Digital Twin" management system based on BIM+GIS, integrating the monitoring data of the air-ground integrated system into the "Digital Twin" body; Hua Lutao et al. (Research on the Application of the Digital Twin System of Water Conservancy Projects Based on BIM+GIS Technology [J]. Zhejiang Hydrotechnics, 2022, (06): 14-17.) proposed the application of the digital twin system of water conservancy projects based on the "BIM+GIS" technology; Shen Wei et al. (Construction and Application of the Digital Twin System of Highway Engineering Based on BIM+GIS [J]. China ITS Journal, 2023, (05): 130-135.) developed a three-dimensional digital twin system of highway engineering based on BIM+GIS. By combining the "small and precise" BIM three-dimensional model with the "large and wide" GIS geographical data, these systems have realized functions such as the geographical positioning and spatial analysis of buildings. However, although the existing BIM+GIS digital twin systems have achieved results in facility monitoring, they still lack effective means to solve the problem of the GPS positioning accuracy of bridge maintenance workers. Summary of the Invention

[0006] In view of the low positioning accuracy of workers in existing bridge maintenance due to GPS signal drift, this application provides a supervision method for bridge maintenance operations, which improves the supervision and positioning accuracy of workers through a multi-source data correction mechanism and the like.

[0007] This application provides a supervision method for bridge maintenance operations, including: S1, respectively establishing a sensor model, a personnel model, and a bridge BIM model; the sensor model includes an intelligent safety helmet model; S2, registering the sensor model and the bridge BIM model using a geographic coordinate system; S3, integrating the personnel model, the registered sensor model, and the bridge BIM model through a GIS platform to obtain a digital twin scenario based on BIM+GIS; S4, obtaining the GPS coordinate data of the workers through the intelligent safety helmet and converting the GPS coordinate data into coordinate data in the Cartesian coordinate system; S5, establishing a multi-source data correction mechanism to correct the coordinate data in the Cartesian coordinate system in step S4 to obtain corrected coordinate data; S6, according to the corrected coordinate data, the positions of the workers are displayed in real time in the digital twin scenario through the personnel model, and the workers are remotely supervised through the intelligent safety helmet.

[0008] Furthermore, the smart safety helmet is a wearable device integrated with a GPS positioning module, a video call module, and an identity authentication module; the personnel model is used to simulate the operating personnel wearing the smart safety helmet in the digital twin scenario.

[0009] Furthermore, in S2, the sensor model and the bridge BIM model are registered using the geographic coordinate system, including: collecting the geographic data of the target bridge, where the geographic data includes the GPS data of the key reference points, and the key reference points include the expansion joints of the bridge; performing a mapping transformation between the coordinate system of the bridge BIM model and the GPS data of the key reference points to obtain the registered bridge BIM model; setting the installation positions of the sensor model in the registered bridge BIM model according to the component positions of the target bridge and the sensor layout plan to obtain the registered sensor model; the components include the main girder, piers, abutments, cable towers, and arch ribs.

[0010] Furthermore, in S3, the personnel model, the registered sensor model, and the bridge BIM model are integrated through the GIS platform to obtain a digital twin scenario based on BIM+GIS, including: spatially integrating the personnel model, the registered sensor model, and the bridge BIM model through the GIS platform to obtain a multi-dimensional fusion model; according to the multi-dimensional fusion model, using the coordinate transformation method to perform a mapping transformation between the local coordinate system of the registered bridge BIM model and the global coordinate system of the GIS platform to establish the mapping relationship between each component in the bridge BIM model and the corresponding geographical location; according to the mapping relationship, encoding each sensor device in the sensor model, assigning a unique identifier to each sensor device, and establishing a mapping table between the identifier and the corresponding sensor model in the multi-dimensional fusion model; collecting on-site data using the sensors installed on-site, and updating the on-site data to the corresponding sensor model in the multi-dimensional fusion model through the mapping table using the identifier; establishing a digital twin scenario based on BIM+GIS according to the mapping relationship and the multi-dimensional fusion model with updated data.

[0011] Furthermore, in S4, the GPS coordinate data of the operating personnel is obtained through the smart safety helmet and converted into coordinate data in the Cartesian coordinate system, including: collecting the GPS coordinate data of the operating personnel through the smart safety helmet; verifying the identity of the operating personnel using the security authentication and authorization mechanism; after the identity of the operating personnel is verified, transmitting the GPS coordinate data through the TLS or SSL encryption communication protocol; converting the GPS data to the Cartesian coordinate system through the coordinate transformation formula to obtain the coordinate data in the Cartesian coordinate system.

[0012] Furthermore, the coordinate transformation formula is: , where: is the coordinate value in the Cartesian coordinate system, Latitude and longitude coordinates in radians, a is the length of the semi-major axis of the WGS84 ellipsoid, b is the length of the semi-minor axis, and h is the elevation.

[0013] Further, in S5, a multi-source data correction mechanism is established to correct the coordinate data in the Cartesian coordinate system in step S4 to obtain corrected coordinate data, including: obtaining the inertial sensor data of the intelligent safety helmet, where the inertial sensor data includes acceleration and angular velocity; correcting the collected GPS coordinate data according to the inertial sensor data to obtain a drift error E1; the GPS signal above the cross-sea bridge is vulnerable to multipath effects caused by sea surface reflection. This solution can effectively filter out the GPS drift caused by sea surface reflection through the inertial sensor data correction mechanism (E1), improving the positioning stability, especially under harsh weather conditions with variable sea surface states.

[0014] Obtain the sensor model data updated by on-site sensors in the digital twin scenario; according to the obtained sensor model data, use the bridge BIM model to calculate the coordinate offset E2 of the operator's position; the structural deformation of the cross-sea bridge caused by temperature difference, wind load, and vehicle load is more significant than that of ordinary bridges. Through the BIM model deformation field correction mechanism (E2), the system can calculate the three-dimensional deformation field of the bridge in real time and provide accurate deformation compensation for the operator's position.

[0015] Obtain several historical positioning data in the same area from the database of the digital twin scenario; calculate the regional positioning error offset E3 according to the historical positioning data; the marine meteorological conditions are variable, and typhoons, thick fog, etc. will seriously interfere with the quality of GPS signals. This solution uses the historical positioning data correction mechanism (E3) to establish an error model under specific meteorological conditions to achieve intelligent compensation for positioning deviations under different weather conditions, ensuring all-weather supervision capabilities.

[0016] Use the drift error E1, the coordinate offset E2, and the regional positioning error offset E3 to correct the coordinate data in the Cartesian coordinate system in step S4 to obtain corrected coordinate data.

[0017] Further, according to the inertial sensor data, correct the collected GPS coordinate data to obtain a drift error E1, including: establishing the motion state equation of the intelligent safety helmet based on the acceleration and angular velocity in the inertial sensor data to obtain the estimated values of the operator's displacement, velocity, and acceleration; establishing the observation equation between the GPS coordinate data and the inertial sensor data, where the observation equation reflects the relationship between the state variables and the observation variables; calculating the position prediction value and the error covariance matrix of the operator at the current moment according to the motion state equation and the state estimate value at the previous moment; fusing the GPS coordinate data and the position prediction value according to the observation equation to obtain the corrected position estimate value; calculating the vector difference between the corrected position estimate value and the GPS coordinate data as the drift error E1.

[0018] Further, according to the acquired sensor model data, using the bridge BIM model, calculate the coordinate offset E2 of the operator's position, including: taking the acquired sensor model data as input, using the bridge BIM model to calculate the deformation field of the target bridge; according to the coordinate data in the Cartesian coordinate system in step S4, using the mapping relationship between the BIM model and the GIS platform in step S3, determine the relative position of the operator in the target bridge; according to the relative position of the operator in the target bridge and the deformation field of the target bridge, through a spatial interpolation algorithm, calculate the coordinate offset E2 of the operator.

[0019] Further, according to the historical positioning data, calculate the regional positioning error offset E3, including: according to the historical positioning data, construct a spatial interpolation function : , where P is the position to be evaluated, is the known historical positioning data, is the weight coefficient; is the Gaussian radial basis function; is the three-dimensional space coordinate in the Cartesian coordinate system; according to the coordinate data in the Cartesian coordinate system in step S4, determine the current position P of the operator, and use the spatial interpolation function to calculate the regional positioning error offset E3.

[0020] Compared with the prior art, the advantages of the present application are as follows:

[0021] Since the cross-sea bridge is located in an open sea environment, which causes GPS signal multipath effect verification. On the one hand, the present application fuses GPS and inertial data through the motion state equation and the observation equation, compensating for the short-term error of GPS signal drift, especially in the bridge occlusion area; on the other hand, the present application considers the influence of the deformation factor of the bridge in the actual environment on positioning, and compensates for the position deviation caused by the bridge structure deformation through a spatial interpolation algorithm; finally, a spatial interpolation function is constructed using the Gaussian radial basis function, making full use of historical experience data to correct systematic errors and eliminating the positioning deviation that has long existed in a specific area. The present application can correct the GPS coordinates in all directions from the time dimension (historical data), the space dimension (deformation field) and the dynamic dimension (inertial data), fundamentally solving the problem of low positioning accuracy caused by GPS signal drift. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 is an exemplary flowchart of a supervision method for bridge maintenance operations of the present application;

[0023] Figure 2 is a schematic diagram of the positioning effect of a virtual operator of the present application;

[0024] Figure 3 This is an effect diagram of signal transmission of an intelligent safety helmet for this application;

[0025] Figure 4 This is an effect diagram of virtual reality positioning and video transmission of operators for this application. Specific implementation manners

[0026] The following describes this application in detail in conjunction with the specification drawings and specific embodiments.

[0027] As Figure 1 shown, a sensor model, a personnel model, and a bridge BIM model are established respectively; the sensor model includes an intelligent safety helmet model; the sensor model and the bridge BIM model are registered using a geographic coordinate system; the personnel model, the registered sensor model, and the bridge BIM model are integrated through a GIS platform to obtain a digital twin scenario based on BIM+GIS; the GPS coordinate data of the operator is obtained through the intelligent safety helmet, and the GPS coordinate data is converted into coordinate data in a Cartesian coordinate system; a multi-source data correction mechanism is established to correct the coordinate data in the Cartesian coordinate system in step S4 to obtain corrected coordinate data; according to the corrected coordinate data, the position of the operator is displayed in real time in the digital twin scenario through the personnel model, and the operator is remotely supervised through the intelligent safety helmet.

[0028] S1. A sensor model, a personnel model, and a bridge BIM model are established respectively; the sensor model includes an intelligent safety helmet model; the intelligent safety helmet is a wearable device integrated with a GPS positioning module, a video call module, and an identity authentication module; the personnel model is used to simulate an operator wearing an intelligent safety helmet in a digital twin scenario. Among them, a three-dimensional modeling software (such as Revit or Rhino) can be used to construct the geometric model of the sensor. Based on ergonomic standards, a standardized three-dimensional personnel model is constructed. Based on the bridge CAD design drawings and measurement data, a professional BIM modeling software (such as Revit, Bentley Bridge, etc.) is used to construct a detailed bridge model.

[0029] S2. The sensor model and the bridge BIM model are registered using a geographic coordinate system, including: collecting the geographic data of the target bridge, the geographic data includes the GPS data of key reference points, and the key reference points include the expansion joints of the bridge; among them, the expansion joint is a key structural node of the bridge. Specifically, a unified coding rule is established: "bridge code - direction - serial number" (such as XH-N-01 represents the first expansion joint in the north direction of the New Hongqiao Bridge), and the position of each expansion joint is marked by precise measurement on the bridge deck, and each expansion joint is associated with the bridge mileage stake number, and multi-point data is collected for each expansion joint: 3 points on each side, forming a measurement array.

[0030] Map and transform the coordinate system of the bridge BIM model with the GPS data of key reference points to obtain the registered bridge BIM model. The seven-parameter or Helmert transformation method can be used to calculate the optimal set of transformation parameters according to the corresponding relationship of key reference points, and use the calculated transformation parameters to perform an overall coordinate transformation on the BIM model.

[0031] According to the component positions of the target bridge and the sensor layout plan, set the installation positions of the sensor models in the registered bridge BIM model to obtain the registered sensor models. The components include main girders, bridge piers, abutments, cable towers, and arch ribs. Among them, the sensors include but are not limited to strain gauges, accelerometers, displacement gauges, and environmental sensors.

[0032] Specifically, construct a digital twin basic model. The basic 3D model is an important foundation for the 3D simulation of the digital twin platform and provides key support for the simulation of the bridge maintenance management process. The platform model is mainly divided into personnel models, sensor models, and bridge BIM models, etc., and relies on the GIS large scene to realize the integrated loading of various models and the live simulation of the operation environment.

[0033] Perform georegistration on the BIM model, modify the project reference point to the actual geographical coordinates, and process the sensor model based on this reference point, and ensure that the relative positions of the sensor models to the target monitoring components in the bridge BIM are correct, which is convenient for integrated loading in the GIS scene. The personnel model obtains its own longitude and latitude coordinates based on the GPS positioning function integrated in the safety helmet, and is integrated into the GIS scene through coordinate transformation processing, and finally realizes the integrated display of multiple models on the platform.

[0034] The multi-dimensional models are effectively integrated through GIS. The relative spatial positions and relevant monitoring information of the operators and various sensors can be viewed in real time and intuitively inside the model. The outside of the model is the external GIS scene corresponding to the real world, realizing the complementary combination of micro and macro. Relying on the model integration scene, the real-time interaction between the management personnel and the operation site environment and the personnel wearing safety helmets is realized, effectively improving the bridge maintenance management and operation safety monitoring levels. Deeply connect and integrate the model with the scene data, and realize the association mapping between the integrated data and the digital scene, construct a multi-source integrated and multi-dimensional digital twin scene, and complete the integrated implementation of the platform according to the user-friendly design principles such as "usability, visibility".

[0035] By continuously detecting and updating sensor data, maintaining the dynamic association mapping between sensor entities and digital models, presenting real-time monitoring data in the form of charts, etc., and conducting historical statistics, etc., alarm processing such as corresponding model flashing is performed on real-time monitoring data exceeding the threshold. By obtaining the real-time positioning of the patrol personnel wearing intelligent safety helmets in real time, dynamically updating the position of the personnel model in the platform, and relying on functions such as video calls integrated in the safety helmet, the actual pictures of each component at the patrol site are transmitted and presented in real time.

[0036] Through the integrated platform constructed by fusing multi-source data with the model, it effectively avoids switching between different specialized management platforms, reflecting the design concept of "integrating functions with data rather than splitting data by functions". The whole process of early warning patrol from sensor alarm to accurately positioning BIM components, then to the patrol personnel quickly rushing to the target component and transmitting the real-time scene pictures on site is realized within a unified platform, effectively monitoring and managing multi-functional data in real time, achieving the purpose of intelligent management of bridge patrol, and being of great significance for assisting bridge maintenance work.

[0037] S3. By integrating the personnel model, the registered sensor model, and the bridge BIM model through the GIS platform, a digital twin scenario based on BIM+GIS is obtained. Select a GIS platform suitable for bridge engineering applications, and preferably select a professional platform that supports 3D visualization and BIM integration, such as the integration environment of SuperMap GIS, ArcGIS Pro, or QGIS and Cesium.

[0038] Through the GIS platform, spatial integration of the personnel model, the registered sensor model, and the bridge BIM model is carried out to obtain a multi-dimensional fusion model; specifically, develop a data converter for converting the BIM model to the GIS format, supporting multiple formats such as IFC, Revit, 3dsMax, etc.; import the personnel model into the GIS platform in the GLTF or 3D Tiles format; import the registered sensor model, retaining its spatial position and attribute information.

[0039] According to the multi-dimensional fusion model, using the coordinate transformation method, map and transform the local coordinate system of the registered bridge BIM model to the global coordinate system of the GIS platform. Specifically, the seven-parameter coordinate transformation method can be used to process the three-dimensional coordinate system transformation, and preferably, the TIN (triangulated irregular network) interpolation method is used to process the local deformation area.

[0040] Establish the mapping relationship between each component in the bridge BIM model and the corresponding geographical location. Specifically, design the associated table structure, including fields such as component ID, BIM coordinates, and geographical coordinates. Calculate the geographical bounding box (Bounding Box) for each component, and establish a spatial index to improve the efficiency of geographical queries. Convert the BIM components into 3D features in GIS, retain the attribute information and topological relationships of the components, and establish a two-way link mechanism to support locating BIM from GIS or locating GIS from BIM.

[0041] According to the mapping relationship, adopt a hierarchical classification coding system, such as "bridge code - component type - sensor type - serial number", with a unified coding length of 16 bits to ensure the scalability of the system. Coding special cases for different sensor types: Fixed sensors: Assign fixed codes according to the installation location and function; Smart safety helmets: Use dynamic coding to associate with the personnel ID; Temporary monitoring devices: Reserve a coding segment to support the access of temporary devices. Bind the code to the physical device through RFID tags or QR codes, and establish a code verification mechanism to prevent the misuse of devices. Create a sensor identifier mapping table, and design the table structure to include information such as identifier, device type, installation location, and data format. Mark the unique location of each sensor in the multi-dimensional fusion model, and establish a data channel between the sensor model and the physical sensor.

[0042] Collect on-site data using the sensors installed on-site, and update the on-site data to the corresponding sensor model in the multi-dimensional fusion model through the mapping table using the identifier. Based on the mapping relationship and the multi-dimensional fusion model with data updates, establish a digital twin scenario based on BIM + GIS.

[0043] S4. To ensure the security of the transmitted data, this platform establishes an authentication and authorization mechanism to authenticate the account and password bound to the safety helmet. Only the safety helmet terminal that has passed authentication and authorization can establish a connection with the server and transmit data. In terms of transmission, use the Transport Layer Security / Secure Sockets Layer (TLS / SSL) encryption communication protocol to encrypt the transmitted data to prevent eavesdropping or tampering, and establish a secure transmission channel between the smart safety helmet terminal and the data base server.

[0044] The coordinates obtained from the safety helmet positioning are longitude and latitude, while the underlying layer of the digital twin platform serves for graphic rendering and physical simulation, using the Cartesian coordinate system. Therefore, during the use of the positioning coordinates, the obtained longitude and latitude coordinates still need to be converted into Cartesian coordinates through the conversion formula to provide support for the visual display of the platform model. The coordinate conversion formula is:

[0045]

[0046] Where: are the coordinate values in the Cartesian coordinate system, Latitude and longitude coordinates in radians, a is the semi-major axis length of the WGS84 ellipsoid, a = 6,378,137.0 meters (semi-major axis); b = 6,356,752.3142 meters (semi-minor axis); h is the elevation.

[0047] Specifically, on the one hand, GPS signals are reflected on the sea surface to form multipath interference, causing positioning jumps; on the other hand, the steel structure of the bridge blocks and reflects GPS signals. Through an accurate WGS84-Cartesian coordinate conversion and multi-source data fusion correction mechanism, this application realizes the effective filtering and correction of GPS positioning signals in complex marine environments, and the converted high-precision three-dimensional Cartesian coordinate data is seamlessly integrated with the subsequent digital twin scenario based on BIM+GIS, providing a unified spatial reference framework for bridge maintenance supervision.

[0048] On the one hand, in the environment of cross-sea bridges, GPS signal reception faces unique challenges. The sea surface, as an ideal reflector, generates a strong multipath effect, causing the receiver to receive both direct and reflected signals simultaneously. This coordinate conversion scheme establishes a more suitable mathematical basis for physical constraint analysis by converting latitude and longitude into the Cartesian coordinate system. On this basis, the system uses a trajectory continuity test algorithm based on three-dimensional space to implement "physical feasibility filtering", effectively identifying and eliminating non-physical jump points caused by sea surface reflection.

[0049] Specifically, the Z-axis information is clearly separated in the Cartesian system, enabling the system to independently analyze elevation anomalies. Regarding the elevation fluctuations particularly prone to being caused by sea surface reflection, this system establishes a constraint model based on the known elevation of the bridge deck. When it is detected that the elevation data deviates from the expected height of the bridge deck by more than the threshold, the correction mechanism is automatically triggered, significantly improving the positioning stability.

[0050] On the other hand, the large steel structures of cross-sea bridges not only block GPS signals but also produce complex reflection and diffraction phenomena, forming "signal canyons" and "false signal areas". The coordinate conversion system of this application combines the accurate geometric information of the BIM model to create a "spatial visibility map", pre-marking possible weak and abnormal GPS signal areas.

[0051] In the Cartesian coordinate system, the system can accurately calculate the relative position relationship between the operator and the steel structure. When it is detected that the person enters a known signal abnormal area, the weight of the inertial navigation system is automatically increased, while the credibility of GPS data is reduced. In addition, by analyzing a large amount of historical positioning data, the system establishes an "error characteristic model" specific to each section of the bridge, and implements differential correction for systematic deviations in different structural environments.

[0052] Finally, the conversion of GPS coordinates to the Cartesian coordinate system not only solves the signal problem, but also provides fundamental technical support for the digital twin platform. The converted three-dimensional coordinates are directly compatible with the spatial expression system of mainstream 3D rendering engines and can be used for scene rendering without secondary conversion, significantly improving the real-time performance of the system.

[0053] Especially in the application of bridge deformation monitoring, the linear characteristics of the Cartesian coordinate system make the calculation of minute deformations more direct and accurate. The system can express the three-dimensional displacement vectors of various parts of the bridge with millimeter-level accuracy, providing a high-quality data basis for structural health monitoring. At the same time, this coordinate system is convenient for docking with finite element analysis systems and supports advanced engineering analysis functions such as stress fields and vibration modes. In terms of interdisciplinary collaboration, the unified Cartesian coordinate system breaks the traditional boundaries between BIM (Building Information Modeling) and GIS (Geographic Information System), achieving a seamless scale transition from the macroscopic geographical environment to the microscopic component details. This integration enables the comprehensive analysis of the positions of maintenance personnel, sensor data, and the bridge structure state within the same reference framework, significantly enhancing the data correlation and decision-making support capabilities.

[0054] As Figure 2 shown, for the converted model coordinates, it is necessary to continue with re-registration according to the environmental model to minimize the errors generated in aspects such as positioning accuracy and coordinate conversion algorithms. Ultimately, it is possible to have on-site personnel wearing smart safety helmets standing at the expansion joint, and the virtual human model on the digital twin model is accurately positioned and displayed at the corresponding expansion joint model in the BIM model.

[0055] The safety helmet management system can manage the list of safety helmets under an account. It can be bidirectionally bound through the unique ID of the safety helmet terminal. After successful binding, the safety helmet terminal can perform interactive operations such as positioning acquisition and video calls after login authentication. The safety helmet terminal integrates an automatic boot login function (the terminal needs to insert a SIM card or connect to a Wi-Fi network to ensure its normal network function), automatically sending a login request carrying the terminal's unique ID to the server. After the server successfully verifies it, it will modify the status of the corresponding terminal under the account of this terminal to online and open the interactive permissions.

[0056] The platform user login intelligent safety helmet system adopts a dual authentication mode of "username password + token". The user submits a login request to the server by entering the username and password. The server verifies the user's identity. If the verification is successful, the login is successful and a token code for subsequent instruction verification is returned. If the verification fails, the login is blocked. After successful login, the user can obtain the list of safety helmet terminal information by sending a request (the request needs to carry the token code, the same below). The list contains information such as each terminal ID, terminal alias, whether the terminal is online, and terminal location. Bind the platform personnel model to the online terminals in the list. The initial coordinates of the model are the system coordinates converted from the current geographical coordinates of the terminal returned by the list. For terminals that need to bind the location in real time, establish a WebSocket two-way communication channel between the platform and the server by sending a real-time location acquisition request carrying the corresponding ID. After success, the platform can obtain the real-time location of the terminal for binding the mobile personnel model. It should be noted that once the personnel model is bound, the user cannot freely control the movement of the model, but is bound and moved in real time by the geographical information coordinates transmitted by the safety helmet terminal.

[0057] S5. Establish a multi-source data correction mechanism to correct the coordinate data in the Cartesian coordinate system in step S4 to obtain the corrected coordinate data. Obtain the inertial sensor data of the intelligent safety helmet. The inertial sensor data includes acceleration and angular velocity. The acceleration can be collected by a MEMS accelerometer, and the angular velocity can be collected by a MEMS gyroscope.

[0058] First, correct the collected GPS coordinate data according to the inertial sensor data to obtain the drift error E1. Among them, the inertial sensor data can be collected by a three-axis accelerometer, a three-axis gyroscope, and a three-axis magnetometer.

[0059] In this embodiment, according to the inertial sensor data, a 15-dimensional state vector is established, including position (3 dimensions), velocity (3 dimensions), attitude (3 dimensions), accelerometer zero bias (3 dimensions), and gyroscope zero bias (3 dimensions).

[0060] Implement a continuous-time nonlinear state equation: position differential equation: ; velocity differential equation: , represents the acceleration measurement value; attitude quaternion differential equation: ; accelerometer zero bias differential equation: , where represents the accelerometer zero bias vector; gyroscope zero bias differential equation: , represents the gyroscope three-axis zero bias vector; among them, is the attitude rotation matrix, q represents the quaternion of the attitude, g is the gravity vector, to For system noise, the fourth-order Runge-Kutta method is used to discretize the state equation to adapt to the characteristics of the nonlinear system.

[0061] Construct the observation equation including the GPS observation equation and the IMU observation equation. Establish the GPS observation equation: , where represents the three-dimensional position vector in the Cartesian coordinate system; is the observation noise; the observation noise covariance matrix is dynamically adjusted according to the GPS signal quality:

[0062] ; . Among them, is the horizontal dilution of precision, representing the influence of the GPS satellite geometric distribution on the horizontal precision; SNR is the signal-to-noise ratio, representing the GPS signal strength; α and β are weight coefficients, and base_var represents the reference observation variance, representing the minimum variance value of GPS measurement under ideal conditions; IMU observation equation: Establish the acceleration observation equation: , represents the observed value of the accelerometer; represents the accelerometer bias; represents the acceleration observation noise. represents the acceleration component measured by the accelerometer; Establish the angular velocity observation equation: , are the observed quantities of GPS position, acceleration, and angular velocity respectively; represents the angular velocity component measured by the gyroscope; represents the gyroscope bias; represent the observation noises corresponding to GPS position, acceleration, and angular velocity respectively. Construct the zero velocity update (ZUPT) observation equation to detect the stationary state and provide additional constraints.

[0063] Predict the state at the next moment based on the motion state equation: , is the predicted value of the state vector at time k + 1; represents the estimated value of the current state; represents the control input at the current moment; is the time step; represents the nonlinear state equation; represents the control input vector at time k; Calculate the state prediction covariance: ; where is the state transition matrix (Jacobian matrix of the state equation), represents the predicted value of the covariance matrix at time k + 1; is the covariance matrix of the current state; is the process noise covariance; the Jacobian matrix is calculated using the numerical differentiation method to adapt to the characteristics of the nonlinear system. represents the state transition matrix, which is the Jacobian matrix of the nonlinear state equation f(*) at the current operating point.

[0064] Specifically, in this embodiment, the fourth-order Runge-Kutta (RK4) numerical integration method is used to derive the nonlinear state equation from the continuous-time differential equation , and for the attitude quaternion integration, quaternion multiplication operation () is used and normalization is performed after each step of integration; to adapt to the high sampling rate of IMU data, a variable step-size integration strategy is used, and it is adaptively adjusted according to the time difference between IMU and GPS data ; the Jacobian matrix is calculated by central difference numerical differentiation: ; where represents the element in the i-th row and j-th column of the Jacobian matrix, indicating the influence of the change of the j-th component of the state vector on the i-th component of the state derivative; represents the j-th standard basis vector, a 15-dimensional vector with 1 in the j position and 0 in other positions; δ represents the perturbation size of the numerical differentiation, and in this embodiment, it is taken as . represents the value of the i-th component of the state equation after the state vector is positively perturbed by δ in the j-th direction; represents the value of the i-th component of the state equation after the state vector is negatively perturbed by δ in the j-th direction.

[0065] Calculate the Kalman gain:

[0066] , represents the Kalman gain matrix; represents the observation matrix; represents the observation noise covariance matrix; update the state estimate: ; update the covariance estimate:

[0067] , and the Joseph form is applied to update the covariance to ensure numerical stability and positive definiteness. represents the actual observation vector at time k + 1; represents the expected observation value calculated based on the state prediction value. represents the identity matrix, with the same size as the dimension of the state vector.

[0068] Implement adaptive noise estimation based on the residual sequence: , where is the observation residual, and α is the forgetting factor (0.05 - 0.1); and They represent the observation noise covariance matrices estimated at times k and k-1 respectively;

[0069] Calculate the drift error E1 and extract the corrected position estimate: ; Obtain the original GPS measurement values: ; Calculate the drift error vector: ; Calculate the error norm: .

[0070] Then, calculate the coordinate offset E2 caused by the bridge deformation field. Based on the bridge BIM model, establish a simplified finite element calculation model, input the real-time sensor data as boundary conditions and constraint conditions, solve the static equilibrium equation, and calculate the full-bridge deformation field: , where is the stiffness matrix, is the displacement vector, is the external force vector, and post-process the calculation results to obtain the three-dimensional space deformation field distribution.

[0071] Based on the actual deformation data of the measurement points, establish a spatial deformation field interpolation model: , where is the deformation of any point, is the deformation of the known measurement points, is the weight coefficient; is the radial basis function, such as the Wendland function or the cubic spline function; is the influence radius, which is determined according to the structural characteristics and sensor distribution; apply the finite element results to constrain the interpolation to ensure the physical rationality of the deformation field.

[0072] Using the Cartesian coordinate data in step S4 and combining with the mapping relationship in step S3, calculate the position of the operator in the local coordinate system of the bridge : , where M is the transformation matrix from the local coordinate to the global coordinate, and determine the bridge component where the operator is located and the relative position parameters; represents the position of the operator in the global coordinate system.

[0073] According to the relative position of the operator, query the deformation field data and extract the three-dimensional deformation vector at the corresponding position: , convert the deformation vector in the local coordinate system to the global coordinate system: , calculate the influence of the deformation on the position, and generate the coordinate offset E2.

[0074] Finally, perform statistical calibration on E3 based on historical data, establish a spatio-temporal database to store historical positioning data and corresponding calibration results, and design a spatial index structure such as an R-tree or a quadtree to support fast spatial queries. According to the current position, query the historical positioning data in the spatial neighborhood: Neighbor Set = Spatial Query(Current Position, Radius).

[0075] Construct a spatial interpolation function and define the Gaussian radial basis function: , where r is the spatial distance and σ is the smoothing parameter, which is adaptively adjusted according to the data density. Construct the interpolation system equation: ; where is the function value matrix and e is the historical error vector; use the regularized least squares method to solve the weight vector w: ; where λ is the regularization parameter to prevent overfitting and singularity problems.

[0076] According to the current position P, calculate the distances to each historical data point: ; calculate the values of each basis function: ; synthesize the error offset: ; decompose it into three-dimensional components: .

[0077] Use the drift error E1, the coordinate offset E2, and the regional positioning error offset E3 to correct the coordinate data in the Cartesian coordinate system in step S4 to obtain the corrected coordinate data. Specifically, define the fusion correction model: , where is the adaptive weight coefficient, satisfying . Obtain the Cartesian coordinate data in step S4: , calculate the comprehensive correction amount: ; apply the correction to obtain the final coordinates: , and perform a rationality check on the correction results to eliminate the obviously abnormal correction results.

[0078] S6. According to the corrected coordinate data, the positions of the operators are displayed in real time in the digital twin scenario through the personnel model, and the operators are remotely supervised through intelligent safety helmets. On the premise that the user ensures that the intelligent safety helmet is turned on and in a good network environment, the platform has functions such as video calls, danger alarms, and trajectory playback.

[0079] 1) Video call. The platform can make video calls to the safety helmet devices bound to the current user account and online. By clicking on the personnel model corresponding to the safety helmet terminal ID that needs a video call, a call request is sent to the server. After verification, the server calls the corresponding camera module, voice module, etc. of the safety helmet terminal to respond to the request. After the safety helmet emits a "ding-dong" prompt sound to alert the wearer, a video call connection is formally established with the platform. This function can be used to improve the accuracy and real-time nature of managers' understanding of the job site situation, and greatly ensure the accuracy and timeliness of management decisions.

[0080] 2) Danger alarm. The platform has established a monitoring and alarm area for areas on the bridge inspection that may be dangerous. The positions of each safety helmet terminal in the platform are monitored and compared in real time. When the terminal approaches within 5 meters of the danger alarm area, the alarm light of the safety helmet flashes to give a danger reminder. Once it enters the danger alarm area, the buzzer in the safety helmet terminal sounds an alarm to drive away the personnel. This function can effectively guarantee the safety of the operating personnel and improve the real-time nature of inspection safety management.

[0081] 3) Trajectory playback. The server records the position and time information of the online safety helmet terminals in real time and stores them in the database. The platform realizes the function of backtracking and displaying the personnel operation trajectory by querying the terminal coordinate list at a specified time or of a specified batch and showing the connection lines. This function can be used in scenarios such as personnel work attendance, checking for missed inspections, and statistics of inspection mileage.

[0082] Since the virtual personnel models on the platform have the characteristics of the height, body shape, etc. of ordinary people, they can intuitively reflect information such as the operation space and operation difficulty. This information will be transmitted in real time to the comprehensive managers at the platform's user end for more effective remote operation guidance. When the data transmitted back by the bridge sensors is abnormal, the managers can quickly locate the target sensor position on the platform display end and notify the inspection personnel to go and confirm and handle it. After the personnel arrive at the scene, they can turn on the video call function of the safety helmet and transmit the on-site situation to the management center in real time, facilitating the managers to grasp the real on-site situation in real time for more accurate remote guidance decisions.

[0083] Such as Figure 3As shown in the figure, this application uses the WebSocket two-way communication protocol to establish a persistent connection between the intelligent safety helmet terminal and the data base server, so as to realize real-time acquisition and transmission of the current position coordinates of the safety helmet terminal and the instructions issued by the platform. As the HTTP protocol, which is the basis of Web communication, most of the time and network transmission volume of its requests are used to establish the connection between the server and the browser. After the traditional HTTP request completes the connection, it only completes the push and pull of request data once. For multiple repeated requests of the same type, it shows a large waste of time and network costs. The HTML5 WebSocket protocol establishes a TCP socket connection after the first request connection, so that each request does not need to repeatedly establish the connection between the server and the browser frequently, basically achieving real-time communication response and greatly increasing the communication efficiency.

[0084] As Figure 4 shown in the figure, the platform accelerates the network transmission of audio and video signals between the safety helmet terminal and the user terminal in real time, and adopts packet loss recovery technologies such as "forward error correction" and "retransmission mechanism" to achieve 71% packet loss resistance, correct and compensate for the lost audio and video signals during the transmission process, and greatly ensure the quality of the call; adopts broadband management technology to degrade other request data during the audio and video transmission process, ensuring the priority and stability of the audio and video signal transmission, and realizing the full-link QoS guarantee. The audio and video calls under narrowband are still stable and smooth; adopts the full-band audio processing technology to process the frequency range of the audio signal, realizing effects such as intelligent echo and noise cancellation, ensuring the high quality of the audio, greatly improving the clarity of the audio, and transmitting and restoring real emotions more realistically.

[0085] Based on BIM+GIS as the scene and the personnel model as the carrier, after converting and processing the real-time transmitted position coordinate information of the safety helmet terminal, the platform drives the coordinates of the personnel model in the platform to realize the dynamic simulation of the operation scene and relative position of front-line workers and other operation site information. Relying on the video call function integrated in the safety helmet terminal, the obtained audio and video information is displayed in the video information box above the personnel model, realizing the multi-source data fusion display of the simulated digital scene and the real audio and video pictures, providing more comprehensive and real on-site information for the management personnel.

[0086] The above has schematically described the present invention and its implementation manners. This description is not restrictive. Without departing from the spirit or basic features of the present application, the present application can be implemented in other specific forms. What is shown in the drawings is only one of the implementation manners of the present invention, and the actual structure is not limited thereto. Any reference signs in the claims should not limit the claimed claims. Therefore, if those of ordinary skill in the art are inspired by it and, without departing from the purpose of the present invention, design similar structural manners and embodiments to the technical solution without creative efforts, they shall fall within the protection scope of this patent. In addition, the term "comprising" does not exclude other elements or steps, and the word "a" before an element does not exclude including "a plurality of" such elements. The multiple elements stated in the product claims can also be implemented by one element through software or hardware. The terms such as "first" and "second" are used to indicate names and do not represent any specific order.

Claims

1. A supervision method for bridge maintenance operations, characterized in that, Including: S1. Respectively establish a sensor model, a personnel model, and a bridge BIM model; the sensor model includes an intelligent safety helmet model; S2. Register the sensor model and the bridge BIM model using a geographic coordinate system; S3. Integrate the personnel model, the registered sensor model, and the bridge BIM model through a GIS platform to obtain a digital twin scenario based on BIM+GIS; S4. Obtain the GPS coordinate data of the operating personnel through the intelligent safety helmet, and convert the GPS coordinate data into coordinate data in the Cartesian coordinate system; S5. Establish a multi-source data correction mechanism to correct the coordinate data in the Cartesian coordinate system in step S4 to obtain corrected coordinate data; S6. According to the corrected coordinate data, the position of the operating personnel is displayed in real time in the digital twin scenario through the personnel model, and the operating personnel are remotely supervised through the intelligent safety helmet; S5. Obtain the corrected coordinate data, including: Obtain the inertial sensor data of the intelligent safety helmet, and the inertial sensor data includes acceleration and angular velocity; According to the inertial sensor data, correct the collected GPS coordinate data to obtain a drift error E1; Obtain the sensor model data updated by on-site sensors in the digital twin scenario; According to the obtained sensor model data, use the bridge BIM model to calculate the coordinate offset E2 of the position of the operating personnel; Obtain several historical positioning data in the same area from the database of the digital twin scenario; According to the historical positioning data, calculate the regional positioning error offset E3; Use the drift error E1, the coordinate offset E2, and the regional positioning error offset E3 to correct the coordinate data in the Cartesian coordinate system in step S4 to obtain corrected coordinate data.

2. The supervision method for bridge maintenance operations according to claim 1, characterized in that: The intelligent safety helmet is a wearable device integrated with a GPS positioning module, a video call module, and an identity authentication module; The personnel model is used to simulate the operating personnel wearing an intelligent safety helmet in the digital twin scenario.

3. The supervision method for bridge maintenance operations according to claim 1, characterized in that: S2. Register the sensor model and the bridge BIM model using a geographic coordinate system, including: Collect the geographic data of the target bridge, and the geographic data includes the GPS data of key reference points, and the key reference points include the expansion joints of the bridge; Map and transform the coordinate system of the bridge BIM model with the GPS data of the key reference points to obtain the registered bridge BIM model; According to the component positions of the target bridge and the sensor layout plan, set the installation positions of the sensor model in the registered bridge BIM model to obtain the registered sensor model; the components include main girders, piers, abutments, cable towers, and arch ribs.

4. The supervision method for bridge maintenance operations according to claim 1, characterized in that: S3. Obtain the digital twin scenario based on BIM+GIS, including: Perform spatial integration on the personnel model, the registered sensor model, and the bridge BIM model through a GIS platform to obtain a multi-dimensional fusion model; According to the multi-dimensional fusion model, using the coordinate transformation method, map and transform the local coordinate system of the registered bridge BIM model with the global coordinate system of the GIS platform to establish the mapping relationship between each component in the bridge BIM model and the corresponding geographical location; According to the mapping relationship, encode each sensor device in the sensor model, assign a unique identifier to each sensor device, and establish a mapping table between the identifier and the corresponding sensor model in the multi-dimensional fusion model; Collect on-site data using the sensors installed on-site, and update the on-site data to the corresponding sensor model in the multi-dimensional fusion model through the mapping table using the identifier; Establish a digital twin scenario based on BIM+GIS according to the mapping relationship and the multi-dimensional fusion model with updated data.

5. The supervision method for bridge maintenance operations according to claim 1, characterized in that: S4. Convert the GPS coordinate data into coordinate data in the Cartesian coordinate system, including: Collect the GPS coordinate data of the operator through the intelligent safety helmet; Verify the identity of the operator using the security authentication and authorization mechanism; After the operator's identity is verified, transmit the GPS coordinate data through the TLS or SSL encryption communication protocol; Convert the GPS data to the Cartesian coordinate system through the coordinate conversion formula to obtain the coordinate data in the Cartesian coordinate system.

6. The supervision method for bridge maintenance operations according to claim 5, characterized in that: The coordinate conversion formula is: where: (X, Y, Z) are coordinate values in a Cartesian coordinate system, are latitude and longitude coordinates in radians, a is the length of the semi-major axis of the WGS84 ellipsoid, b is the length of the semi-minor axis, and h is the elevation.

7. The supervision method for bridge maintenance operations according to claim 1, characterized in that: Obtain the drift error E1, including: According to the acceleration and angular velocity in the inertial sensor data, establish the motion state equation of the intelligent safety helmet to obtain the estimated values of the displacement, velocity, and acceleration of the operator; Establish the observation equation between the GPS coordinate data and the inertial sensor data, and the observation equation reflects the relationship between the state variables and the observation variables; According to the motion state equation and the state estimate value at the previous moment, calculate the position prediction value and the error covariance matrix of the operator at the current moment; According to the observation equation, fuse the GPS coordinate data and the position prediction value to obtain the corrected position estimate value; Calculate the vector difference between the corrected position estimate value and the GPS coordinate data as the drift error E1.

8. The supervision method for bridge maintenance operations according to claim 1, characterized in that: Calculate the coordinate offset E2 of the operator's position, including: Use the obtained sensor model data as input and calculate the deformation field of the target bridge using the bridge BIM model; According to the coordinate data in the Cartesian coordinate system in step S4, and using the mapping relationship between the BIM model and the GIS platform in step S3, determine the relative position of the operator in the target bridge; According to the relative position of the operator in the target bridge and the deformation field of the target bridge, calculate the coordinate offset E2 of the operator through the spatial interpolation algorithm.

9. The supervision method for bridge maintenance operations according to claim 1, characterized in that: Calculate the regional positioning error offset E3, including: Construct the spatial interpolation function E(x, y, z) according to the historical positioning data: Among them, P is the position to be evaluated, P i is the known historical positioning data, w i is the weight coefficient; φ(||P - P i ||) is the Gaussian radial basis function; (x, y, z) are the three-dimensional space coordinates in the Cartesian coordinate system; Based on the coordinate data in the Cartesian coordinate system in step S4, determine the current position P of the operator, and use the spatial interpolation function E(x, y, z) to calculate the regional positioning error offset E3.

Images