Pipeline deformation monitoring method, system and equipment based on BIM and AR and medium

Abstract

The invention relates to the technical field of pipeline deformation monitoring, in particular to a BIM (Building Information Modeling) and AR (Augmented Reality)-based pipeline deformation monitoring method, system and equipment and a medium. According to the method, firstly, an accurate BIM model is established through multi-site scanning, then, model simplification and optimization are performed to adapt to AR display, then, real-time image data are acquired by using a depth sensor and an SLAM technology, virtual-real superposition is realized through feature matching, and finally, deformation parameters are calculated based on a superposition effect and a maintenance scheme is generated. Omnibearing monitoring of pipeline deformation is achieved, the deformation condition is visually displayed in a virtual-real superposition mode, and more scientific and efficient technical support is provided for a pipe network maintenance decision.

Description

Technical Field

[0001] The present application relates to the technical field of pipeline deformation monitoring, and in particular to a pipeline deformation monitoring method, system, equipment and medium based on BIM and AR. Background Art

[0002] Urban underground pipeline networks are a vital component of urban infrastructure, crucial for the normal operation of cities and the lives of residents. With the acceleration of urbanization and the increase in service life, underground pipelines are becoming increasingly vulnerable to deformation, damage, and other problems, posing a threat to the safe operation of cities. Therefore, it is imperative to establish efficient and accurate methods for monitoring pipeline deformation.

[0003] Currently, pipeline deformation monitoring relies primarily on in-pipeline robotic inspection technology and ground-based radar detection. In-pipeline robots use cameras and sensors to scan and inspect the pipeline's interior; ground-based radar uses electromagnetic waves to detect the pipeline's geometric contours and structural condition. In-pipeline robotic inspection is limited by pipeline size and accessibility; while ground-based radar detection accuracy is significantly affected by geological conditions, and data processing is complex, making it incapable of quickly and intuitively displaying pipeline deformation. This situation requires further improvement. Summary of the invention

[0004] In order to solve the problem that existing pipeline deformation monitoring cannot intuitively and quickly display pipeline deformation conditions, this application provides a pipeline deformation monitoring method, system, equipment and medium based on BIM and AR, which adopts the following technical solutions: In a first aspect, the present application provides a pipeline deformation monitoring method based on BIM and AR, comprising the following steps: Obtain the geometric information and attribute information of the pipeline and establish a BIM model of the pipeline; generating a lightweight model for AR display based on the BIM model; Based on the lightweight model, AR equipment is used to scan and image the actual pipeline to obtain real-time image data of the pipeline; Based on the real-time image data, the BIM model is registered and fused with the AR real scene to obtain a virtual and real superimposed display effect of the pipeline; Based on the virtual-real superposition display effect, the actual deformation parameters of the pipeline are calculated to determine the deformation state of the pipeline; A pipeline maintenance decision plan is generated according to the deformation state.

[0005] By adopting the above technical solution, the present application first establishes an accurate BIM model through multi-site scanning, then simplifies and optimizes the model to adapt to AR display, then uses a depth sensor and SLAM technology to obtain real-time image data, realizes virtual-real superposition through feature matching, and finally calculates deformation parameters based on the superposition effect and generates a maintenance plan; achieving all-round monitoring of pipeline deformation, and intuitively showing the deformation situation through the virtual-real superposition method, providing more scientific and efficient technical support for the decision-making of pipe network maintenance.

[0006] Optionally, the geometric information includes the size, burial depth and spatial position of the pipeline; the attribute information includes the material, connection method and service life of the pipeline; obtaining the geometric information and attribute information of the pipeline and establishing a BIM model of the pipeline specifically includes the following steps: Perform multi-site scanning on the pipeline, and obtain a point cloud model according to the size, burial depth and spatial position of the pipeline; Generate a three-dimensional grid model of the pipeline based on the point cloud model to obtain the geometric surface of the pipeline; Set parameterized features according to the geometric surface of the pipeline to obtain a regularized three-dimensional model of the pipeline; Establish an attribute database according to the material, connection method and service life of the pipeline; Perform associative mapping between the regularized three-dimensional model and the attribute database, and finally generate a BIM model of the pipeline.

[0007] By adopting the above technical solution, the present application first obtains complete pipeline point cloud data through multi-site scanning, and then generates an accurate surface model of the pipeline through a mesh reconstruction algorithm; then based on the parametric feature extraction technology, converts the irregular mesh into a regularized three-dimensional model; at the same time, establishes an attribute database containing information such as material, connection method and service life; finally, integrates the geometric model and attribute information through associative mapping; improving the acquisition accuracy of geometric information and realizing the systematic integration of attribute data.

[0008] Optionally, generating a lightweight model for AR display according to the BIM model specifically includes the following steps: Import the BIM model into processing software, simplify the mesh and optimize the geometry of the model to obtain a simplified model; Perform texture mapping and material optimization based on the simplified model to generate a lightweight display model; Set AR marker points according to the lightweight display model to establish a spatial reference system; Perform model coordinate transformation based on the spatial reference system to obtain positioning parameters adapted to the AR device; Integrate the positioning parameters with the lightweight display model, and finally generate a lightweight model for AR display.

[0009] By adopting the above technical solutions, the present application first performs mesh simplification and geometric optimization on the BIM model to reduce the data volume while maintaining key features; then improves the display effect through texture mapping and material optimization; then sets AR marker points to establish a unified spatial reference system; then calculates the positioning parameters adapted to the AR device based on the spatial reference system; and finally integrates the positioning parameters with the lightweight model; reducing the data processing load and improving the smoothness of AR display.

[0010] Optionally, the AR device includes a depth sensor, an inertial measurement unit, and a GNSS receiver; the AR device is used to scan and image the actual pipeline to obtain real-time image data of the pipeline, which specifically includes the following steps: Combining the pipeline network spatial data in the GIS system and the GNSS signal to determine the absolute position of the AR device; Based on the depth sensor to obtain the environmental feature point cloud, constructing a local environmental map through SLAM technology to determine the relative position; Starting the image acquisition module of the AR device to obtain the real-time video stream of the pipeline; Associating the absolute position and the relative position with the real-time video stream to obtain the pipeline contour features; Performing real-time tracking and positioning calculation on the pipeline contour features, and finally generating the real-time scan data of the pipeline.

[0011] By adopting the above technical solutions, the present application first combines the GIS system and the GNSS signal to determine the absolute spatial position of the AR device; then uses the depth sensor to collect the environmental feature point cloud and constructs an accurate local environmental map through SLAM technology; then starts the image acquisition module to obtain the real-time video stream; then performs multi-source data fusion on the absolute position and relative position information with the video stream to extract the pipeline contour features; finally, through the real-time tracking algorithm to maintain continuous positioning update, improving the scanning accuracy and realizing the real-time tracking and continuous acquisition of pipeline features.

[0012] Optionally, according to the real-time image data, registering and fusing the BIM model with the AR real scene to obtain the virtual-real superposition display effect of the pipeline, which specifically includes the following steps: Obtaining the pipeline feature points and contour lines in the real-time scan data to establish a three-dimensional feature data set; Performing feature matching between the feature data set and the lightweight model to calculate the position deviation parameters; Performing coordinate transformation and spatial alignment according to the position deviation parameters; Based on the spatial alignment result, generating the superposition display of the virtual model and the real scene; Real-time updating the position parameters of the superposition display to obtain the virtual-real superposition display effect including the pipeline geometric contour, material features, and position information.

[0013] By adopting the above technical solutions, the present application first extracts the feature points and contour lines of the pipeline from the real-time scan data to construct a complete three-dimensional feature data set; then performs feature matching between the feature data set and the lightweight model to calculate the position deviation parameters; then performs precise coordinate transformation and spatial alignment based on the deviation parameters; then generates the superimposed display of the virtual model and the real scene; and finally continuously optimizes the display effect through a real-time update mechanism; improving the accuracy of feature matching and effectively suppressing the accumulation of alignment errors.

[0014] Optionally, the deformation parameters include the cross-sectional deformation amount, deformation area, and deformation rate of the pipeline; the deformation states include three levels: normal, warning, and danger; calculating the actual deformation parameters of the pipeline and determining the deformation state based on the virtual-real superimposed display effect specifically includes the following steps: Extract the actual contour data of the pipeline in the virtual-real superimposed display effect; Generate a transverse cross-section of the pipeline at the deformation position and calculate the cross-sectional deformation amount between the actual cross-section and the standard cross-section of the BIM model; Perform surface fitting on the deformed part, compare it with the standard surface of the BIM model, and calculate the deformation area; Calculate the deformation rate of the pipeline based on the cross-sectional deformation amount and the deformation area; Perform hierarchical evaluation according to the preset deformation parameter thresholds to determine the deformation state level of the pipeline.

[0015] By adopting the above technical solutions, the present application first extracts the actual contour data of the pipeline from the virtual-real superimposed display effect; then generates a transverse cross-section at the deformation position and compares it with the standard cross-section to calculate the cross-sectional deformation amount; then calculates the area parameter of the deformed area through surface fitting technology; then calculates the deformation rate based on these data; and finally performs comprehensive evaluation according to the preset multi-level threshold system; realizing the all-round quantification of deformation parameters and providing more comprehensive deformation feature data.

[0016] Optionally, the method further includes the following steps: Obtain the pipeline BIM model and real-time scan data in the virtual-real superimposed display effect; Establish a split-screen display interface in the AR display to simultaneously display the pipeline BIM model and the real-time scene; Perform color marking and numerical marking on the deformed part according to the deformation state level; Real-time display the cross-sectional deformation amount, deformation area, and deformation rate data of the deformed part.

[0017] By adopting the above technical solution, the present application first obtains the BIM model of the pipeline and real-time scanning data; then constructs a split-screen display interface in the AR display to realize the synchronous display of the standard model and the actual scene; then performs intelligent color marking and numerical marking on the deformed part; finally, updates and displays various deformation parameter data in real time; provides an intuitive comparison between the standard state and the actual state, and realizes the instant transmission of deformation information through color coding and numerical marking, and at the same time supports interactive viewing and analysis, greatly improving the work efficiency of on-site monitoring.

[0018] In a second aspect, the present application provides a pipeline deformation monitoring system based on BIM and AR, including: A BIM model establishment module, configured to obtain the geometric information and attribute information of the pipeline and establish a BIM model of the pipeline; A lightweight model generation module, configured to generate a lightweight model for AR display according to the BIM model; A real-time image data acquisition module, configured to scan and image the actual pipeline by using an AR device based on the lightweight model to obtain real-time image data of the pipeline; A superimposed display effect acquisition module, configured to register and fuse the BIM model with the AR real scene according to the real-time image data to obtain a virtual-real superimposed display effect of the pipeline; A deformation state determination module, configured to calculate the actual deformation parameters of the pipeline and determine the deformation state of the pipeline based on the virtual-real superimposed display effect; A maintenance decision plan generation module, configured to generate a pipeline maintenance decision plan according to the deformation state.

[0019] In a third aspect, the present application provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the steps of the above-mentioned pipeline deformation monitoring method based on BIM and AR are implemented.

[0020] In a fourth aspect, the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned pipeline deformation monitoring method based on BIM and AR are implemented.

[0021] In summary, the present application includes at least one of the following beneficial technical effects: 1. This application first establishes an accurate BIM model through multi-site scanning, then simplifies and optimizes the model to adapt to AR display. Next, it uses a depth sensor and SLAM technology to obtain real-time image data, realizes virtual-real superposition through feature matching, and finally calculates deformation parameters based on the superposition effect and generates a maintenance plan; achieving all-round monitoring of pipeline deformation, and intuitively showing the deformation situation through the virtual-real superposition method, providing more scientific and efficient technical support for pipeline network maintenance decision-making; 2. This application first obtains complete pipeline point cloud data by multi-site scanning, and then generates an accurate surface model of the pipeline through a mesh reconstruction algorithm; then, based on parametric feature extraction technology, it converts the irregular mesh into a regular three-dimensional model; at the same time, it establishes an attribute database containing information such as material, connection method, and service life; finally, it integrates the geometric model and attribute information through associative mapping; improving the acquisition accuracy of geometric information and realizing the systematic integration of attribute data; 3. This application first simplifies the mesh and optimizes the geometry of the BIM model, reducing the data volume while maintaining key features; then improves the display effect through texture mapping and material optimization; next, it sets AR marker points to establish a unified spatial reference system; then calculates the positioning parameters adapted to the AR device based on the spatial reference system; finally, it integrates the positioning parameters with the lightweight model; reducing the data processing load and improving the smoothness of AR display. Description of the Drawings

[0022] Figure 1 is a schematic flow chart of a pipeline deformation monitoring method based on BIM and AR according to an embodiment of the present application; Figure 2 is a schematic flow chart of step S110 in a pipeline deformation monitoring method based on BIM and AR according to an embodiment of the present application; Figure 3 is a schematic flow chart of step S120 in a pipeline deformation monitoring method based on BIM and AR according to an embodiment of the present application; Figure 4 is a schematic flow chart of step S130 in a pipeline deformation monitoring method based on BIM and AR according to an embodiment of the present application; Figure 5 is a schematic flow chart of step S140 in a pipeline deformation monitoring method based on BIM and AR according to an embodiment of the present application; Figure 6 is a schematic flow chart of step S150 in a pipeline deformation monitoring method based on BIM and AR according to an embodiment of the present application; Figure 7 is a schematic flow chart of split-screen display in a pipeline deformation monitoring method based on BIM and AR according to an embodiment of the present application; Figure 8It is a schematic diagram of the modules of a pipeline deformation monitoring system based on BIM and AR according to an embodiment of the present application; Figure 9 It is an internal structure diagram of an electronic device according to an embodiment of the present application. Specific implementation manners

[0023] The terms used in the following embodiments of the present application are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in the specification and appended claims of the present application, the singular forms "a", "an", "the", "above-mentioned", "said", and "this" are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used in the present application refers to any or all possible combinations including one or more of the listed items.

[0024] Hereinafter, the terms "first" and "second" are only used for descriptive purposes and cannot be construed as implying or suggesting relative importance or implicitly indicating the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include one or more of such features. In the description of the embodiments of the present application, unless otherwise specified, the meaning of "a plurality" is two or more.

[0025] The following further describes the embodiments of the present application in conjunction with the accompanying drawings of the specification.

[0026] In a first aspect, the present application provides a pipeline deformation monitoring method based on BIM and AR. Refer to Figure 1 , and includes the following steps: S110. Obtain the geometric information and attribute information of the pipeline, and establish a BIM model of the pipeline.

[0027] In this embodiment, the geometric information of the pipeline includes basic geometric parameters such as the size, burial depth, and spatial position of the pipeline, and the attribute information includes characteristic data such as the material, connection method, and service life of the pipeline.

[0028] Specifically, a laser scanner is used to perform on-site measurement on the pipeline to obtain the spatial point cloud data of the pipeline; the geometric dimensions and burial information of the pipeline are extracted by using the existing design drawings and construction data; the attribute information such as the material parameters and usage status of the pipeline is obtained through on-site investigation and file query. For example, for the urban underground drainage network in a certain city, a three-dimensional laser scanner can be used to scan at multiple inspection well positions to obtain the actual size and spatial position data of the pipeline, and a complete BIM model can be established in combination with the attribute information such as the pipeline material being reinforced concrete, the connection method being socket joint, and the service life being 15 years.

[0029] S120. Generate a lightweight model for AR display according to the BIM model.

[0030] In this embodiment, in order to improve the real-time performance and smoothness of AR display, it is necessary to optimize and simplify the original BIM model to generate a lightweight model suitable for AR device display. The lightweight processing mainly includes links such as model simplification, texture optimization, and positioning parameter setting. On the premise of ensuring the basic geometric features and key attribute information, the data volume and calculation load of the model are reduced.

[0031] Specifically, a mesh simplification algorithm is used to perform polygon reduction and geometric optimization on the BIM model to remove unnecessary detail features; through texture mapping and material optimization, the display effect of the model is improved; appropriate AR marker points and spatial reference systems are set to ensure that the model can be accurately positioned and displayed. For example, import the BIM model of a certain section of pipeline into the optimization software, simplify the original 500,000 patches to 50,000 patches, and compress the texture on the pipeline surface, and finally generate a lightweight model with a 90% reduction in data volume.

[0032] S130. Based on the lightweight model, use an AR device to scan and image the actual pipeline to obtain real-time image data of the pipeline.

[0033] In this embodiment, the AR device performs real-time scanning and imaging on the actual pipeline through various integrated sensors and camera modules to obtain image data including spatial position, geometric contour, and surface features.

[0034] Specifically, use an AR glasses or tablet device equipped with a depth camera to perform 360-degree scanning and collection around the pipeline; determine the scanning position through the built-in positioning system of the device, and use image processing algorithms to extract the contour features of the pipeline. For example, use a certain AR smart glasses to scan the underground drainage pipeline at the inspection well to obtain the depth image and surface features of the pipeline inner wall.

[0035] S140. According to the real-time image data, register and fuse the BIM model with the AR real scene to obtain the virtual-real overlay display effect of the pipeline.

[0036] In this embodiment, through feature matching and spatial registration algorithms, the lightweight BIM model is fused and overlaid with the AR real scene image to achieve the precise alignment of the virtual model and the actual pipeline.

[0037] Specifically, extract the feature points and contour lines in the real scene image, perform feature matching and position alignment with the BIM model; through real-time tracking and pose estimation, maintain the dynamic alignment of the virtual model and the actual pipeline. For example, in the AR display interface, it can be seen that the blue semi-transparent BIM model perfectly coincides with the actual pipeline, and when deformation occurs, the deviation between the two can be clearly observed.

[0038] S150. Calculate the actual deformation parameters of the pipeline based on the virtual-real superposition display effect, and determine the deformation state of the pipeline.

[0039] In this embodiment, based on the virtual-real superposition effect, the actual deformation parameters of the pipeline are calculated, including indicators such as the cross-sectional deformation amount, deformation area, and deformation rate. By comparing with the preset threshold values, the deformation state of the pipeline can be divided into three levels: normal, warning, and dangerous.

[0040] Specifically, a transverse cross-section is generated at the deformation position, and the deviation between the actual contour and the standard contour is calculated; the deformation area and deformation rate are calculated by performing surface fitting on the deformation region. For example, it is detected that a certain pipe section has an elliptical deformation. Through calculation, the maximum cross-sectional deformation amount is 30 mm, the deformation area is 0.5 square meters, and the deformation rate is 5%. According to the evaluation standard, it is determined to be at the warning level.

[0041] S160. Generate a pipeline maintenance decision-making plan according to the deformation state.

[0042] Specifically, different treatment plans are selected according to the degree of deformation, such as strengthening monitoring, local repair, or overall replacement, etc.; a detailed implementation plan and emergency plan are formulated.

[0043] In one embodiment, the geometric information includes the size, burial depth, and spatial position of the pipeline; the attribute information includes the material, connection method, and service life of the pipeline; referring to Figure 2 , in step S110, the geometric information and attribute information of the pipeline are obtained, and a BIM model of the pipeline is established, which specifically includes the following steps: S111. Perform multi-site scanning on the pipeline, and obtain a point cloud model according to the size, burial depth, and spatial position of the pipeline.

[0044] In this embodiment, multi-site scanning refers to performing three-dimensional laser scanning at multiple characteristic positions along the pipeline. By reasonably arranging the scanning sites, it is ensured to obtain the complete spatial information of the pipeline. The scanning data contains key geometric information such as the size parameters, spatial coordinates, and burial depth of the pipeline, and these information constitute the basic data for pipeline modeling.

[0045] Specifically, scanning sites are set at the inspection wells, corners, and key interface positions of the pipeline, and a three-dimensional laser scanner is used for omnidirectional scanning. For example, for a 200-meter-long sewage pipeline, a scanning site is set every 20 meters, a scanner with a resolution of 2 mm is used for data acquisition, the scanning time for a single site is about 15 minutes, and finally a point cloud data set containing more than 10 million spatial points is obtained.

[0046] S112. Generate a three-dimensional mesh model of the pipeline based on the point cloud model to obtain the geometric surface of the pipeline.

[0047] In this embodiment, the point cloud model contains a large number of discrete spatial point sets, and it is necessary to convert it into a continuous mesh model through a three-dimensional reconstruction algorithm.

[0048] Specifically, point cloud filtering and mesh reconstruction algorithms are used to process the original point cloud data into a regular triangular mesh model. For example, first, noise reduction and thinning processing are performed on the point cloud to optimize the point density; then triangular meshes are generated to obtain a smooth and continuous pipeline surface model.

[0049] S113. Set parameterized features according to the geometric surface of the pipeline to obtain a regular three-dimensional model of the pipeline.

[0050] In this embodiment, although the mesh model can represent the geometric shape of the pipeline, it is not convenient for parametric modification and feature extraction. By setting parameterized features, the irregular mesh model can be converted into a regular model with clear geometric constraints.

[0051] Specifically, identify the basic geometric features of the pipeline, such as cylindrical surfaces, planes, and transition surfaces, etc., and establish a parametric feature model. For example, for a circular pipeline with a diameter of 600 mm, the center line and cross-section parameters of the pipeline are determined by cylindricity fitting, the wall thickness is set, and an editable parametric model is generated.

[0052] S114. Establish an attribute database according to the material, connection method, and service life of the pipeline.

[0053] In this embodiment, the attribute database contains non-geometric information of the pipeline, such as important features like material type, interface form, installation date, etc.

[0054] Specifically, design a structured data table to record various attribute parameters and related descriptions of the pipeline. For example, establish a relational database including a material attribute table, a connection feature table, and a time information table. The material attribute table records information such as the material of the pipeline being ductile iron, the strength grade being T2, and the anti-corrosion grade being П. The connection feature table records data such as the model and sealing method of flange connection.

[0055] S115. Perform an association mapping between the regular three-dimensional model and the attribute database, and finally generate the BIM model of the pipeline.

[0056] In this embodiment, the association mapping is a process of establishing a corresponding relationship between the regular geometric model and the attribute database. By establishing the mapping relationship, unified management of geometric information and attribute information can be realized, and a complete BIM model can be formed.

[0057] Specifically, a unique identification code can be assigned to each component of the model to establish a one-to-one correspondence between the geometric entity and the attribute record in the database.

[0058] In one embodiment, referring to Figure 3 , in step S120, generating a lightweight model for AR display according to the BIM model, specifically including the following steps: S121. Import the BIM model into the processing software, simplify the mesh and optimize the geometry of the model to obtain a simplified model.

[0059] S122. Perform texture mapping and material optimization based on the simplified model to generate a lightweight display model.

[0060] S123. Set AR marker points according to the lightweight display model and establish a spatial reference system.

[0061] In this embodiment, the AR marker points are key reference positions for aligning the virtual model with the actual scene. By setting distinct marker points on the model, a unified spatial reference system is established.

[0062] Specifically, set AR marker points at the characteristic positions of the pipeline (such as flanges, valves, inspection openings, etc.) and define local coordinate systems. For example, set a marker point every 5 meters on the pipeline. Each marker point contains a unique feature code and three-dimensional coordinate information, and uses a black-and-white circular marker pattern with a marker size of 10 cm × 10 cm to ensure that the AR device can quickly and accurately identify and locate.

[0063] S124. Perform model coordinate transformation based on the spatial reference system to obtain positioning parameters adapted to the AR device.

[0064] In this embodiment, the coordinate transformation is to convert the original coordinate system of the model into a spatial reference system suitable for the AR device, including transformation operations such as translation, rotation, and scaling.

[0065] Specifically, establish the conversion relationship between the world coordinate system, the model coordinate system, and the device coordinate system, and calculate the corresponding transformation matrix. For example, set the local coordinate origin of the pipeline model at the starting inspection well position, establish a right-handed coordinate system, and realize the corresponding relationship with the AR device coordinate system through a 4×4 transformation matrix to support the accurate positioning and attitude adjustment of the model in the AR environment.

[0066] S125. Integrate the positioning parameters with the lightweight display model to finally generate a lightweight model for AR display.

[0067] In one embodiment, the AR device includes a depth sensor, an inertial measurement unit, and a GNSS receiver; referring to Figure 4 , in step S130, use the AR device to scan and image the actual pipeline to obtain real-time image data of the pipeline, specifically including the following steps: S131. Combine the pipeline spatial data in the GIS system and the GNSS signals to determine the absolute position of the AR device.

[0068] In this embodiment, by integrating the GIS pipeline data and the GNSS positioning information, the accurate position of the AR device in the global coordinate system can be determined. The GIS system provides the spatial distribution information of the pipeline network, while the GNSS signals provide real-time position references. The combination of the two can achieve precise positioning of the device.

[0069] Specifically, the RTK differential positioning technology is adopted to improve the GNSS positioning accuracy and perform matching calibration with the GIS pipeline data. For example, a GNSS receiver that supports both Beidou and GPS systems can achieve centimeter-level positioning accuracy in areas covered by the reference station; at the same time, the pipeline GIS data is loaded, and the device position is compared with the known pipeline nodes to finally determine the spatial position of the device in the pipeline coordinate system.

[0070] S132. Based on the depth sensor, obtain the environmental feature point cloud, construct a local environmental map through SLAM technology, and determine the relative position.

[0071] In this embodiment, the depth sensor obtains the three-dimensional information of the environment through active projection structured light or TOF technology, and combines the SLAM (Simultaneous Localization and Mapping) algorithm to achieve relative positioning of the device. This method can obtain reliable position information in the underground space with weak GNSS signals.

[0072] Specifically, use an AR device equipped with a structured light depth camera to collect the depth images of the surrounding environment, extract feature points, and establish a local map. Through feature point matching and pose estimation, calculate the six-degree-of-freedom position parameters of the device relative to the local environment.

[0073] S133. Start the image acquisition module of the AR device to obtain the real-time video stream of the pipeline.

[0074] S134. Associate the absolute position and the relative position with the real-time video stream to obtain the pipeline contour features.

[0075] In this embodiment, by performing spatio-temporal alignment of the device position information and the video stream data, the spatial position relationship of the pipeline contour in the video can be determined.

[0076] Specifically, use the attitude data provided by the inertial measurement unit (IMU) to perform real-time correction and spatial registration of the video images. For example, when the device moves, compensate for the image jitter caused by the movement through the IMU data, and at the same time, according to the known position information, extract the pipeline contour in the video and establish the mapping relationship between the contour features and the spatial coordinates.

[0077] S135. Perform real-time tracking and positioning calculation on the pipeline contour features, and finally generate the real-time scan data of the pipeline.

[0078] In one embodiment, referring to Figure 5 , in step S140, according to the real-time image data, register and fuse the BIM model with the AR real scene to obtain the virtual-real superposition display effect of the pipeline, which specifically includes the following steps: S141. Obtain the pipeline feature points and contour lines in the real-time scan data, and establish a three-dimensional feature data set.

[0079] In this embodiment, the three-dimensional feature data set is the key feature information of the pipeline extracted based on the real-time scan data, including feature points with accurate spatial positions and contour lines reflecting the pipeline shape.

[0080] Specifically, use edge detection and feature extraction algorithms to identify the significant features of the pipeline from the scan data. For example, use the Canny operator to extract the pipeline edge, select the positions with significant curvature changes as feature points, and select 10 - 15 feature points per meter of the pipeline; at the same time, obtain the continuous pipeline contour line through B-spline curve fitting, and store the three-dimensional coordinate information of the feature points and contour lines as a structured data set.

[0081] S142. Perform feature matching between the feature data set and the lightweight model, and calculate the position deviation parameter.

[0082] In this embodiment, feature matching is a process of pairing and comparing the features obtained from the actual scan with the corresponding positions in the lightweight model. By calculating the position deviation between the features, the spatial relationship between the actual pipeline and the virtual model can be determined.

[0083] Specifically, use the matching algorithm based on ICP (Iterative Closest Point) to calculate the corresponding relationship between the feature data set and the lightweight model. For example, first perform rough matching, select the closest feature point pairs to establish the initial corresponding relationship; then through iterative optimization, gradually adjust the position parameters until the average distance error between the feature point pairs is less than 5 millimeters, and finally obtain the accurate position deviation parameter.

[0084] S143. Perform coordinate transformation and spatial alignment according to the position deviation parameter.

[0085] In this embodiment, spatial alignment is to adjust the virtual model according to the position deviation parameter to make it exactly coincide with the actual pipeline in spatial position.

[0086] Specifically, a rigid body transformation matrix is established to adjust the lightweight model in space. For example, according to the calculated position deviation, a 4×4 homogeneous transformation matrix is constructed, which includes a translation vector and a rotation matrix; the virtual model is adjusted to the correct position through matrix transformation to achieve precise alignment with the actual pipeline.

[0087] S144. Based on the spatial alignment result, generate an overlay display of the virtual model and the real scene.

[0088] In this embodiment, the overlay display is to synthesize the virtual model after spatial alignment with the real scene image to generate a mixed reality scene containing virtual and real information.

[0089] Specifically, real-time rendering technology is adopted to overlay the virtual model onto the real scene video stream. For example, OpenGLES is used for 3D rendering. Considering the influence of ambient light on the virtual model, an appropriate transparency is set to make the virtual model blend naturally with the real scene; at the same time, occlusion relationships are processed to ensure that the virtual model can be correctly displayed in front of and behind real scene objects.

[0090] S145. Continuously update the position parameters of the overlay display to obtain an overlay display effect of virtual and real that includes the geometric profile, material characteristics, and position information of the pipeline.

[0091] In one embodiment, the deformation parameters include the cross-sectional deformation amount, deformation area, and deformation rate of the pipeline; the deformation states include three levels: normal, warning, and danger; referring to Figure 6 , in step S150, based on the virtual-real overlay display effect, calculate the actual deformation parameters of the pipeline and determine the deformation state of the pipeline, which specifically includes the following steps: S151. Extract the actual contour data of the pipeline from the virtual-real overlay display effect.

[0092] S152. Generate a transverse section of the pipeline at the deformation position and calculate the cross-sectional deformation amount between the actual section and the standard section of the BIM model.

[0093] In this embodiment, the calculation of the cross-sectional deformation amount requires establishing a transverse section at the deformation position and comparing the actual section with the standard section in the BIM model. This comparison can reflect the deformation degree of the pipeline at the local position.

[0094] Specifically, a transverse section orthogonal to the pipeline centerline is automatically generated at the deformation. For example, select the most obvious deformation point, take a plane perpendicular to the pipeline axis, and extract the intersection line of the actual contour and this plane as the actual section; at the same time, extract the standard section at the same position in the BIM model, and calculate the maximum deviation value between the two sections through the least squares method to obtain the cross-sectional deformation amount.

[0095] S153. Perform surface fitting on the deformed part, compare it with the standard surface of the BIM model, and calculate the deformation area.

[0096] In this embodiment, the calculation of the deformed area requires three-dimensional surface reconstruction of the deformed part, and the range and area of the deformed area are determined by comparing with the standard surface.

[0097] Specifically, the NURBS surface is used to fit and model the deformed area. For example, the point cloud data within 1 meter around the deformed area is selected, and the 3rd-order NURBS surface is used for fitting, with the number of control points being 20×20; the fitted surface is compared with the standard surface of the BIM model, and the area of the region exceeding the tolerance range is calculated to obtain the deformed area value.

[0098] S154. Calculate the deformation rate of the pipeline based on the cross-sectional deformation amount and the deformed area.

[0099] Among them, the deformation rate is a comprehensive index characterizing the overall deformation degree of the pipeline, and the influence of the cross-sectional deformation amount and the deformed area needs to be comprehensively considered.

[0100] S155. Perform hierarchical evaluation according to the preset deformation parameter threshold to determine the deformation state level of the pipeline.

[0101] In this embodiment, the deformation state classification is to evaluate and classify the safety status of the pipeline based on the calculated deformation parameters. Through the preset threshold interval, the deformation state of the pipeline can be divided into different warning levels.

[0102] Specifically, a scientific and reasonable classification standard is set for status evaluation. For example, when the deformation rate γ < 3%, it is determined to be in the normal state and a green identifier is displayed; when 3% ≤ γ < 5%, it is determined to be in the warning state and a yellow identifier is displayed; when γ ≥ 5%, it is determined to be in the dangerous state, a red identifier is displayed and an alarm prompt is triggered.

[0103] In one embodiment, referring to Figure 7 , the method further includes the following steps: S710. Obtain the pipeline BIM model and real-time scan data in the virtual-real superposition display effect.

[0104] S720. Establish a split-screen display interface in the AR display to simultaneously display the pipeline BIM model and the real-time scene.

[0105] Among them, the establishment of the split-screen display interface requires reasonable planning of the display area to ensure that the BIM model and the real-time scene can be clearly compared and displayed.

[0106] Specifically, design a split-screen display scheme with two windows, divide the screen of the AR display into two equal-width display areas on the left and right, display the three-dimensional rendering effect of the BIM model on the left, supporting zooming and rotating operations; display the on-site image collected in real time on the right, superimposing the real-time detection data, and the two windows maintain the same viewing angle.

[0107] S730. Color-code and numerically label the deformed part according to the deformation status level.

[0108] Among them, the labeling display of the deformed part needs to select an appropriate visualization scheme according to the deformation status level. Through the combined labeling of color and numerical value, the position and degree of pipeline deformation can be intuitively reflected.

[0109] Specifically, a hierarchical color matching scheme and dynamic labeling technology are adopted. For example, the normal state is displayed in green, the warning state is displayed in yellow, and the dangerous state is displayed in red; a semi-transparent color mask is added to the deformed part, and at the same time, the numerical label of the deformation parameter is displayed, and the label size is adaptively adjusted according to the observation distance.

[0110] S740. Real-time display the cross-sectional deformation amount, deformation area, and deformation rate data of the deformed part.

[0111] It should be understood that the magnitudes of the sequence numbers of the steps in the above embodiments do not mean the order of execution. The execution order of each process should be determined according to its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.

[0112] In a second aspect, the present application provides a pipeline deformation monitoring system based on BIM and AR. Below, in combination with the above pipeline deformation monitoring method based on BIM and AR, the pipeline deformation monitoring system based on BIM and AR of the present application will be described.

[0113] Refer to Figure 8 , a pipeline deformation monitoring system based on BIM and AR, including: A BIM model establishment module, configured to obtain the geometric information and attribute information of the pipeline and establish a BIM model of the pipeline; A lightweight model generation module, configured to generate a lightweight model for AR display according to the BIM model; A real-time image data acquisition module, configured to perform scanning imaging on the actual pipeline by using an AR device based on the lightweight model to obtain real-time image data of the pipeline; An overlay display effect acquisition module, configured to register and fuse the BIM model with the AR real scene according to the real-time image data to obtain a virtual-real overlay display effect of the pipeline; A deformation status determination module, configured to calculate the actual deformation parameters of the pipeline and determine the deformation status of the pipeline based on the virtual-real overlay display effect; A maintenance decision-making plan generation module, configured to generate a pipeline maintenance decision-making plan according to the deformation status.

[0114] In one embodiment, the BIM model establishment module includes: A point cloud scanning unit for performing multi-site scanning on a pipeline and obtaining a point cloud model according to the size, burial depth, and spatial position of the pipeline; A grid model generation unit for generating a three-dimensional grid model of the pipeline based on the point cloud model to obtain the geometric surface of the pipeline; A regularized model generation unit for setting parametric features according to the geometric surface of the pipeline and obtaining a regularized three-dimensional model of the pipeline; An attribute database establishment unit for establishing an attribute database according to the material, connection method, and service life of the pipeline; An association mapping unit for associating and mapping the regularized three-dimensional model with the attribute database to finally generate a BIM model of the pipeline.

[0115] In one embodiment, the lightweight model generation module includes: A model simplification unit for importing the BIM model into processing software, performing mesh simplification and geometric optimization on the model to obtain a simplified model; A lightweight display unit for performing texture mapping and material optimization based on the simplified model to generate a lightweight display model; An AR marker setting unit for setting AR marker points according to the lightweight display model to establish a spatial reference system; A coordinate transformation unit for performing model coordinate transformation based on the spatial reference system to obtain positioning parameters adapted to the AR device; A model integration unit for integrating the positioning parameters with the lightweight display model to finally generate a lightweight model for AR display.

[0116] In one embodiment, the real-time image data acquisition module includes: An absolute positioning unit for determining the absolute position of the AR device by combining the pipe network spatial data in the GIS system and the GNSS signal; A relative positioning unit for obtaining environmental feature point clouds based on a depth sensor, constructing a local environmental map through SLAM technology, and determining the relative position; A video acquisition unit for starting the image acquisition module of the AR device to obtain a real-time video stream of the pipeline; A position association unit for associating the absolute position and the relative position with the real-time video stream to obtain pipeline contour features; A tracking and positioning unit for performing real-time tracking and positioning calculation on the pipeline contour features to finally generate real-time scan data of the pipeline.

[0117] In one embodiment, the superimposed display effect acquisition module includes: A feature extraction unit for obtaining pipeline feature points and contour lines in the real-time scan data and establishing a three-dimensional feature data set; A feature matching unit, configured to perform feature matching between a feature data set and a lightweight model, and calculate a position deviation parameter; A spatial alignment unit, configured to perform coordinate transformation and spatial alignment according to the position deviation parameter; A superimposition generation unit, configured to generate a superimposed display of the virtual model and the real scene based on the spatial alignment result; A real-time update unit, configured to real-time update the position parameter of the superimposed display, and obtain a virtual-real superimposed display effect including the pipeline geometric profile, material features, and position information.

[0118] In one embodiment, the deformation state determination module includes: A contour extraction unit, configured to extract the actual contour data of the pipeline in the virtual-real superimposed display effect; A cross-section calculation unit, configured to generate a transverse cross-section of the pipeline at the deformation position, and calculate the cross-section deformation amount between the actual cross-section and the standard cross-section of the BIM model; An area calculation unit, configured to perform surface fitting on the deformed part, compare it with the standard surface of the BIM model, and calculate the deformation area; A deformation rate calculation unit, configured to calculate the deformation rate of the pipeline based on the cross-section deformation amount and the deformation area; A state evaluation unit, configured to perform hierarchical evaluation according to a preset deformation parameter threshold, and determine the deformation state level of the pipeline.

[0119] In one embodiment, it further includes a visualization display module, and the visualization display module includes: A data acquisition unit, configured to acquire the pipeline BIM model and real-time scan data in the virtual-real superimposed display effect; A split-screen display unit, configured to establish a split-screen display interface in the AR display, and simultaneously display the pipeline BIM model and the real-time scene; A marking display unit, configured to perform color marking and numerical marking on the deformed part according to the deformation state level; A parameter display unit, configured to real-time display the cross-section deformation amount, deformation area, and deformation rate data of the deformed part.

[0120] In one embodiment, the present application provides an electronic device, and the electronic device may be a server, and its internal structure diagram may be as Figure 9As shown in the figure. The electronic device includes a processor, a memory, and a network interface connected through a system bus. Among them, the processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the electronic device is used to store data. The network interface of the electronic device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, it implements a method for pipeline deformation monitoring based on BIM and AR.

[0121] Those skilled in the art can understand that Figure 9 the structure shown in the figure is only a block diagram of some structures related to the solution of this application, and does not constitute a limitation on the electronic device to which the solution of this application is applied. The specific electronic device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0122] In one embodiment, an electronic device is further provided, including a memory and a processor. A computer program is stored in the memory. When the processor executes the computer program, the steps in the above method embodiments are implemented.

[0123] Those of ordinary skill in the art can understand that all or part of the processes of implementing the above method embodiments can be completed by instructing relevant hardware through a computer program. The above computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the above method embodiments. Among them, any reference to the memory, storage, database, or other media used in the various embodiments provided in this application can include at least one of non-volatile and volatile memories. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical memory, etc. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0124] The above are all preferred embodiments of this application. The protection scope of this application is not limited accordingly. Therefore, all equivalent changes made according to the structure, shape, and principle of this application should be covered within the protection scope of this application.

Claims

1. A pipeline deformation monitoring method based on BIM and AR, characterized in that It includes the following steps: Obtain the geometric information and attribute information of the pipeline, and establish a BIM model of the pipeline; Generate a lightweight model for AR display according to the BIM model; Based on the lightweight model, use an AR device to scan and image the actual pipeline to obtain real-time image data of the pipeline; According to the real-time image data, register and fuse the BIM model with the AR real scene to obtain a virtual-real superposition display effect of the pipeline; Based on the virtual-real superposition display effect, calculate the actual deformation parameters of the pipeline and determine the deformation state of the pipeline; Generate a pipeline maintenance decision-making plan according to the deformation state; 2. The method for monitoring pipeline deformation based on BIM and AR according to claim 1, wherein The geometric information includes the size, burial depth and spatial position of the pipeline; the attribute information includes the material, connection method and service life of the pipeline; obtaining the geometric information and attribute information of the pipeline and establishing a BIM model of the pipeline specifically includes the following steps: Perform multi-site scanning on the pipeline, and obtain a point cloud model according to the size, burial depth and spatial position of the pipeline; Generate a 3D mesh model of the pipeline based on the point cloud model to obtain the geometric surface of the pipeline; Set parametric features according to the geometric surface of the pipeline to obtain a regularized 3D model of the pipeline; Establish an attribute database according to the material, connection method and service life of the pipeline; Perform associated mapping between the regularized 3D model and the attribute database, and finally generate a BIM model of the pipeline; 3. The method for monitoring pipeline deformation based on BIM and AR according to claim 2, wherein, Generating a lightweight model for AR display according to the BIM model specifically includes the following steps: Import the BIM model into processing software, simplify the mesh and optimize the geometry of the model to obtain a simplified model; Perform texture mapping and material optimization based on the simplified model to generate a lightweight display model; Set AR marker points according to the lightweight display model to establish a spatial reference system; Perform model coordinate transformation based on the spatial reference system to obtain positioning parameters adapted to the AR device; Integrate the positioning parameters with the lightweight display model, and finally generate a lightweight model for AR display; 4. The pipeline deformation monitoring method based on BIM and AR according to claim 1, wherein, The AR device includes a depth sensor, an inertial measurement unit and a GNSS receiver; using the AR device to scan and image the actual pipeline to obtain real-time image data of the pipeline specifically includes the following steps: Combine the pipeline network spatial data and GNSS signals in the GIS system to determine the absolute position of the AR device; Obtain environmental feature point clouds based on the depth sensor, construct a local environmental map through SLAM technology, and determine the relative position; Start the image acquisition module of the AR device to obtain a real-time video stream of the pipeline; Associate the absolute position and relative position with the real-time video stream to obtain the pipeline contour features; 5. The pipeline deformation monitoring method based on BIM and AR according to claim 4, wherein, Perform real-time tracking and positioning calculation on the pipeline contour features, and finally generate real-time scan data of the pipeline; According to the real-time image data, register and fuse the BIM model with the AR real scene to obtain a virtual-real superposition display effect of the pipeline, specifically including the following steps: Obtain the pipeline feature points and contour lines in the real-time scan data, and establish a 3D feature data set; Perform feature matching between the feature data set and the lightweight model, and calculate the position deviation parameters; Perform coordinate transformation and spatial alignment according to the position deviation parameters; Generate the superimposed display of the virtual model and the real scene based on the spatial alignment result; Update the position parameters of the superimposed display in real time to obtain the virtual-real superimposed display effect including the pipeline geometric profile, material characteristics, and position information.

6. The method for monitoring pipeline deformation based on BIM and AR according to claim 1, characterized in that, The deformation parameters include the cross-sectional deformation amount, deformation area, and deformation rate of the pipeline; the deformation states include three levels: normal, warning, and danger; based on the virtual-real superimposed display effect, calculate the actual deformation parameters of the pipeline and determine the deformation state of the pipeline, which specifically includes the following steps: Extract the actual contour data of the pipeline in the virtual-real superimposed display effect; Generate a transverse section of the pipeline at the deformation position and calculate the cross-sectional deformation amount between the actual section and the standard section of the BIM model; Perform surface fitting on the deformed part, compare it with the standard surface of the BIM model, and calculate the deformation area; Calculate the deformation rate of the pipeline based on the cross-sectional deformation amount and the deformation area; Conduct hierarchical evaluation according to the preset deformation parameter threshold to determine the deformation state level of the pipeline.

7. The pipeline deformation monitoring method based on BIM and AR according to claim 6, wherein, The method further includes the following steps: Obtain the pipeline BIM model and real-time scan data in the virtual-real superimposed display effect; Establish a split-screen display interface in the AR display to simultaneously display the pipeline BIM model and the real-time scene; Perform color marking and numerical marking on the deformed part according to the deformation state level; Real-time display the cross-sectional deformation amount, deformation area, and deformation rate data of the deformed part.

8. A pipeline deformation monitoring system based on BIM and AR, characterized in that, Include: The BIM model establishment module is used to obtain the geometric information and attribute information of the pipeline and establish the BIM model of the pipeline; The lightweight model generation module is used to generate a lightweight model for AR display according to the BIM model; The real-time image data acquisition module is used to scan and image the actual pipeline using an AR device based on the lightweight model to obtain the real-time image data of the pipeline; The superimposed display effect acquisition module is used to register and fuse the BIM model with the AR real scene according to the real-time image data to obtain the virtual-real superimposed display effect of the pipeline; The deformation state determination module is used to calculate the actual deformation parameters of the pipeline and determine the deformation state of the pipeline based on the virtual-real superimposed display effect; The maintenance decision plan generation module is used to generate a pipeline maintenance decision plan according to the deformation state.

9. An electronic device, characterized in that, It includes a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the BIM and AR-based pipeline deformation monitoring method according to any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the BIM and AR-based pipeline deformation monitoring method according to any one of claims 1-7.

Images