A method and system for inspecting underground pipelines based on WEB AR technology

Abstract

This invention discloses a method and system for underground pipeline inspection based on Web AR technology, belonging to the field of underground pipeline network inspection. The method includes: uploading pipeline network information and obtaining an AR service page; calculating camera pose by fusing GPS and IMU information, requesting nearby pipeline attribute information based on the camera's GPS location, and modeling based on the attribute information; obtaining real-time video stream information using Web RTC; and rendering model information using Web GL, achieving virtual-real fusion. On the web interface, users can interact by clicking to view the current AR model information and upload maintenance information by submitting a form. The system includes: a server module, a file acquisition module, a pose calculation module, a 3D modeling module, an AR rendering module, and an AR interaction module. This application, by applying Web AR, provides excellent cross-platform compatibility, low learning costs, short development cycles, and requires no application installation, making it very convenient to use.

Description

Technical Field

[0001] This invention relates to the field of augmented reality, and more particularly to a method and system for inspecting underground pipelines based on WEB AR technology. Background Technology

[0002] Mobile augmented reality (AR) is a technology that combines virtual objects and information with the real world. This technology typically uses mobile devices, such as smartphones or tablets, to achieve augmented reality. Mobile AR can be implemented through software applications or browser plugins.

[0003] Currently, most pipeline AR visualization methods primarily rely on mobile applications. Developing mobile applications requires specialized software development kits (SDKs), such as Vuforia, AR Core, and AR Kit, and these applications can only run on their respective platforms (Android or iOS). Mobile applications lack cross-platform capabilities and are limited by SSDs. Therefore, mobile augmented reality applications face challenges in large-scale deployment and promotion. Web AR (Web-based Augmented Reality) is a web-based augmented reality technology. It uses a browser as a platform and leverages web technologies such as HTML, CSS, and JavaScript to implement augmented reality applications. The advantage of Web AR technology is that it does not require additional SSDs or plugins; users only need to access a webpage through a browser to use augmented reality applications. Web AR applications are easy to use, require no deployment, and have good cross-platform compatibility. Furthermore, compared to developing SSD-based AR mobile applications, Web AR applications have a lower learning curve and shorter development time. Summary of the Invention

[0004] The purpose of this invention is to address the shortcomings of existing technologies by proposing a method and system for inspecting underground pipelines based on WEB AR technology.

[0005] The objective of this invention is achieved through the following technical solution: a method for inspecting underground pipelines based on WEB AR technology, the method comprising:

[0006] S1. Input the pipeline network data into the database, use the web terminal to access the AR service page on the remote server, and obtain the AR support file to support the Web AR service.

[0007] S2. The web terminal calls the browser's positioning API to obtain positioning information and the device sensors to obtain IMU information, calculate the current camera pose information, and continuously update the camera pose information by periodically obtaining positioning information and IMU information.

[0008] S3: The Web terminal requests pipeline attribute data near the camera from the remote server based on the camera's current location, determines the pipeline's pose information, and performs 3D modeling based on the pipeline's basic attribute information.

[0009] S4. The Web terminal obtains real-time video streams through Web RTC and renders 3D models through Web GL to achieve virtual-real fusion and obtain the pipeline AR model.

[0010] S5. Enter the pipeline network information that needs to be inspected into the database through the AR service page. The web terminal can view the pipeline network attribute data and historical maintenance records by clicking on the pipeline network AR model for subsequent maintenance.

[0011] Furthermore, the pipeline network data is obtained from exploration data and data maps, including basic and real-time information of the pipeline network; the basic information includes: location information, pipeline radius, pipeline type, pipeline material, pipeline inner diameter, pipeline name, and pipeline number;

[0012] Real-time information includes: whether the operation is normal or stopped.

[0013] Furthermore, the AR support files in S1 include: an AR pipeline inspection service page, an AR support script, and a rendering support JavaScript script; the acquired AR pipeline inspection service page and AR rendering-related JavaScript script are cached in a local web browser.

[0014] Furthermore, the specific steps for calculating the current camera pose information in S2 include:

[0015] S21. The camera's pose information uses spherical Mercator coordinates as the spatial coordinate system; the camera's positioning information is converted from GPS coordinates (latitude, longitude) obtained from the browser's positioning API into spatial coordinates (x, y, z) in the spherical Mercator coordinate system.

[0016] The projection formula for the spherical Mercator projection is as follows:

[0017]

[0018]

[0019] x=R×λ

[0020]

[0021] Where longitude is the longitude value in GPS coordinates, latitude is the latitude value in GPS coordinates, and λ is the longitude value in radians. y is the radian representation of the latitude value; R is the circumference of the reference sphere in the Mercator projection; the y value is the height information in the spatial coordinate system to simulate the height of the camera in a real scene.

[0022] S22. The camera's pose information is obtained from the IMU information acquired from the device's sensors. The Euler angles in the IMU information are used to determine the camera's pose.

[0023] S23. The spatial coordinates and pose information of the camera are fused to obtain the pose information of the camera.

[0024] Furthermore, S3 specifically includes:

[0025] S31. The server retrieves pipeline data and pose information from the database based on the current device location;

[0026] S32. The locations of each node in the pipeline network are converted from their original GPS coordinates to spatial coordinates; the conversion formula is the spherical Mercator projection formula.

[0027] S33. Model the pipeline network using AR support files based on pipeline network data and pose information.

[0028] Furthermore, S4 includes: obtaining real-time video stream information via Web RTC, converting the network coordinates from spatial Mercator coordinates to image coordinates, and performing 3D registration of the network AR model using JavaScript scripts through rendering.

[0029] Further, S5 includes:

[0030] The S50 and Web terminal provide AR model viewing of basic pipeline information, real-time information and maintenance records. Maintenance requests and maintenance type information can be submitted through the Web terminal for pipeline maintenance services. After submitting the information, a maintenance record is generated, including three parts: maintenance time, maintenance location and maintenance type.

[0031] S51. The Web terminal allows users to view basic pipeline information, real-time information, and maintenance records through an AR model via click-to-interact interaction.

[0032] S52, the Web terminal provides a method for uploading real-time information of the current pipeline network. Users can upload the pipeline network information that needs to be inspected to the remote server to assist in pipeline network maintenance.

[0033] According to another aspect of the specification, an underground pipeline inspection system based on Web AR technology is provided, the system comprising:

[0034] Server module: Used to store basic attribute information, real-time status information, inspection status information, and AR service-related files of pipeline facilities;

[0035] The file acquisition module is used by the web terminal to request AR service pages and AR service-related files from a remote server.

[0036] The pose calculation module is used to acquire camera position information data and pose information data, calculate and track camera pose information, and at the same time, the acquired video stream data is used for virtual-real fusion of the background.

[0037] The 3D modeling module obtains pipeline network-related information based on the positioning data and builds a 3D model of the pipeline network based on its attribute information.

[0038] The AR rendering module is used to render the pipeline model and blend the AR model with real-time images.

[0039] The beneficial effects of this invention are:

[0040] It primarily utilizes web AR technology to achieve AR inspection of underground pipelines. Compared to mobile applications limited by device platforms and related SDKs, it achieves cross-platform support, has a lower learning curve, and a shorter development cycle. It improves upon traditional methods by using spherical Mercator projection to calculate spatial coordinates, reducing computational costs per frame and enhancing the real-time performance of AR applications. Attached Figure Description

[0041] Figure 1 This is a flowchart of an underground pipeline inspection method based on WEB AR technology according to an embodiment of the present invention;

[0042] Figure 2 This is a structural diagram of an underground pipeline inspection system based on WEB AR technology according to an embodiment of the present invention;

[0043] Figure 3 This is a flowchart for pose calculation. Detailed Implementation

[0044] The specific embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0045] According to an embodiment of the present invention, a method for inspecting underground pipelines based on Web AR technology is provided.

[0046] like Figure 1 As shown, the underground pipeline inspection method based on Web AR technology according to an embodiment of the present invention includes:

[0047] S1. Input the pipeline network data into the database, use the web terminal to access the AR service page on the remote server, and obtain the AR support file to support the Web AR service.

[0048] S2. The web terminal calls the browser's positioning API to obtain positioning information and the device's sensors to obtain IMU information, calculates the current camera pose information, and continuously updates the camera pose information by periodically obtaining positioning information and IMU information; the web terminal requests the AR service page and AR support files from the remote server; the AR service page is written in HTML and is used to view the underground pipeline AR model in real time and interact with the AR model, and the AR support files are written in JavaScript and are used to provide AR service support. Neither of them needs to be opened by an application; they can be run by a web browser.

[0049] like Figure 3 As shown, the pose calculation process according to an embodiment of the present invention includes:

[0050] Coordinate transformation: Periodically acquire GPS coordinate data and convert it into spherical Mercator space coordinate system coordinate data;

[0051] Attitude acquisition: Attitude data is acquired periodically using IMU sensor data to perform spatial attitude calculation;

[0052] Data fusion: This involves fusing spatial coordinate data and device posture data to calculate the current device pose data;

[0053] Pose Update: Periodically update GPS coordinate data and IMU sensor data to calculate the current new device pose data.

[0054] The specific method for pose calculation is as follows:

[0055] S21. The camera's pose information uses spherical Mercator coordinates as the spatial coordinate system; the camera's position information is converted from GPS coordinates (latitude, longitude) obtained from the browser's positioning API into spatial coordinates (x, y, z) in the spherical Mercator coordinate system.

[0056] S22. The projection formula for the spherical Mercator projection is as follows:

[0057]

[0058]

[0059] x=R×λ

[0060]

[0061] Where longitude is the longitude value in GPS coordinates, latitude is the latitude value in GPS coordinates, and λ is the longitude value in radians. This is the latitude value expressed in radians. R is the circumference of the reference sphere in the Mercator projection. The value of R is 20037508.3427892 meters. The y-value represents the altitude in the spatial coordinate system, with a default value of 1.5 to simulate the camera's altitude in a real-world scene.

[0062] The reference sphere parameters of the spherical Mercator projection are relatively simple, and the calculation formula is concise. It consumes fewer computational resources than other projection methods while achieving a similar projection effect. This is beneficial for reducing computational overhead and improving program smoothness when updating camera pose in real time.

[0063] S23. The camera's pose information is obtained from IMU information acquired from the device's sensors. The Euler angles in the IMU information can be used to determine the camera's pose.

[0064] S24. By fusing the spatial coordinates and pose information of the camera obtained in the above two steps, the pose information of the camera can be obtained.

[0065] S3: The web terminal requests pipeline attribute data near the camera from a remote server based on the camera's current location to determine the pipeline's pose information. 3D modeling is then performed based on the pipeline's basic attribute information. After the user successfully requests the AR service page, the AR service page first calls the browser's positioning API to obtain GPS positioning information to determine the current device's location and the device's sensors to obtain IMU information to determine the device's pose information. After obtaining the GPS positioning and IMU pose information, the AR service page calls the pose calculation service in the AR support file to perform pose calculation. It calculates the camera's current spatial position information using spherical Mercator projection and calculates the camera's current pose information using Euler angles provided by the IMU information. The calculated spatial position and pose information are then fused to obtain the camera's pose information, supporting subsequent AR rendering operations. The camera's real-time pose is calculated by periodically listening to and obtaining GPS positioning and IMU information, continuously updating the camera's pose information. During real-time viewing, the device's pose information is calculated in this way.

[0066] While the user device calculates the camera pose information in real time, the AR service page simultaneously sends a request to the server based on the current GPS location information to obtain the pipeline network data near the current camera. The obtained pipeline network data is used to build a 3D model of the pipeline network in real time through AR support files, and the attribute information of the pipeline network is stored in the browser cache to provide pipeline network information in the AR interaction steps.

[0067] S4. The Web terminal obtains real-time video streams through Web RTC and renders 3D models through Web GL to achieve virtual-real fusion and obtain the pipeline AR model. Specifically, the user device obtains real-time video stream information from the camera through Web RTC, the 3D model established in the previous step is rendered through Web GL, and the rendered 3D model is fused with the real-time video stream information to obtain the AR model. Users can view the underground pipeline AR model in the nearby environment in real time.

[0068] S5. Enter the pipeline network information that needs to be inspected into the database through the AR service page. The web terminal can view the pipeline network attribute data and historical maintenance records by clicking on the pipeline network AR model for subsequent maintenance.

[0069] The AR support files include: pose calculation service for the camera, 3D modeling service for the network model, real-time video stream acquisition service, model rendering service, and virtual-real fusion service.

[0070] In addition, the method also includes: after obtaining positioning information from the positioning API and IMU information from the device sensors, when performing pose estimation, it is necessary to first convert the positioning coordinates from latitude and longitude coordinates to spherical Mercator coordinates, that is, spatial coordinates, to determine the true position of the camera, and calculate the camera pose through the IMU information obtained by the sensors.

[0071] Among them, the Web terminal requests pipeline attribute data near the camera from the remote server based on the camera's current location, including: obtaining pipeline facility attribute information within a fixed radius range based on the current coordinates and using it for real-time rendering to reduce the rendering pressure on the pipeline system;

[0072] The WebGL-based 3D model rendering process includes: calculating the relative pose of the pipeline model based on the camera pose, and then performing 3D rendering and virtual-real fusion.

[0073] The web terminal allows users to view basic information, real-time information, and historical maintenance records of the pipeline network by clicking on the AR model. The basic information includes the pipeline network's geometric attributes, materials, location, and name. The real-time information includes the current operating status of the facilities. The historical maintenance records contain previous maintenance information for reference. The current real-time status information of the pipeline network can be submitted to a remote server via a form for subsequent maintenance work.

[0074] According to an embodiment of the present invention, an underground pipeline inspection system based on Web AR technology is provided.

[0075] like Figure 2 As shown, the underground pipeline inspection method based on Web AR technology according to an embodiment of the present invention includes:

[0076] Server module 201 is used to store AR service files, AR rendering files, pipeline information data, pipeline maintenance information and other related files;

[0077] The file acquisition module 202 is used for the Web terminal to send an AR service request to the remote server, and the remote server to retrieve the AR service file, AR rendering file, etc. and send them to the Web terminal according to the AR service request.

[0078] The pose calculation module 203 is used to perform pose fusion estimation and calculate pose information based on the GPS coordinate information obtained in real time by the positioning API and the pose information obtained in real time by the device sensor IMU.

[0079] AR modeling module 204: After obtaining nearby pipeline information based on location information, the AR modeling module can perform real-time 3D modeling of the pipeline network for subsequent model rendering operations.

[0080] AR rendering module 205 renders the AR model using Web GL and performs virtual-real fusion operation with the real-time video stream obtained by Web RTC.

[0081] AR interaction module 206 supports interaction with the rendered AR model, used to view basic information, real-time information and maintenance records of the current pipeline network.

[0082] The principle and effects of the above technical solution: This WEB AR-based underground pipeline inspection method and system is implemented and deployed using both a server and a browser. Using this technology, the pipeline network model can be placed on its actual ground location, allowing for real-time viewing of information and submission of maintenance requests. Written in HTML and supported by JavaScript, the web-based AR service requires no application downloads or additional device restrictions. Users simply access the AR service page to access Web AR services, which is very convenient. During inspection, users can click on the AR model to view the basic information and operating status of underground pipelines in real time. If a fault is found in the current pipeline network, a maintenance request can be submitted in real-time through the AR service page, ensuring timeliness and efficiency.

[0083] The above embodiments are used to explain and illustrate the present invention, but not to limit the present invention. Any modifications and changes made to the present invention within the spirit and scope of the claims shall fall within the protection scope of the present invention.

Claims

1. A method for inspecting underground pipelines based on WEB AR technology, characterized in that, The method includes: S1. Input the pipeline network data into the database, use the web terminal to access the AR service page on the remote server, and obtain the AR support file to support the Web AR service. S2. The web terminal calls the browser's positioning API to obtain positioning information, and the device sensors obtain IMU information to calculate the current camera pose information. The camera pose information is continuously updated by periodically acquiring positioning information and IMU information. The specific steps in S2 for calculating the current camera pose information include: S21. The camera's pose information uses spherical Mercator coordinates as the spatial coordinate system; the camera's positioning information is converted from GPS coordinates (latitude, longitude) obtained from the browser's positioning API into spatial coordinates (x, y, z) in the spherical Mercator coordinate system. The projection formula for the spherical Mercator projection is as follows: Where longitude is the longitude value in GPS coordinates, latitude is the latitude value in GPS coordinates, and λ is the longitude value in radians. y is the radian representation of the latitude value; R is the circumference of the reference sphere in the Mercator projection; the y value is the height information in the spatial coordinate system to simulate the height of the camera in a real scene. S22. The camera's pose information is obtained from the IMU information acquired from the device's sensors. The Euler angles in the IMU information are used to determine the camera's pose. S23. The spatial coordinates and pose information of the camera are fused to obtain the pose information of the camera; S3, the web terminal requests pipeline attribute data near the camera from a remote server based on the camera's current location, determines the pipeline's pose information, and performs 3D modeling based on the basic pipeline attribute information; specifically including: S31. The server retrieves pipeline data and pose information from the database based on the current device location; S32. The locations of each node in the pipeline network are converted from their original GPS coordinates to spatial coordinates; the conversion formula is the spherical Mercator projection formula. S33. Model the pipeline network using AR support files based on pipeline network data and pose information; S4. The Web terminal obtains real-time video streams through Web RTC and renders 3D models through Web GL to achieve virtual-real fusion and obtain the pipeline network AR model; including: obtaining real-time video stream information through Web RTC, converting pipeline network coordinates from spatial Mercator coordinates to image coordinates, and performing 3D registration of the pipeline network AR model through rendering support JavaScript scripts. S5. Enter the pipeline network information that needs to be inspected into the database through the AR service page. The web terminal can view the pipeline network attribute data and historical maintenance records by clicking on the pipeline network AR model for subsequent maintenance.

2. The method for inspecting underground pipelines based on WEB AR technology according to claim 1, characterized in that, Pipeline network data is obtained from exploration data and data maps, including basic and real-time information of the pipeline network; the basic information includes: location information, pipeline radius, pipeline type, pipeline material, pipeline inner diameter, pipeline name, and pipeline number; Real-time information includes: whether the operation is normal or stopped.

3. The method for inspecting underground pipelines based on WEB AR technology according to claim 1, characterized in that, The AR support files in S1 include: an AR pipeline inspection service page, an AR support script, and a rendering support JavaScript script; the acquired AR pipeline inspection service page and AR rendering-related JavaScript script are cached in a local web browser.

4. The method for inspecting underground pipelines based on WEB AR technology according to claim 1, characterized in that, The S5 includes: The S50 and Web terminal provide AR model viewing of basic pipeline information, real-time information and maintenance records. Maintenance requests and maintenance type information can be submitted through the Web terminal for pipeline maintenance services. After submitting the information, a maintenance record is generated, including three parts: maintenance time, maintenance location and maintenance type. S51. The web terminal allows users to view basic pipeline information, real-time information, and maintenance records through an AR model via click-to-interact interaction. S52, the Web terminal provides a method for uploading real-time information of the current pipeline network. Users can upload the pipeline network information that needs to be inspected to the remote server to assist in pipeline network maintenance.

5. A Web AR-based underground pipeline inspection system for implementing the method of any one of claims 1-4, characterized in that, The system includes: Server module: Used to store basic attribute information, real-time status information, inspection status information, and AR service-related files of pipeline facilities; The file acquisition module is used by the web terminal to request AR service pages and AR service-related files from a remote server. The pose calculation module is used to acquire camera position information data and pose information data, calculate and track camera pose information, and at the same time, the acquired video stream data is used for virtual-real fusion of the background. The 3D modeling module obtains pipeline network-related information based on the positioning data and builds a 3D model of the pipeline network based on its attribute information. The AR rendering module is used to render the pipeline model and blend the AR model with real-time images.

Images