Augmented reality electric power pipeline inspection system for realizing SAR (Synthetic Aperture Radar) space stability

Abstract

The invention relates to the field of intelligent inspection, in particular to an augmented reality electric power pipeline inspection system for realizing SAR space stability, and the system comprises a data acquisition module which is used for obtaining GIS surveying and mapping data and positioning data of an electric power pipeline and a construction CAD drawing; the data normalization module is used for normalizing the data acquired by the data acquisition module, projecting the content data of the construction CAD drawing to a coordinate system map taking geodetic coordinates as a standard, and performing multi-vertex alignment and binding on the construction CAD drawing and a base map of the map so as to acquire an SAR map containing a power pipeline model; and the augmented reality module is used for collecting an inspection picture image and inspector positioning information during inspection, obtaining a power pipeline model section in the geodetic coordinate through the positioning information, and projecting the power pipeline model section into the inspection picture image through an AR technology. By adopting the technical scheme of the invention, the convenience of power pipeline inspection is improved by using the augmented reality technology through the SAR.

Description

Technical Field

[0001] The present invention relates to the field of intelligent inspection, and in particular to an augmented reality power pipeline inspection system for achieving SAR spatial stabilization. Background Art

[0002] With the development and progress of social economy, the demand for basic energy in urban residents, commerce, industry, administration and other industries is also increasing. Due to the rapid economic development, in order to ensure the development needs and the full launch of cable underground projects, the distribution of urban underground cable pipelines is complicated and the laying and updating speed is fast. The management is difficult, and the traditional GIS two-dimensional data expression of underground cable pipeline system can no longer meet the current user needs.

[0003] AR technology is a technology that superimposes virtual information on the physical world in real time. It combines computer graphics, image recognition, spatial positioning and other technologies to enable users to interact with virtual information in a real environment. In AR inspections, this technology is used to provide inspectors with real-time equipment status information, operation instructions, etc., thereby improving the efficiency and accuracy of inspections. Spatial augmented reality technology is a technology that applies virtual information to the real world. It allows the real environment and virtual objects to be superimposed on the same screen or space in real time and exist simultaneously. Since cable pipelines are normally buried underground, an inspection system is needed that can project cable pipelines through AR technology for easy viewing by inspectors. Summary of the invention

[0004] In order to solve the above problems, the present invention provides an augmented reality power pipeline inspection system for achieving SAR spatial stabilization, which is used to improve the convenience of power pipeline inspection by using augmented reality technology through SAR.

[0005] In order to achieve the above-mentioned purpose, the technical solution of the present invention is as follows: an augmented reality power pipeline inspection system for realizing SAR spatial stabilization, comprising: Data acquisition module: used to obtain GIS surveying and mapping data and positioning data of power pipelines and construction CAD drawings; Data normalization module: normalize the GIS surveying and mapping data, construction CAD drawings and positioning data of the power pipeline, project the content data of the construction CAD drawings into the coordinate system map based on the geodetic coordinates, and perform multi-vertex alignment and binding of the construction CAD drawings and the map base map. Then, the positions and parameter information of each point, line, and surface in the binding data are uniformly converted with the binding point as the core, so as to obtain the SAR map containing the power pipeline model. Augmented reality module: used to collect inspection screen images and inspector positioning information during inspection, obtain the power pipeline model segment in the geodetic coordinates through the positioning information, and project the power pipeline model segment into the inspection screen image through AR technology.

[0006] The above scheme has the following beneficial effects: 1. In this solution, the data acquisition module is used to obtain the GIS surveying and mapping data and positioning data of the power pipeline and the construction CAD drawings, so as to facilitate modeling based on positioning, and standardize the GIS surveying and mapping data, construction CAD drawings and positioning data together to form a model of mapping coordinates, ground details and power pipeline details in the geodetic coordinates. When conducting inspections, the inspectors carry equipment that can collect positioning information and inspection screen images, and project the power pipeline model segments in the geodetic coordinates into the inspection screen images through AR technology.

[0007] 2. In this solution, inspectors can use AR technology to observe the power pipeline model on the ground where it was originally impossible to see through it, so as to understand the distribution of underground power pipelines, assist inspectors in locating the position of power pipelines, and complete the inspection work conveniently.

[0008] Furthermore, the augmented reality module calculates the pose of the current frame relative to the active SAR map in real time through the collected inspection screen images, matches the SAR map features by minimizing the reprojection error, adds key frames and feature points to the active SAR map, removes redundant elements, and uses vision or visual inertial navigation BA to optimize the SAR map.

[0009] Beneficial effects: The pose calculation of the current frame relative to the active SAR map not only determines whether the current frame is used as a new keyframe, but also involves the estimation of the body velocity and IMU bias in visual inertial mode. Optimizing the SAR map through vision or visual inertial BA can enhance the stability of SAR.

[0010] Furthermore, the GIS surveying and mapping data, construction CAD drawings and positioning data in the data normalization module are all marked with MD5 files, and are identified and checked for duplication with existing data.

[0011] Beneficial effect: After the file is identified by MD5, a unique identifier can be generated, which is convenient for identifying and detecting the duplication of file data.

[0012] Furthermore, the data normalization module stores the summary data of each file together with the source file, creation time, and name in a unified data version. For CAD and GIS data, the editing time inside the file is read to expand the file identifier. Finally, the data version is managed and controlled with the internal editing time record of CAD and GIS as the first priority and the creation and editing time of the file as the second priority.

[0013] Beneficial effect: The power pipeline may be changed in subsequent maintenance, so the power pipeline model needs to be updated. Therefore, it is necessary to record each data update and perform version management.

[0014] Furthermore, in the data normalization module, CAD and GIS data include the name, range and mileage of each power pipeline. If there are multiple pipelines with the same name, the data with a later editing time will be used first; if multiple pipelines have different names but the same direction, range and mileage, they will be regarded as the same pipeline and merged version management will be carried out.

[0015] Beneficial effects: Data should be detailed to each pipeline for refined management. Pipelines with the same name and data should be selected or merged to simplify mixed data and make the data clearer.

[0016] Furthermore, the geodetic coordinates adopt the WGS84 coordinate system.

[0017] Beneficial effect: The origin of the WGS84 coordinate system is the center of mass of the earth, and the Z axis of the spatial rectangular coordinate system points to the direction of the Earth's pole (CTP) defined by BIH (1984.0), that is, the internationally agreed origin CIO. Using the WGS84 coordinate system can standardize the coordinate system.

[0018] Furthermore, the augmented reality module includes a display on which a camera, a GPS module, an electronic compass, an inertial sensor and a gyroscope are provided.

[0019] Beneficial effects: Inspectors can carry out inspections with handheld displays and take images containing the ground during the inspection process. The augmented reality module will enhance the ground image to display the power pipeline model section in the image. Inspectors can quickly find the target inspection location based on the guidance of the power pipeline model section.

[0020] Furthermore, the augmented reality module includes a helmet body, on which a camera, a GPS module, an electronic compass, an inertial sensor, a gyroscope and AR glasses are provided.

[0021] Beneficial effects: Inspectors can wear helmets, which will capture images of their visual field. The augmented reality module will enhance the visual field images and project them on AR glasses, freeing the inspectors' hands and allowing them to guide the power pipelines through AR glasses.

[0022] Furthermore, inertial sensors and gyroscopes are used to collect six-axis attitude information, and the position of the power pipeline model on the inspection screen image is adjusted in real time according to the attitude and direction information.

[0023] Beneficial effects: Inertial sensors and gyroscopes can obtain imaging angle information, thereby assisting in adjusting the position of the power pipeline model on the inspection screen image and improving the display format.

[0024] Furthermore, it also includes a marking module, which is used for inspectors to mark the power pipeline model information. The power pipeline model information is bound to the positioning information and six-axis posture information of the GPS module. The marking is also used to change the display color of the power pipeline model segment.

[0025] Beneficial effects: Inspectors can use the power pipeline model to quickly mark, for example, if there are hidden dangers in the power pipeline, the power pipeline is damaged, the power pipeline does not match the power pipeline model, etc., the positioning information based on the GPS module can quickly locate the marked location node, and the six-axis attitude information can obtain the perspective information, thereby further locating the marked position of the information, making it easier to find the marked indication area later. At the same time, changing the display color of the power pipeline model segment can facilitate the rapid finding of the marked power pipeline location during subsequent power pipeline inspections.

[0026] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 It is a module schematic diagram of an embodiment of an augmented reality power pipeline inspection system for realizing SAR spatial stabilization according to the present invention; Figure 2 The figure is a schematic diagram of the augmented reality display effect of an augmented reality power pipeline inspection system embodiment for achieving SAR spatial stabilization according to the present invention. DETAILED DESCRIPTION

[0028] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0029] In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicating the orientation or positional relationship, are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", and "third" are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance.

[0030] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0031] The following is further described in detail through specific implementation methods: Example 1 As attached Figure 1-Figure 2 As shown: An augmented reality power pipeline inspection system for achieving SAR spatial stabilization, comprising: Data acquisition module: used to obtain GIS surveying and mapping data and positioning data of power pipelines and construction CAD drawings; Data normalization module: normalize the GIS surveying and mapping data, construction CAD drawings and positioning data of the power pipeline, project the content data of the construction CAD drawings into the coordinate system map based on the geodetic coordinates, and perform multi-vertex alignment and binding of the construction CAD drawings and the map base map. Then, the positions and parameter information of each point, line, and surface in the binding data are uniformly converted with the binding point as the core, so as to obtain the SAR map containing the power pipeline model. Augmented reality module: used to collect inspection screen images and inspector positioning information during inspection, obtain the power pipeline model segment in the geodetic coordinates through the positioning information, and project the power pipeline model segment into the inspection screen image through AR technology.

[0032] In the data normalization module, each file summary data, source file, creation time, and name are uniformly stored in the data version; for CAD and GIS data, the internal editing time of the file is read to expand the file identification. Finally, the data version is managed and controlled with the internal editing time record of CAD and GIS as the first priority, and the creation and editing time of the file as the second priority.

[0033] In the data normalization module, CAD and GIS data include the name, range and mileage of each power pipeline. If there are multiple pipelines with the same name, the data with a later editing time will be used first; if multiple pipelines have different names but the same direction, range and mileage, they will be regarded as the same pipeline and merged version management will be carried out.

[0034] Augmented reality module: used to collect inspection screen images and inspector positioning information during inspection, obtain the power pipeline model segment in the geodetic coordinates through the positioning information, and project the power pipeline model segment into the inspection screen image through AR technology.

[0035] The augmented reality module includes a display, which is equipped with a camera, a GPS module, an electronic compass, an inertial sensor and a gyroscope. The inertial sensor and the gyroscope are used to collect six-axis attitude information and adjust the position of the power pipeline model on the inspection screen image in real time according to the attitude and direction information.

[0036] When using it, you need to build a power pipeline model first. During the inspection, since most power pipelines are located underground, the first step for the inspector is to reach the ground location of the power pipeline. Therefore, it is necessary to build a SAR map bound to the inspector's positioning coordinates and compare it with the ground images taken by the inspector before or during the inspection to determine the detailed location of the power pipeline.

[0037] The origin of the WGS84 coordinate system is the center of mass of the earth, and the Z axis of the spatial rectangular coordinate system points to the direction of the Earth's pole (CTP) defined by BIH (1984.0), that is, the internationally agreed origin CIO. Using the WGS84 coordinate system can standardize the coordinate system.

[0038] The GIS surveying and mapping data, construction CAD drawings and positioning data of the power pipeline are used to build the power pipeline model in the geodetic coordinates. The GIS surveying and mapping data can obtain the coordinate information of each part of the power pipeline, and the construction CAD drawings can obtain the structural parameters to facilitate modeling, thereby forming a power pipeline model bound to the coordinates. By obtaining the inspector's positioning information, the power pipeline model segment and ground details can be read.

[0039] In the augmented reality module, the pose of the current frame relative to the active SAR map is calculated in real time through the collected inspection screen images, the SAR map features are matched by minimizing the reprojection error, key frames and feature points are added to the active SAR map, redundant elements are removed, and the SAR map is optimized using vision or visual inertial navigation BA. The pose calculation of the current frame relative to the active SAR map not only determines whether the current frame is used as a new key frame, but also involves the estimation of the body velocity and IMU bias in the visual inertial mode. Optimizing the SAR map through vision or visual inertial navigation BA can enhance the stability of SAR.

[0040] The imported GIS surveying and mapping data, construction CAD drawings and positioning data exist in multiple copies at multiple time nodes. Therefore, file identification through MD5 can generate a unique identifier, which is convenient for identifying and duplication detection of file data.

[0041] The power pipeline may be changed in subsequent maintenance, so the power pipeline model needs to be updated. Therefore, it is necessary to record each data update and perform version management to reduce the disorder of data files and interfere with the imaging process.

[0042] The data needs to be detailed to each pipeline for refined management. Pipelines with the same name and data should be selected or merged to simplify the mixed data and make it clearer.

[0043] During the inspection, the inspector can hold a handheld display for inspection and use the camera of the display to view the inspection screen image. The inspection screen image includes the capture of the ground screen and obtains its own precise positioning information through GPS. After obtaining the precise positioning information, the positioning information will be brought into the geodetic coordinates to obtain the power pipeline model segment and ground details respectively. The electronic compass, inertial sensor and gyroscope detect the six-axis posture information of the current inspector's shooting picture to determine the viewing angle of the camera. The display angle of the power pipeline model segment is processed based on the viewing angle, and the three-dimensional power pipeline model segment with depth is projected into the inspection screen image based on the collected ground details. The inspector can observe the three-dimensional power pipeline model segment displayed at the underground depth on the surface, which is convenient for quickly finding the target location and reducing the difficulty of inspection.

[0044] Example 2 The difference from the above embodiment is that the augmented reality module includes a helmet body, on which a camera, a GPS module, an electronic compass, an inertial sensor, a gyroscope and AR glasses are provided.

[0045] The inspector can wear a helmet, which can serve as a safety helmet. The camera on the helmet can capture the inspector's field of view, thereby obtaining an inspection image corresponding to the inspector's vision. The augmented reality module will project the power pipeline model segment on the inspection image, and the inspector can directly observe the power pipeline model segment imaging through AR glasses, so as to quickly find the target location and reduce the difficulty of inspection.

[0046] Example 3 The difference from the above embodiment is that it also includes a marking module, which is used for inspectors to mark the power pipeline model information. The power pipeline model information is bound to the positioning information and six-axis posture information of the GPS module. The marking is also used to change the display color of the power pipeline model segment.

[0047] Inspectors can use the power pipeline model to quickly mark, for example, if there are hidden dangers in the power pipeline, the power pipeline is damaged, the power pipeline does not match the power pipeline model, etc. The positioning information based on the GPS module can quickly locate the marked location node, and the six-axis attitude information can obtain the viewing angle information, thereby further locating the marked position of the information, making it easier to find the marked indication area later. At the same time, changing the display color of the power pipeline model segment can facilitate the rapid finding of the marked power pipeline location during subsequent power pipeline inspections.

[0048] Obviously, the above embodiments are merely examples for the purpose of clear explanation, and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived therefrom are still within the scope of protection of the invention.

Claims

1. An augmented reality power pipeline inspection system for achieving SAR spatial stabilization, characterized in that: include: Data acquisition module: used to obtain GIS surveying and mapping data and positioning data of power pipelines and construction CAD drawings; Data normalization module: normalize the GIS surveying and mapping data, construction CAD drawings and positioning data of the power pipeline, project the content data of the construction CAD drawings into the coordinate system map based on the geodetic coordinates, and perform multi-vertex alignment and binding of the construction CAD drawings and the map base map. Then, the positions and parameter information of each point, line, and surface in the binding data are uniformly converted with the binding point as the core, so as to obtain the SAR map containing the power pipeline model. Augmented reality module: used to collect inspection screen images and inspector positioning information during inspection, obtain the power pipeline model segment in the geodetic coordinates through the positioning information, and project the power pipeline model segment into the inspection screen image through AR technology.

2. The augmented reality power pipeline inspection system for achieving SAR spatial stabilization according to claim 1, characterized in that: In the augmented reality module, the pose of the current frame relative to the active SAR map is calculated in real time through the collected inspection screen images, the SAR map features are matched by minimizing the reprojection error, key frames and feature points are added to the active SAR map, redundant elements are removed, and the SAR map is optimized using vision or visual inertial navigation BA.

3. The augmented reality power pipeline inspection system for achieving SAR spatial stabilization according to claim 2, characterized in that: In the data normalization module, GIS surveying and mapping data, construction CAD drawings and positioning data are all marked with MD5 files, and identified and checked for duplication with existing data.

4. The augmented reality power pipeline inspection system for achieving SAR spatial stabilization according to claim 3, characterized in that: In the data normalization module, each file summary data, source file, creation time, and name are uniformly stored in the data version; for CAD and GIS data, the internal editing time of the file is read to expand the file identification. Finally, the data version is managed and controlled with the internal editing time record of CAD and GIS as the first priority, and the creation and editing time of the file as the second priority.

5. The augmented reality power pipeline inspection system for achieving SAR spatial stabilization according to claim 4, characterized in that: In the data normalization module, CAD and GIS data include the name, range and mileage of each power pipeline. If there are multiple pipelines with the same name, the data with a later editing time will be used first; if multiple pipelines have different names but the same direction, range and mileage, they will be regarded as the same pipeline and merged version management will be carried out.

6. The augmented reality power pipeline inspection system for achieving SAR spatial stabilization according to claim 5, characterized in that: The geodetic coordinates use the WGS84 coordinate system.

7. The augmented reality power pipeline inspection system for achieving SAR spatial stabilization according to claim 6, characterized in that: The augmented reality module includes a display on which a camera, a GPS module, an electronic compass, an inertial sensor and a gyroscope are arranged.

8. The augmented reality power pipeline inspection system for achieving SAR spatial stabilization according to claim 6, characterized in that: The augmented reality module includes a helmet body, on which are provided a camera, a GPS module, an electronic compass, an inertial sensor, a gyroscope and AR glasses.

9. The augmented reality power pipeline inspection system for achieving SAR spatial stabilization according to claim 8, characterized in that: Inertial sensors and gyroscopes are used to collect six-axis attitude information, and adjust the position of the power pipeline model on the inspection screen image in real time based on the attitude and direction information.

10. The augmented reality power pipeline inspection system for achieving SAR spatial stabilization according to claim 9, characterized in that: It also includes a marking module, which is used for inspectors to mark the power pipeline model information. The power pipeline model information is bound to the positioning information and six-axis attitude information of the GPS module. The marking is also used to change the display color of the power pipeline model segment.

Images