AR glasses-based electric power underground hidden project real-time display method and device

Abstract

The invention discloses a real-time display method and device for an electric power underground concealed project based on AR glasses. The method comprises the following steps: acquiring an RTK geographic positioning coordinate through an RTK positioning module integrated with the AR glasses; constructing a coordinate conversion relation through an RTK geographic positioning coordinate, an AR system coordinate and an AR geographic due north correction angle theta; extracting a geographic coordinate set of the target model from the underground engineering model database, and establishing a relative position mapping relation from the geographic coordinates of the model to the current RTK coordinate system; based on the established coordinate conversion relation, converting the geographic coordinates of the underground engineering model into relative coordinates under the coordinate system of the AR system; and dynamically binding the calculated relative coordinates to a physical space through a visual tracking module of AR glasses, and generating an underground model holographic image consistent with the perspective relationship of a real earth surface scene. The method aims at achieving high-precision and stable visualization of the three-dimensional model of the electric power underground hidden project and improving the safety level of electric power inspection and construction.

Description

TECHNICAL FIELD

[0001] The application relates to a real-time display method for electric underground concealed projects, in particular to a real-time display method and device for electric underground concealed projects based on AR glasses. BACKGROUND

[0002] In the field of electric power facility operation and maintenance, the inspection of underground concealed projects (such as underground high-voltage cables, pipe galleries and grounding grids) has long been faced with the dual challenges of visualization deficiency and positioning accuracy deficiency. In the traditional operation mode, operation and maintenance personnel rely on two-dimensional drawings, handheld terminals and personal experience to determine the direction of underground facilities, and in complex site environments, positioning deviation is easily caused due to the lack of surface reference, which may lead to major accidents such as mis-digging of cables. The existing technology attempts to improve the inspection efficiency through AR glasses, but still has fundamental limitations, such as positioning dependence on markers, geographical coordinate misalignment and spatial matching distortion, which cannot meet the accuracy requirements of long-distance cable inspection.

[0003] Therefore, it is of great significance to provide a real-time display method and device for electric underground concealed projects based on AR glasses. SUMMARY

[0004] The purpose of the application is to provide a real-time display method and device for electric underground concealed projects based on AR glasses, which realizes accurate spatial registration and real-time rendering of underground concealed project models and improves the real-time display and positioning accuracy of AR models.

[0005] Technical scheme: The real-time display method for electric underground concealed projects based on AR glasses comprises the following steps:

[0006] (1) obtaining RTK geographical positioning coordinates through an RTK positioning module integrated in the AR glasses;

[0007] (2) constructing a coordinate conversion relationship by using the RTK geographical positioning coordinates, AR system coordinates obtained through a multi-sensor positioning mode of the AR glasses and a pre-calibrated AR geographical north correction angle θ, and realizing the alignment of the AR system coordinate system and the real geographical coordinate system;

[0008] (3) extracting a geographical coordinate set of a target model from a pre-constructed underground project model database; associating the model geographical coordinates with the RTK real-time positioning coordinates in space, and establishing a relative position mapping relationship of the model geographical coordinates to the current RTK coordinate system;

[0009] (4) converting the geographical coordinates of the underground project model into relative coordinates in the AR system coordinate system based on the coordinate conversion relationship established in step (2);

[0010] (5) Through the visual tracking module of the AR glasses, the relative coordinates calculated in step (4) are dynamically bound to the physical space, and an AR rendering pipeline is called to generate a holographic image of the underground model consistent with the perspective relationship of the real ground surface scene.

[0011] Further, the step (1) RTK positioning coordinate acquisition method comprises:

[0012] The RTK positioning module integrated in the AR glasses receives satellite differential signals, and obtains centimeter-level precision geographic coordinates and azimuth angle data in real time, outputs the coordinates in the RTK fixed solution state, and obtains the RTK positioning coordinates.

[0013] Further, the multi-sensor positioning mode in step (2) is to obtain device position data by combining the inertial measurement unit (IMU) and the visual SLAM system built-in the AR glasses and the historical RTK positioning coordinates.

[0014] The construction of the coordinate conversion relationship specifically comprises: translating the origin of the AR system coordinate system to the RTK geographic coordinate point, and rotating the Z-axis of the AR system coordinate system to align with the geographic north direction by using the north correction angle θ.

[0015] Further, the step (4) converts the geographic coordinates of the underground engineering model into relative coordinates in the AR system coordinate system, and the formula is:

[0016]

[0017] wherein, is the three-dimensional relative coordinates of the model in the AR coordinate system, is a rotation matrix constructed by θ, is the geographic coordinates of the model, is the RTK geographic coordinates of the AR glasses.

[0018] Further, the model data stored in the underground engineering model database comprises geographic latitude and longitude, depth information, size parameters and facility types.

[0019] Further, the step (5) specifically comprises: the AR glasses continuously track the real scene feature points by the SLAM engine, and dynamically bind the calculated model relative coordinates to the physical space; combined with the environmental lighting parameters and the modeling data of the underground engineering model, an AR rendering pipeline is called to generate a holographic image of the underground model consistent with the perspective relationship of the real ground surface scene, so as to realize three-dimensional superimposed display with correct occlusion relationship.

[0020] Further, the method further comprises: when it is detected that the positioning error accumulated by the AR glasses itself sensor exceeds a preset value, the RTK positioning module changes from a fixed solution state to a floating solution state, or a user issues a manual calibration instruction, re-executing step (1) to obtain new RTK geographic coordinates, and triggering the re-matching of the coordinate system and the update of the model display.

[0021] The AR glasses-based electric power underground concealed engineering real-time display device provided by the application comprises:

[0022] The coordinate positioning module is used to obtain RTK geographic positioning coordinates through an RTK positioning module integrated in the AR glasses.

[0023] The coordinate conversion matching module is used to: construct a coordinate conversion relationship through the RTK geographic positioning coordinates, AR system coordinates obtained by the AR glasses through a multi-sensor positioning mode, and a pre-calibrated AR geographic north correction angle θ, realize the alignment of the AR system coordinate system and the real geographic coordinate system, extract a geographic coordinate set of a target model from a pre-constructed underground engineering model database, associate the model geographic coordinates with the RTK real-time positioning coordinates in space positions, establish a relative position mapping relationship of the model geographic coordinates to the current RTK coordinate system, and convert the geographic coordinates of the underground engineering model into relative coordinates in the AR system coordinate system based on the established coordinate conversion relationship.

[0024] The AR model rendering module is used to: through a visual tracking module of the AR glasses, dynamically bind the calculated relative coordinates to a physical space, and call an AR rendering pipeline to generate a holographic image of the underground model consistent with the perspective relationship of the real ground scene.

[0025] The augmented information display module is used to: when it is detected that the positioning error accumulated by the AR glasses itself sensor exceeds a preset value, the RTK positioning module changes from a fixed solution state to a floating solution state, or a user issues a manual calibration instruction, re-obtain new RTK geographic coordinates, and trigger the re-matching of the coordinate system and the update of the model display.

[0026] The computer device provided by the application comprises one or more processors, a memory and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the programs are used to implement the steps of the AR glasses-based electric power underground concealed engineering real-time display method.

[0027] The computer readable storage medium provided by the application has a computer program stored thereon, and the computer program is used to implement the steps of the AR glasses-based electric power underground concealed engineering real-time display method.

[0028] Beneficial effects: compared with the prior art, the present application has the following remarkable advantages:

[0029] (1) The present application introduces RTK centimeter-level positioning to ensure high-precision registration of underground engineering models and real space, avoiding the cumulative error of traditional reliance on visual features;

[0030] (2) Through multi-sensor fusion positioning and north correction angle θ calibration, the matching of AR coordinate system and geographic coordinate system is realized, and the stability of model display is improved;

[0031] (3) Through database standardized modeling and dynamic coordinate transformation, large-scale underground power engineering can be quickly retrieved and displayed;

[0032] (4) Through the joint calibration mechanism of AR glasses fusion positioning system and RTK positioning coordinates, the model can still maintain accurate spatial alignment in long-time display, thereby improving the safety and efficiency of power operation and construction inspection. BRIEF DESCRIPTION OF DRAWINGS

[0033] Figure 1 The method flowchart of the present application is shown.

[0034] Figure 2 The schematic diagram of RTK positioning coordinates and AR system coordinate matching transformation of the present application is shown.

[0035] Figure 3 The device module diagram of the present application is shown. DETAILED DESCRIPTION

[0036] The technical solutions of the present application will be further described below with reference to the accompanying drawings.

[0037] Example 1

[0038] As shown in the figure, the present application provides a real-time display method for power underground concealed engineering based on AR glasses, which comprises: Figure 1

[0039] S1, obtaining RTK geographic positioning coordinates through the RTK positioning module integrated in AR glasses.

[0040] RTK (Real-Time Kinematic, Real-Time Kinematic) positioning technology is a high-precision positioning method based on satellite navigation system, which receives satellite signals through a reference station and calculates positioning error, and sends error correction data to mobile terminal (such as AR glasses) in real time to correct position data in real time, realizing dynamic positioning with centimeter-level precision.

[0041] ​The reference station is a control point (such as a city surveying reference point) fixedly arranged at a known coordinate, adopts a dual-frequency GNSS antenna (supports Beidou B1I / B2I, GPS L1 / L2, and GLONASS G3 frequency bands), receives carrier phase data of satellite signals; the mobile station is integrated in an AR glass (or an external portable terminal), receives satellite signals, and receives error correction data (including satellite orbit error and ionospheric / tropospheric delay error) sent by the reference station through a 4G / 5G private network or Bluetooth, and eliminates system error through carrier phase differential solution.

[0042] Through carrier phase differential solution between the reference station and the mobile station, when the mobile station enters a fixed solution state (success rate of solution is greater than or equal to 95%), geographic coordinates (longitude Lon, latitude Lat, and height H) are output, in which horizontal error is less than or equal to 1 cm, and height error is less than or equal to 2 cm; if in a floating solution state (precision is less than or equal to 10 cm), the system triggers an audible and light reminder, prompting an operation and maintenance personnel to move to an open area to reacquire a fixed solution. The RTK high-precision coordinate provides an absolute spatial reference for matching of an underground engineering model and real geographic coordinates.

[0043] The device for acquiring the RTK high-precision coordinate can adopt two integrated modes: one is to directly embed an RTK positioning module into the AR glass body; and the other is to connect a portable RTK positioning terminal through a Type-C or Bluetooth, to ensure portability in a mobile scene.

[0044] S2, a coordinate conversion relationship is constructed through the RTK geographic positioning coordinate, an AR system coordinate acquired by the AR glass through a multi-sensor positioning mode, and a pre-calibrated AR geographic north correction angle θ, to realize alignment of the AR system coordinate system and the real geographic coordinate system.

[0045] The geographic coordinate system is a global coordinate system, and describes an absolute position by using longitude and latitude (Lon, Lat) and height H, but cannot be directly used for three-dimensional rendering of the AR device; the AR system coordinate system is a local Cartesian coordinate system (origin is an optical center of the AR glass, X axis is horizontal to the right, Y axis is vertical upward, and Z axis is horizontal forward) of the AR glass, and needs to be associated with the geographic coordinate system through coordinate conversion.

[0046] The AR system coordinate system is a local Cartesian coordinate system (X, Y, Z) of the AR glasses device, with the origin at the optical center. The AR system coordinate is obtained through a multi-sensor fusion positioning mode, and the AR glasses obtain real-time pose data (six degrees of freedom, 6-DoF) through a multi-sensor fusion positioning mode. The pose is a set of six degrees of freedom (6-DoF) parameters describing the position (Position) and orientation (Orientation) of an object in an AR three-dimensional space. The position is the coordinates (X, Y, Z) of the origin of the object in the coordinate system, and the orientation refers to the rotation state of the object coordinate system relative to the reference coordinate system, which is usually represented by a rotation matrix R, a quaternion (x, y, z, w), or Euler angles (Roll, Pitch, Yaw). Specifically, it includes: an inertial measurement unit (IMU) collects angular velocity and acceleration to realize short-term pose prediction; a visual SLAM (simultaneous localization and mapping) matches feature points to calculate position changes based on environment images captured by a camera; and a historical RTK positioning coordinate is combined to correct the cumulative errors of the IMU and the SLAM, and finally the position (X, Y, Z) and orientation angle in the AR system coordinate system are output.

[0047] As shown in Figure 2 , by obtaining the true north geographic correction angle θ (i.e. the angle between the AR system coordinate system Z axis and the true geographic north), a two-step alignment operation is performed: the origin of the AR system coordinate system is translated to the RTK high-precision coordinate (Lon, Lat, H) obtained in step S1 to realize origin coincidence; and the AR system coordinate system is rotated around the Y axis by θ angle to make the AR coordinate system Z axis consistent with the geographic north, thereby establishing a rigid conversion relationship between the AR system coordinate system and the geographic coordinate system, and the conversion matrix is denoted as R. After that, the origin in the AR system coordinate system can be coincided with the current RTK high-precision coordinate, and the coordinate axis angle can be corrected.

[0048] S3, from the pre-constructed underground engineering model database, extracting the geographic coordinate set of the target model; correlating the model geographic coordinates with the RTK real-time positioning coordinates in space position, and establishing a relative position mapping relationship of the model geographic coordinates to the current RTK coordinate system.

[0049] A pre-constructed three-dimensional model database of underground concealed power engineering is provided, and the model needs to carry a complete set of geographic coordinates, including: engineering body coordinates (such as cable start and end point line latitude and longitude coordinates, pipeline top elevation, power equipment foundation coordinates); depth information (such as cable burial depth, pipeline soil cover thickness); associated attribute information (such as device model, laying time, maintenance record, voltage grade, etc.), and the model attributes are stored in a relational database, and the associated field is "model ID-coordinate point ID-attribute information".

[0050] The AR glasses send a retrieval request and a search radius R to the cloud server through its own coordinates P. The server screens the target model according to P and R, and returns the model coordinate set and associated attribute information. The model geographic coordinates are associated with the RTK real-time positioning coordinates in step S1 in space position, the relative position mapping relationship of the model geographic coordinates to the current RTK coordinate system is established, and more associated attribute information can be further displayed through the mapping relationship.

[0051] S4, based on the coordinate conversion relationship established in S2, the geographic coordinates of the underground engineering model are converted into relative coordinates in the AR system coordinate system.

[0052] The calculation formula is as follows:

[0053]

[0054] wherein, is the three-dimensional relative coordinates of the model in the AR coordinate system, is a rotation matrix constructed with theta, is the geographic coordinates of the model, is the RTK geographic coordinates of the AR glasses.

[0055] S5, through the visual tracking module of the AR glasses, the relative coordinates calculated in step S4 are dynamically bound to the physical space, and the AR rendering pipeline is called to generate a holographic image of the underground model consistent with the perspective relationship of the real ground scene.

[0056] Specifically, the AR glasses realize dynamic matching of the model and the physical space through visual tracking and environment feature binding, including feature point detection, coordinate binding, and occlusion processing. Then, the rendering pipeline built-in the AR glasses is called to superimpose the rendered model image and the environment image collected by the camera in real time, and output to the display screen of the AR glasses.

[0057] S6, when it is detected that the positioning error accumulated by the sensor of the AR glasses exceeds the preset value, the RTK positioning module changes from the fixed solution state to the floating solution state, or the user issues a manual calibration instruction, step S1 is re-executed to obtain new RTK geographic coordinates, and the coordinate system is re-matched and the model display is updated.

[0058] The AR glasses use its own multi-sensor fusion positioning mode to maintain the stability of the three-dimensional model display. Once the AR glasses itself accumulates error and produces augmented reality model deviation, the RTK fixed solution can be re-acquired. After the RTK high-precision coordinates are re-acquired, the coordinate system is re-aligned to ensure the re-matching accuracy of the model geographic coordinates and the AR system coordinates, and the iterative updating effect is realized.

[0059] When the index is triggered abnormally, such as AR augmented information module shows that the error is too large, RTK module fails to obtain fixed solution, automatic calibration can be performed, coordinate conversion module updates conversion matrix based on new RTK coordinates, and model matching and rendering module recalculates and refreshes display; manual calibration trigger: the user starts the above process through the physical button or voice instruction of the AR glasses, which is suitable for the scene of automatic calibration failure.

[0060] Embodiment 2

[0061] As shown in Figure 3 The embodiment also provides an AR glasses-based real-time display device for electric underground concealed works, which comprises:

[0062] A coordinate positioning module is configured to obtain RTK geographic positioning coordinates through an RTK positioning module integrated in the AR glasses.

[0063] A coordinate conversion matching module is configured to: construct a coordinate conversion relationship by using the RTK geographic positioning coordinates, AR system coordinates obtained by the AR glasses through a multi-sensor positioning mode, and a pre-calibrated AR geographic north correction angle θ, so as to align the AR system coordinate system with the real geographic coordinate system; extract a geographic coordinate set of a target model from a pre-constructed underground works model database; associate the model geographic coordinates with the RTK real-time positioning coordinates in space, and establish a relative position mapping relationship from the model geographic coordinates to the current RTK coordinate system; and convert the geographic coordinates of the underground works model into relative coordinates in the AR system coordinate system based on the established coordinate conversion relationship.

[0064] An AR model rendering module is configured to: dynamically bind the calculated relative coordinates to a physical space through a vision tracking module of the AR glasses, and call an AR rendering pipeline to generate a holographic image of the underground model consistent with the perspective relationship of the real ground scene.

[0065] An augmented information display module is configured to: when it is detected that the positioning error accumulated by the sensors of the AR glasses exceeds a preset value, the RTK positioning module changes from a fixed solution state to a floating solution state, or a manual calibration instruction is issued by the user, new RTK geographic coordinates are reacquired, and re-matching of the coordinate system and updating of the model display are triggered.

[0066] Embodiment 3

[0067] A computer device comprises one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the programs are executed by the processors to implement the steps of the AR glasses-based real-time display method for electric underground concealed works.

[0068] Embodiment 4

[0069] A computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the method for real-time display of underground concealed engineering based on AR glasses.

Claims

1. A method for real-time display of concealed underground power engineering projects based on AR glasses, characterized in that, include: (1) Obtain RTK geolocation coordinates through the RTK positioning module integrated in AR glasses; (2) By using RTK geolocation coordinates, AR system coordinates obtained by AR glasses through multi-sensor positioning mode, and pre-calibrated AR geographic north correction angle θ, a coordinate transformation relationship is constructed to achieve alignment between the AR system coordinate system and the real geographic coordinate system; (3) Extract the geographic coordinate set of the target model from the pre-built underground engineering model database; associate the geographic coordinates of the model with the real-time RTK positioning coordinates to establish the relative position mapping relationship between the geographic coordinates of the model and the current RTK coordinate system; (4) Based on the coordinate transformation relationship established in step (2), convert the geographic coordinates of the underground engineering model into relative coordinates in the AR system coordinate system; (5) Through the visual tracking module of the AR glasses, the relative coordinates calculated in step (4) are dynamically bound to the physical space, and the AR rendering pipeline is called to generate an underground model holographic image that is consistent with the perspective relationship of the real ground scene.

2. The method for real-time display of underground power concealed works based on AR glasses according to claim 1, characterized in that, The method for obtaining RTK positioning coordinates in step (1) includes: The AR glasses integrate an RTK positioning module to receive satellite differential signals, calculate and obtain centimeter-level precision geographic coordinates and azimuth data in real time, and output coordinates in RTK fixed solution state, thus obtaining RTK positioning coordinates.

3. The method for real-time display of underground power concealed works based on AR glasses according to claim 1, characterized in that, In step (2), the multi-sensor positioning mode combines the inertial measurement unit (IMU) built into the AR glasses, the visual SLAM system, and historical RTK positioning coordinates to obtain device location data. The construction of the coordinate transformation relationship specifically includes: translating the origin of the AR system coordinate system to the RTK geographic coordinate point, and using the north correction angle θ to rotate the Z-axis of the AR system coordinate system to align with geographic north.

4. The method for real-time display of underground power concealed works based on AR glasses according to claim 1, characterized in that, Step (4) converts the geographic coordinates of the underground engineering model into relative coordinates in the AR system coordinate system, using the following formula: in, The model's three-dimensional relative coordinates in the AR coordinate system. It is a rotation matrix constructed using θ. The geographic coordinates of the model, These are the RTK geographic coordinates for the AR glasses.

5. A method for real-time display of underground power concealed works based on AR glasses according to claim 1, characterized in that, The model data stored in the underground engineering model database includes geographical latitude and longitude, depth information, size parameters, and facility types.

6. A method for real-time display of underground power concealed works based on AR glasses according to claim 1, characterized in that, The specific steps (5) include: AR glasses continuously track feature points of the real scene through the SLAM engine, dynamically bind the calculated model relative coordinates to the physical space; adjust the model display effect according to the ambient lighting parameters, and perform correct occlusion processing, so that the underground facility model is embedded in the real surface scene in a way that conforms to perspective.

7. A method for real-time display of underground power concealed works based on AR glasses according to claim 1, characterized in that, The method further includes: when the AR glasses' own sensor accumulates a positioning error exceeding a preset value, the RTK positioning module changes from a fixed solution state to a floating solution state, or the user issues a manual calibration command, then step (1) is re-executed to obtain new RTK geographic coordinates, and the coordinate system is re-matched and the model display is updated.

8. A real-time display device for underground power concealed engineering projects based on AR glasses, characterized in that, include: Coordinate positioning module: Used to obtain RTK geographic positioning coordinates through the RTK positioning module integrated into AR glasses; Coordinate transformation and matching module: By using RTK geolocation coordinates, AR system coordinates obtained by AR glasses through multi-sensor positioning mode, and the pre-calibrated AR geographic north correction angle θ, a coordinate transformation relationship is constructed to align the AR system coordinate system with the real geographic coordinate system; the geographic coordinate set of the target model is extracted from the pre-built underground engineering model database; the geographic coordinates of the model are spatially associated with the RTK real-time positioning coordinates to establish a relative position mapping relationship from the geographic coordinates of the model to the current RTK coordinate system; Based on the established coordinate transformation relationship, the geographic coordinates of the underground engineering model are converted into relative coordinates in the AR system coordinate system; AR Model Rendering Module: Through the visual tracking module of AR glasses, the calculated relative coordinates are dynamically bound to the physical space, and the AR rendering pipeline is called to generate a holographic image of the underground model that is consistent with the perspective relationship of the real ground scene. Enhanced information display module: When the AR glasses' own sensor accumulates a positioning error exceeding a preset value, the RTK positioning module changes from a fixed solution state to a floating solution state, or the user issues a manual calibration command, new RTK geographic coordinates are reacquired, and the coordinate system is rematched and the model display is updated.

9. A computer device, characterized in that, It includes one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the programs, when executed by the processors, implement the steps of a real-time display method for underground power concealment projects based on AR glasses as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the real-time display method for underground power concealment projects based on AR glasses as described in any one of claims 1-7.

Images