High-precision fusion positioning system and method for stable display of AR cable pipeline

Abstract

The invention relates to the field of data processing, in particular to a high-precision fusion positioning system and method which are used for AR underground cable pipelines and are stably displayed when a user terminal moves, and is used for solving the problem that a target object deviates from the original position along with movement of equipment in common AR application. The system comprises a cable pipeline display module which is used for constructing an AR map containing a cable pipeline model, the cable pipeline model is bound with the AR map through positioning coordinates, the cable pipeline display module is provided with an accelerometer and a gyroscope which are used for collecting inertia vectors, the cable pipeline display module is used for inputting the positioning coordinates and the inertia vectors, and the positioning coordinates and the inertia vectors are used for positioning the AR map. Outputting and displaying a cable pipeline model in the AR map; and the ORB-SLAM3-L correction module is used for improving the spatial reliability of the AR map through a unique ORB-SLAM3-L algorithm. By adopting the technical scheme of the invention, the stability of AR cable pipeline display can be optimized through the ORB-SLAM3-L algorithm.

Description

Technical Field

[0001] The present invention relates to the field of data processing, and in particular to a high-precision fusion positioning system and method for stable display of AR cable pipelines. Background Art

[0002] In the daily inspection process of urban cable pipelines, due to the high concealment of urban underground cable pipelines, the use of traditional underground cable pipeline GIS information not only has a certain threshold for map recognition, but also due to changes in environmental reference objects such as environmental changes, geological changes, and vegetation changes in real scenes, it is impossible to determine the specific actual location of the underground cable pipeline in the GIS information. At the same time, due to the lack of spatial sense of viewing pipelines in GIS information, the quality of line inspection by line inspectors cannot be guaranteed, and the accuracy of pipeline excavation cannot be guaranteed.

[0003] Therefore, an augmented reality technology based on GIS high-precision positioning technology, intelligent north calibration, and ORB-SLAM3-L is needed to improve the accuracy of AR applications and enhance the spatial reliability of AR maps. Summary of the invention

[0004] To solve the above problems, the present invention provides a high-precision fusion positioning system and method for stable display of AR cable pipelines, which are used to improve the matching degree between AR map images and cable pipeline model information.

[0005] In order to achieve the above object, the technical solution of the present invention is as follows: a high-precision fusion positioning system for stable display of AR cable pipelines, comprising:

[0006] Cable pipeline display module: used to build an AR map containing a cable pipeline model. The cable pipeline model is bound to the AR map through positioning coordinates. The cable pipeline display module is equipped with an accelerometer and a gyroscope for collecting inertial vectors. The cable pipeline display module is used to input positioning coordinates and inertial vectors, and output the cable pipeline model in the AR map; ORB-SLAM3-L correction module: used to obtain AR map images and correct them. By initializing monocular or binocular SLAM and running it in the shortest time, a map consisting of k camera poses and hundreds of points is obtained. The map is optimized using visual BA only. The camera pose is converted to the context of the map for reference to obtain the trajectory T - 0:k =[R, p - ] 0:k , where T - With p - Variable used to identify the next scale;

[0007] Using T - Inertial measurements between 0:k and keyframes estimate the inertial variables:

[0008]

[0009] Where s is the visual-only scale factor, R is a rotation matrix used to calculate the gravity vector in the reference frame, g = R wg gI , gI=(0,0,G)T, G is the magnitude of gravity; b=(b a ,b g )∈R6 is the bias of the accelerometer and gyroscope, which is a constant in the initialization stage. is the body velocity from the first keyframe to the last keyframe;

[0010] Consider only the inertial measurement set I 0:k = {I 0,1 ...I k-1,k}, the maximized posterior distribution of the MAP estimation problem is stated as:

[0011]

[0012] Where p(I 0:k |Y k ) represents the likelihood estimate, p(Y k ) represents the prior value;

[0013] Considering the independence of measurements, the inertial MAP estimation problem alone is:

[0014]

[0015] Taking the negative logarithm and assuming IMU pre-integration and Gaussian errors of the prior distribution, this ultimately leads to the optimization problem:

[0016]

[0017] The latest trajectory estimated by visual SLAM is treated as a constant, and a priori residual is added to force the IMU bias to be close to zero. The range of values ​​of the IMU bias is predetermined by the covariance matrix ∑b. When formally estimating, a contraction is defined to update R during the estimation optimization process. wg , since there is a rotation around the direction of gravity without assuming that gravity has changed, the update uses two angles (δα g , δβ g ) to parameterize:

[0018]

[0019] In order to ensure that the scale factor remains greater than 0 during the optimization process, its update is defined as:

[0020] snew =s old exp(δs)

[0021] After the inertial optimization is completed, the frame pose, velocity, and 3D map points are scaled by the estimated scale factor and rotated to align the z-axis with the estimated gravity direction. The bias is updated and the IMU pre-integration is repeated to reduce future linearization errors and perform joint visual-inertial optimization.

[0022] The above scheme has the following beneficial effects:

[0023] 1. In this solution, ORB-SLAM3-L is used for correction. Its core method is to obtain data through the IMU unit, and use the attitude, velocity, and position differential equations to derive solutions and convert them into approximate forms under discrete time. This makes it possible to solve navigation information under discrete time sampling. Using the angular velocity and acceleration measured by the IMU, the navigation information at the current moment is calculated based on the navigation information at the previous moment, including attitude solution, velocity solution, position solution, etc., to achieve a more accurate fit of SLAM to the real world space, thereby achieving accurate display of AR cable pipelines.

[0024] 2. In this scheme, a scale error of 5% is achieved under a 2-second trajectory. In order to further improve the initial estimate, visual inertial BA is performed a few seconds or tens of seconds after initialization, converging to a scale error of 1%. After these BAs, the overall AR spatial map can be confirmed that the scale, IMU parameters, gravity direction, etc. have been accurately estimated, and the AR space is reliable.

[0025] Furthermore, it also includes a high-precision positioning module, which is used to input positioning coordinates into the cable pipeline display module.

[0026] Beneficial effects: In addition to optimizing the AR map to improve accuracy, the positioning coordinates of the input cable pipeline display module can also be optimized to improve the accuracy of the paired AR map.

[0027] Furthermore, the high-precision positioning module includes a north-pointing calculation device, which is provided with an independent plug-in isolated from the magnetic force, and the independent plug-in isolated from the magnetic force is used for positioning correction in the true north direction.

[0028] Beneficial effect: The magnetic field in the environment will affect the north-guiding effect, so by designing an independent plug-in that isolates the magnetic force, the magnetic interference is reduced and the north-guiding effect is corrected.

[0029] Furthermore, the high-precision positioning module includes an RTK mode: by receiving orbital information and time information from satellite data, a formula is established using the distance and known coordinates of each satellite to calculate the current precise coordinates of RTK. As the number of received satellites increases, the known quantities increase, thereby improving the positioning accuracy of RTK.

[0030] Beneficial effects: RTK mode is based on calculations of multiple satellites. The calculation accuracy is improved according to the satellites that can be received. It is suitable for most scenarios, but it has high requirements on the hardware performance of the ground receiving end.

[0031] Furthermore, the high-precision positioning module includes a CORS mode: used to correct the positioning data to ensure the accuracy of the fixed solution calculation results; the CORS mode is based on a preset ground reference station, and uses a continuously running reference station established by real-time dynamic differential technology of a multi-base station network to measure the position, and calculates the positioning coordinates based on the coordinates of the known reference station.

[0032] Beneficial effects: CORS mode can perform signal turnover through ground reference stations, without direct communication with satellites, and directly calculate positioning coordinates based on the coordinates of ground reference stations. Ground reference stations can reduce the difficulty of communication, but the cost of establishing ground reference stations is high, and they are suitable for deployment in downtown areas.

[0033] Furthermore, in the cable pipeline display module, the positioning coordinates use WGS84 geodetic coordinates.

[0034] Beneficial effect: The origin of the WGS84 coordinate system is the center of mass of the earth, and the Z axis of the spatial rectangular coordinate system points to the direction of the Earth's pole (CTP) defined by BIH (1984.0), that is, the internationally agreed origin CIO. Using the WGS84 coordinate system can standardize the coordinate system.

[0035] A high-precision fusion positioning method for the stable display of AR cable pipelines is proposed. The AR map image is corrected by the ORB-SLAM3-L correction module, and the high-precision positioning module is used to obtain and input the positioning coordinates.

[0036] Beneficial effect: This method can improve the stability of AR map image maps and enhance the cable pipeline display accuracy in terms of AR map image map accuracy and input positioning coordinate accuracy.

[0037] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 It is a schematic diagram of an embodiment of a high-precision fusion positioning system and method for stable display of AR cable pipelines according to the present invention. DETAILED DESCRIPTION

[0039] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0040] In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicating the orientation or positional relationship, are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", and "third" are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance.

[0041] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0042] The following is further described in detail through specific implementation methods:

[0043] As attached Figure 1 Shown: High-precision fusion positioning system for stable display of AR cable pipelines, including:

[0044] Cable pipeline display module: used to build a cable pipeline model. The cable pipeline model is bound to the map in the AR map image through positioning coordinates. The cable pipeline display module is used to input positioning coordinates and output the cable pipeline model in the display map. The positioning coordinates use WGS84 geodetic coordinates.

[0045] ORB-SLAM3-L correction module: used to obtain AR map images and correct them. By initializing monocular or binocular SLAM and running it in the shortest time, a map consisting of k camera poses and hundreds of points is obtained. The map is optimized using visual BA only. The camera pose is converted to the context of the map for reference to obtain the trajectory T - 0: k = [R, p - ]0:k, where T - With p - Variable used to identify the next scale;

[0046] Using T - Inertial measurements between 0:k and keyframes estimate the inertial variables:

[0047]

[0048] Where s is the visual scale factor, R is a rotation matrix used to calculate the gravity vector in the reference frame, g = R wg gI , gI=(0,0,G)T, G is the magnitude of gravity; b=(b a ,b g )∈R6 is the bias of the accelerometer and gyroscope, which is a constant in the initialization stage, v - 0:k ∈R3 is the body velocity from the first keyframe to the last keyframe;

[0049] Consider only the inertial measurement set I 0:k = {I 0,1 ...I k-1,k}, the maximized posterior distribution of the MAP estimation problem is stated as:

[0050]

[0051] Where p(I 0:k |Y k ) represents the likelihood estimate, p(Y k ) represents the prior value;

[0052] Considering the independence of measurements, the inertial MAP estimation problem alone is:

[0053]

[0054] Taking the negative logarithm and assuming IMU pre-integration and Gaussian errors of the prior distribution, this ultimately leads to the optimization problem:

[0055]

[0056] The latest trajectory estimated by visual SLAM is treated as a constant, and a priori residual is added to force the IMU bias to be close to zero. The range of values ​​of the IMU bias is predetermined by the covariance matrix ∑b. When formally estimating, a contraction is defined to update R during the estimation optimization process. wg , since there is a rotation around the direction of gravity without assuming that gravity has changed, the update uses two angles (δα g , δβ g ) to parameterize:

[0057]

[0058] In order to ensure that the scale factor remains greater than 0 during the optimization process, its update is defined as:

[0059] s new =s old exp(δs)

[0060] After the inertial optimization is completed, the frame pose, velocity, and 3D map points are scaled by the estimated scale factor and rotated to align the z-axis with the estimated gravity direction. The bias is updated and the IMU pre-integration is repeated to reduce future linearization errors and perform joint visual-inertial optimization.

[0061] High-precision positioning module: The high-precision positioning module is used to input the positioning coordinates into the cable pipeline display module. The high-precision positioning module includes a north-pointing calculation device. The north-pointing calculation device is provided with an independent plug-in isolated from the magnetic force. The independent plug-in isolated from the magnetic force is used for positioning correction in the true north direction.

[0062] The high-precision positioning module includes the RTK mode: by receiving orbital information and time information from satellite data, a formula is established using the distance and known coordinates of each satellite to calculate the current precise coordinates of RTK. As the number of received satellites increases, the known quantity increases, thereby improving the positioning accuracy of RTK.

[0063] The high-precision positioning module includes the CORS mode: used to correct the positioning data to ensure the accuracy of the fixed solution calculation results; the CORS mode is based on a preset ground reference station, using a continuously running reference station established by real-time dynamic differential technology of a multi-base station network to measure the position, and calculates the positioning coordinates based on the coordinates of the known reference station.

[0064] When in use, the AR augmented reality technology modified by ORB-SLAM3-L is used to combine the high-precision positioning GIS data of the cable pipeline, establish a visualization data application for underground cable pipeline inspection, integrate the three-dimensional spatial information of underground cables with the real-time positioning information of mobile terminals and other devices, and superimpose the virtual underground pipeline with the real-space data screen to produce a WYSIWYG real-time display of cable pipelines, so as to accurately find the locations of cables, pipes, work wells, etc. during inspection, construction, rescue and other operations, and provide various basic data information such as the direction, parameters, and burial depth of cables and pipes, thereby greatly improving the efficiency of line inspection. When supervising and guiding non-power construction, there is no need for professionals to follow and check cable drawings on site, which improves work efficiency and ensures the safety of operations.

[0065] The WGS84 geodetic coordinate system is used as the data standard for data normalization. ORB-SLAM3-L is used for correction. Its core method is to obtain data through the IMU unit, and use the attitude, velocity, and position differential equations to derive solutions and convert them into approximate forms under discrete time. This makes it possible to solve navigation information under discrete time sampling. Using the angular velocity and acceleration measured by the IMU, the navigation information at the current moment is calculated based on the navigation information at the previous moment, including attitude solution, velocity solution, position solution, etc., to achieve a more accurate fit of SLAM to the real world space, thereby achieving accurate display of AR cable pipelines.

[0066] In a 2-second trajectory, a 5% scale error was achieved. To further improve the initial estimate, visual-inertial BA was performed a few seconds or tens of seconds after initialization, converging to a 1% scale error. After these BAs, the overall AR spatial map can be confirmed that the scale, IMU parameters, gravity direction, etc. have been accurately estimated, and the AR space is reliable.

[0067] In addition to optimizing AR maps to improve accuracy, the positioning coordinates of the input cable pipeline display module can also be optimized to improve the accuracy of the paired AR map.

[0068] To improve the accuracy of AR maps, we must first determine the camera imaging angle, collect the true north direction through the north-pointing computing device, and then determine the accurate imaging direction in the AR map. The magnetic field in the use environment will affect the north-pointing effect, so we design an independent plug-in that isolates the magnetic force to reduce magnetic interference and correct the north-pointing effect.

[0069] The high-precision positioning module has multiple modes based on different usage scenarios. The RTK mode performs calculations based on multiple satellites. The calculation accuracy is improved according to the number of satellites that can be received. It is suitable for most scenarios, but it has high requirements on the hardware performance of the ground receiving end and the mobile device is larger.

[0070] The CORS mode can perform signal turnover through ground reference stations, without direct communication with satellites, and directly calculate positioning coordinates based on the coordinates of ground reference stations. Ground reference stations can reduce the difficulty of communication, but the cost of establishing ground reference stations is high, and they are suitable for deployment in frequent application scenarios such as urban areas.

[0071] A high-precision fusion positioning method for the stable display of AR cable pipelines is proposed. The AR map image is corrected by the ORB-SLAM3-L correction module, and the high-precision positioning module is used to obtain and input the positioning coordinates.

[0072] This method can improve the stability of AR map image maps and enhance the display accuracy of cable pipelines in terms of AR map accuracy and input positioning coordinate accuracy.

[0073] Obviously, the above embodiments are merely examples for the purpose of clear explanation, and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived therefrom are still within the scope of protection of the invention.

Claims

1. A high-precision fusion positioning system for stable display of AR underground cables and pipelines, characterized by: include: Cable pipeline display module: used to build an AR map of the cable pipeline model containing the current high-precision positioning position. The cable pipeline model is bound to the AR map through high-precision positioning coordinates. The underground cable pipeline display module is equipped with an accelerometer and a gyroscope for collecting inertial vectors. The cable pipeline display module is used to input positioning coordinates and inertial vectors, and output the cable pipeline model in the AR map that displays the current position and viewing angle; ORB-SLAM3-L correction module: used to obtain the AR map space and the current image of the terminal device camera and correct its positioning to stabilize the target position offset caused by the error generated in the movement of the AR space. By initializing the monocular or binocular camera and collecting the SLAM space feature points, and running the SLAM space in the shortest time through the ROI area principle, a map consisting of k camera positions and postures and hundreds of points is obtained. The map is optimized using visual BA only. The camera position and posture are converted to the context of the map for reference to obtain the trajectory T - 0:k =[R, p - ] 0:k , where T - With p - Variable used to identify the next scale; Using T - Inertial measurements between 0:k and keyframes estimate the inertial variables: Where s is the visual-only scale factor, R is a rotation matrix used to calculate the gravity vector in the reference frame, g = R wg , gI=(0,0,G)T, G is the magnitude of gravity; b=(b a ,b g )∈R6 is the bias of the accelerometer and gyroscope, which is a constant in the initialization stage. is the body velocity from the first keyframe to the last keyframe; Consider only the inertial measurement set I 0:k = {I 0,1 ...I k-1,k }, the maximized posterior distribution of the MAP estimation problem is stated as: Where p(I 0:k |Y k ) represents the likelihood estimate, p(Y k ) represents the prior value; Considering the independence of measurements, the inertial MAP estimation problem alone is: Taking the negative logarithm and assuming IMU pre-integration and Gaussian errors of the prior distribution, this ultimately leads to the optimization problem: The latest trajectory estimated by visual SLAM is treated as a constant, and a priori residual is added to force the IMU bias to be close to zero. The range of values ​​of the IMU bias is predetermined by the covariance matrix ∑b. When formally estimating, a contraction is defined to update R during the estimation optimization process. wg , since there is a rotation around the direction of gravity without assuming that gravity has changed, the update uses two angles (δα g ,δβ g ) to parameterize: In order to ensure that the scale factor remains greater than 0 during the optimization process, its update is defined as: s new =s old exp(δs) After the inertial optimization is completed, the frame pose, velocity, and 3D map points are scaled according to the estimated scale factor and rotated to align the z-axis with the estimated gravity direction. The bias is updated and the IMU pre-integration is repeated to reduce future linearization errors. Joint visual-inertial optimization is performed to achieve a stable display of the AR space and correct the target offset caused by movement.

2. The high-precision fusion positioning system for stable display of AR cable pipelines according to claim 1 is characterized in that: It also includes a high-precision positioning module, which is used to input positioning coordinates into the cable pipeline display module.

3. The high-precision fusion positioning system for stable display of AR cable pipelines according to claim 2 is characterized in that: The high-precision positioning module includes a north-pointing calculation device, which is provided with an independent plug-in isolated from magnetic force, and the independent plug-in isolated from magnetic force is used for positioning correction in the true north direction.

4. The high-precision fusion positioning system for stable display of AR cable pipelines according to claim 3 is characterized in that: The high-precision positioning module includes the RTK mode: by receiving orbital information and time information from satellite data, a formula is established using the distance and known coordinates of each satellite to calculate the current precise coordinates of RTK. As the number of received satellites increases, the known quantity increases, thereby improving the positioning accuracy of RTK.

5. The high-precision fusion positioning system for stable display of AR cable pipelines according to claim 4 is characterized in that: The high-precision positioning module includes the CORS mode: used to correct the positioning data to ensure the accuracy of the fixed solution calculation results; the CORS mode is based on a preset ground reference station, using a continuously running reference station established by real-time dynamic differential technology of a multi-base station network to measure the position, and calculates the positioning coordinates based on the coordinates of the known reference station.

6. The high-precision fusion positioning system for stable display of AR cable pipelines according to claim 5 is characterized in that: In the cable pipeline display module, the positioning coordinates use WGS84 geodetic coordinates.

7. A high-precision fusion positioning method for stable display of AR cable pipelines, based on the method of the high-precision fusion positioning system for stable display of AR cable pipelines of claims 1-6, characterized in that: The AR map image is corrected by the ORB-SLAM3-L correction module, and the high-precision positioning module is used to obtain and input the positioning coordinates.

Images