North pointing calibration system and method in complex outdoor environment

Abstract

The invention relates to the technical field of high-precision positioning, in particular to a north-pointing calibration system and method in a complex outdoor environment, and the system comprises a sensor collection module which is used for collecting images, motion states, real-time positions and geomagnetic information through vision, inertial navigation, a GPS and a magnetometer; the data processing module is used for processing image information, converting coordinates and calculating poses; the preliminary calibration module adopts an ORB-SLAM3 framework and geomagnetic information to generate preliminary north data; the satellite north-pointing module calculates satellite north-pointing data through an RTK or CORS mode; the vision-inertial navigation combined module is combined with multi-source information to generate direction estimation; the comprehensive calibration north-pointing module fuses the primary north-pointing data and the secondary north-pointing data to obtain a final result; and the communication module communicates with the PDA through an IEEE802.11b protocol, sends a state variable, final north-pointing data and satellite north-pointing data, and performs north-pointing calibration on the PDA. The method does not depend on a sensor and a processor of the PDA for calculation, the calculation pressure of the PDA can be relieved, and the north-pointing calibration precision in a complex outdoor environment is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of high-precision positioning, and in particular to a north reference calibration system and method in a complex outdoor environment. Background Art

[0002] At present, the management, inspection, maintenance and other operations of various types of power cables, such as high-voltage and low-voltage cables, especially underground cable pipelines, basically rely on the experience of practitioners and their familiarity with the layout and filling of cable networks in the responsible area. Therefore, the professional quality of practitioners and personnel changes will directly affect the efficiency, safety and timeliness of the operation, maintenance, repair and rescue of power cable pipelines, and even affect the safe operation of the entire power grid. However, due to the different management and digitalization levels in various regions, there are basically problems such as incomplete cable pipeline data, different drawing document formats, inconsistent data versions, lack of data marking, etc., which cause low operation efficiency and maintenance difficulties. Therefore, the SAR augmented reality technology modified by ORB-SLAM3 is combined with the high-precision positioning GIS data of the cable pipeline to establish a visualization data application system for underground cable pipeline inspection, which integrates the three-dimensional spatial information of the underground cable with the real-time positioning information of the mobile terminal, and superimposes the virtual underground pipeline with the real-space data screen to produce a WYSIWYG real-time display of the cable pipeline, so as to accurately find the locations of cables, pipes, work wells, etc. during inspection, construction, rescue and other operations, and provide various basic data information such as the direction, parameters, and burial depth of cables and pipes, thereby greatly improving the efficiency of line inspection. When supervising and guiding non-power construction, there is no need for professionals to follow and check cable drawings on site, which improves work efficiency and ensures the safety of operations.

[0003] The SAR augmented reality technology corrected by ORB-SLAM3 brings users an unprecedented efficient and convenient cable pipeline inspection experience. Before the implementation of SAR augmented reality applications, the basic north-pointing computing devices such as the magnetometer, electronic compass, and gyroscope of the PDA mobile terminal play an indispensable role. In addition, the PDA also uses a unique independent plug-in that isolates magnetic force and is compatible with physical magnetometers to perform north-direction positioning corrections to ensure accurate calibration of electronic compasses, gyroscopes and other equipment.

[0004] In the existing technology, the north calibration system in complex outdoor environments faces many challenges and defects, which directly affect the application effect of SAR augmented reality technology based on ORB-SLAM3 correction in power cable pipeline inspection. Although PDA mobile terminals are equipped with basic north calculation devices such as magnetometers, electronic compasses, and gyroscopes, the accuracy and stability of these devices are often limited by factors such as cost, size, and power consumption. In complex outdoor environments, the accuracy of these devices may not meet the needs of high-precision positioning.

[0005] In summary, how to solve the accuracy and stability of the north calibration system used by PDA mobile terminals in SAR augmented reality technology based on ORB-SLAM3 correction is often limited by factors such as cost, size, and power consumption. In complex outdoor environments, the accuracy of these devices may not meet the needs of high-precision positioning, which has become a difficult problem that needs to be solved urgently in this field. Therefore, it is necessary to propose a north calibration system and method in complex outdoor environments. Summary of the invention

[0006] To solve the above problems, the present invention provides a north calibration system and method in a complex outdoor environment. The present invention does not rely on the PDA's own sensors and processors for calculations, can reduce the calculation pressure of the PDA, improve the north calibration accuracy in a complex outdoor environment, and can meet the needs of high-precision positioning.

[0007] In order to achieve the above object, the technical solution of the present invention is as follows: A north reference calibration system in a complex outdoor environment, comprising:

[0008] Sensor acquisition module, including visual sensor, inertial navigation sensor, GPS positioning sensor and magnetometer.

[0009] Visual sensors are used to collect image information in outdoor environments; inertial navigation sensors are used to collect motion status information in outdoor environments, and obtain real-time speed information and real-time attitude information through integration operations; GPS positioning sensors are used to collect satellite data and real-time position projection coordinate map information in outdoor environments; magnetometers are used to collect real-time geomagnetic information in outdoor environments.

[0010] The data processing module is used to generate feature point parameters and landmark parameters for image information in outdoor environments using image processing algorithms; and to convert real-time position projection coordinate map information into the WGS84 geodetic coordinate system using Gaussian inverse calculation; and to calculate the body posture using real-time speed information and real-time attitude information to generate state variables.

[0011] The preliminary calibration module is used to use the ORB-SLAM3 framework to initialize the IMU using real-time velocity information and real-time attitude information to generate initial attitude information, initial velocity information, initial position information and initial IMU bias; and then combine the geomagnetic information with the initial attitude information, initial velocity information, initial position information and initial IMU bias to generate preliminary north calibration data.

[0012] The satellite north pointing module is used to calculate the satellite data and generate the satellite north pointing calibration data using one of the RTK mode and CORS differential positioning mode.

[0013] The visual-inertial navigation joint module is used to use the visual-inertial navigation joint function of the ORB-SLAM3 framework to combine feature point parameters, landmark parameters, real-time speed information and real-time attitude information to generate the direction information estimated by the ORB-SLAM3 framework;

[0014] The integrated calibration north module projects the direction information estimated by the ORB-SLAM3 framework with the WGS84 geodetic coordinate system to generate secondary north calibration data; the preliminary north calibration data and the secondary north calibration data are combined to generate the final north calibration data.

[0015] The communication module is used to communicate with the PDA using the IEEE802.11b communication protocol, and sends the state variables, the final north calibration data and the satellite's north calibration data to the PDA. The PDA uses the state variables, the final north calibration data and the PDA's own posture information to perform north calibration.

[0016] Furthermore, in the sensor acquisition module, the inertial navigation sensor includes an accelerometer and a gyroscope.

[0017] Furthermore, in the data processing module, the image processing algorithm includes the following steps:

[0018] Step 1: Edge detection: Use the Canny edge detection algorithm to perform edge detection on the collected outdoor environment image and generate edge features.

[0019] Step 2: Feature extraction: Using the SIFT feature extraction algorithm, edge features are extracted to obtain feature points for direction determination, and corresponding feature point parameters are generated.

[0020] Step 3: Image matching: Match the extracted feature points with the feature points in other images collected in real time, find the most similar feature point pairs by comparing the similarity of the SIFT descriptors of the feature points, and generate landmark parameters.

[0021] Furthermore, in the data processing module, Gaussian backcalculation includes the following calculation formula:

[0022]

[0023]

[0024] Where: B is latitude, I is longitude, B f is the latitude of the base point, η f is e′cosB f , V f is (1+η f 2 ) 1 / 2 , t f tanBf , N f for a(1-e 2 sin 2 B f ) 1 / 2 , y represents one of the source coordinate system longitude and the y coordinate system.

[0025] The inverse calculation formula of meridian arc length is:

[0026]

[0027] Dividing both sides by a0 gives:

[0028]

[0029] make:

[0030] Obtain: β=B+P2sin2B+P4sin4B+P5sin6B (6).

[0031] Substituting the trigonometric series back into the formula we get:

[0032]

[0033] q4=-P4+P2 2 -2P2P6+4P2 2 P4 (8).

[0034]

[0035] B f =β+q2sin2β+q4sin4β+q6sin6β (10).

[0036] η f =e′cosB f (11).

[0037] V f =(1+η f 2 ) 1 / 2 (12).

[0038] t f =tanB f (13).

[0039] N f =a(1-e 2 sin 2 B f ) -1 / 2 (14).

[0040] Furthermore, in the data processing module, the body posture calculation includes the following calculation formula:

[0041]

[0042] Where Si is the state variable, Vi is the real-time speed information, Ti is the time information, b i g is the gyroscope bias, b i a is the accelerometer bias.

[0043] Furthermore, in the preliminary calibration module, the initialization operation of the IMU includes the following steps:

[0044] Step 1: Estimate the noise parameters of real-time velocity information and real-time attitude information.

[0045] Step 2: Estimate and correct the deviation between the real-time velocity information and the real-time attitude information.

[0046] Step three, calibrate the scale factors of the accelerometer and gyroscope.

[0047] Step 4: Perform IMU alignment to determine the initial posture of the IMU.

[0048] Step 5: Initialize the IMU using the estimated noise parameters, bias, scale factor, and initial pose.

[0049] Furthermore, in the satellite north guidance module, the RTK mode includes the following steps:

[0050] Step 1: receiving satellite orbit information and time information from satellite data.

[0051] Step 2: Use the distance between two adjacent satellites and the known coordinates of each satellite to calculate the current precise coordinates of RTK using the real-time difference method.

[0052] Step three: Combine the precise coordinates with geomagnetic information to generate the satellite's north calibration data.

[0053] Furthermore, in the satellite north guidance module, the CORS differential positioning mode includes the following steps:

[0054] Step 1: Receive differential information and position information from several continuously operating reference stations through the CORS network.

[0055] Step 2: Use the differential information and position information of the reference station to calculate the positioning result.

[0056] Step three: Combine the positioning results with geomagnetic information to generate the satellite's north reference calibration data.

[0057] Furthermore, the method for calibrating the north in a complex outdoor environment is based on the operation of the north calibration system in the complex outdoor environment, and includes the following steps:

[0058] Step 1: Data collection: Use visual sensors, inertial navigation sensors, GPS positioning sensors and magnetometers to collect image information, motion status information, real-time position projection coordinate map information and real-time geomagnetic information in the outdoor environment.

[0059] Step 2, data processing: Apply image processing algorithms to the collected image information to generate feature point parameters and landmark parameters; convert the real-time position projection coordinate map information into the WGS84 geodetic coordinate system; and use the real-time speed information and real-time attitude information to calculate the body posture and generate state variables.

[0060] Step 3: Preliminary calibration: Use the ORB-SLAM3 framework to initialize the IMU for the real-time velocity information and real-time attitude information to generate initial attitude information, velocity information, position information and IMU bias; then combine the geomagnetic information to generate preliminary north calibration data.

[0061] Step 4: Satellite north calibration: Select one of the RTK mode and CORS differential positioning mode, calculate the satellite data, and generate the satellite north calibration data.

[0062] Step 5: Visual-Inertial Navigation Joint Calibration: Using the visual-inertial navigation joint function of the ORB-SLAM3 framework, combined with feature point parameters, landmark parameters, real-time speed information and real-time attitude information, the direction information estimated by the ORB-SLAM3 framework is generated.

[0063] Step 6: Comprehensive calibration: Project the direction information estimated by the ORB-SLAM3 framework with the WGS84 geodetic coordinate system to generate secondary north calibration data; and combine the preliminary north calibration data with the secondary north calibration data to generate the final north calibration data.

[0064] Step 7, data communication: Use the IEEE 802.11b communication protocol to send the state variables, final northing calibration data and satellite northing calibration data to the PDA for the PDA to perform northing calibration.

[0065] Furthermore, in step seven, the satellite's north calibration data is used to perform multi-source data fusion on the PDA.

[0066] The above scheme has the following beneficial effects:

[0067] 1. The present invention realizes high-precision calibration of the north direction in complex outdoor environments through visual sensors, inertial navigation sensors, GPS positioning sensors, magnetometers and data processing algorithms. This comprehensive method not only improves the accuracy of calibration, but also enhances the robustness of the system, enabling it to adapt to various complex environments.

[0068] 2. The present invention generates accurate north-pointing calibration data by utilizing the visual-inertial navigation joint function of the ORB-SLAM3 framework and combining the real-time collected image information, motion state information and position information. This method not only improves the accuracy of the calibration data, but also reduces the computing pressure on the PDA and improves the operating efficiency of the overall system.

[0069] 3. The present invention processes satellite data by adopting RTK mode or CORS differential positioning mode to generate reliable satellite north calibration data. At the same time, by integrating multiple calibration data (including preliminary north calibration data, secondary north calibration data and satellite north calibration data), the final north calibration data is generated, further improving the accuracy and reliability of calibration. In addition, the calibration data is sent to the PDA using the IEEE 802.11b communication protocol, realizing real-time transmission and sharing of data, and providing strong support for the north calibration of the PDA.

[0070] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] Figure 1 Schematic diagram of the structure of the north calibration system in a complex outdoor environment according to an embodiment of the present invention.

[0072] Figure 2 Schematic diagram of the steps of a northing calibration method in a complex outdoor environment according to an embodiment of the present invention.

[0073] Figure 3 Schematic diagram of the structure of the ORB-SLAM3 framework in an embodiment of the present invention.

[0074] Figure 4 The figure is a schematic diagram of the interface of SAR patrol of power pipeline data by a PDA equipped with the ORB-SLAM3 framework in an embodiment of the present invention. DETAILED DESCRIPTION

[0075] The following is further described in detail through specific implementation methods:

[0076] Embodiment 1:

[0077] As attached Figure 1As shown in the figure: a north pointing calibration system in a complex outdoor environment is mainly composed of a sensor acquisition module, a data processing module, a preliminary calibration module, a satellite north pointing module, a visual-inertial navigation joint module, a comprehensive calibration north pointing module and a communication module; wherein the sensor acquisition module is mainly used to collect images, motion states, real-time position projection coordinates and geomagnetic information in the outdoor environment; the data processing module is mainly used to perform image processing on the collected image information to generate feature points and landmark parameters; convert the real-time position projection coordinates into the WGS84 geodetic coordinate system; use the real-time speed and attitude information to calculate the body posture and generate state variables; the data processing module is mainly used to use the ORB-SLAM3 framework and the real-time speed and attitude information to initialize the IMU and generate the initial attitude, speed, position and IMU bias information; combine the geomagnetic information to generate preliminary north pointing calibration data; the satellite north pointing module The block is mainly used to calculate satellite data using RTK or CORS differential positioning mode to generate satellite north calibration data; the visual-inertial navigation joint module is mainly used to use the visual-inertial navigation joint function of the ORB-SLAM3 framework to combine image features, landmarks, real-time speed and attitude information to generate direction information estimated by the ORB-SLAM3 framework; the comprehensive calibration north module is mainly used to project the direction information estimated by the ORB-SLAM3 framework with the WGS84 geodetic coordinate system to generate secondary north calibration data; the preliminary north calibration data and the secondary north calibration data are combined to generate the final north calibration data; the communication module is mainly used to communicate with the PDA using the IEEE802.11b communication protocol, and send the state variables, the final north calibration data and the satellite's north calibration data to the PDA so that the PDA can use these data for north calibration.

[0078] The north calibration method based on the north calibration system in the complex outdoor environment includes the following steps:

[0079] Step 1: Data collection: Use visual sensors, inertial navigation sensors, GPS positioning sensors and magnetometers to collect image information, motion status information, real-time position projection coordinate map information and real-time geomagnetic information in the outdoor environment.

[0080] Step 2, data processing: Apply image processing algorithms to the collected image information to generate feature point parameters and landmark parameters; convert the real-time position projection coordinate map information into the WGS84 geodetic coordinate system; and use the real-time speed information and real-time attitude information to calculate the body posture and generate state variables.

[0081] Step 3: Preliminary calibration: Use the ORB-SLAM3 framework to initialize the IMU for the real-time velocity information and real-time attitude information to generate initial attitude information, velocity information, position information and IMU bias; then combine the geomagnetic information to generate preliminary north calibration data.

[0082] Step 4: Satellite north calibration: Select one of the RTK mode and CORS differential positioning mode, calculate the satellite data, and generate the satellite north calibration data.

[0083] Step 5: Visual-Inertial Navigation Joint Calibration: Using the visual-inertial navigation joint function of the ORB-SLAM3 framework, combined with feature point parameters, landmark parameters, real-time speed information and real-time attitude information, the direction information estimated by the ORB-SLAM3 framework is generated.

[0084] Step 6: Comprehensive calibration: Project the direction information estimated by the ORB-SLAM3 framework with the WGS84 geodetic coordinate system to generate secondary north calibration data; and combine the preliminary north calibration data with the secondary north calibration data to generate the final north calibration data.

[0085] Step 7, data communication: Use the IEEE 802.11b communication protocol to send the state variables, final northing calibration data and satellite northing calibration data to the PDA for the PDA to perform northing calibration.

[0086] The following is a detailed explanation of the functions of each module:

[0087] Sensor acquisition module, including visual sensor, inertial navigation sensor, GPS positioning sensor and magnetometer.

[0088] Visual sensors are used to collect image information in outdoor environments; inertial navigation sensors are used to collect motion status information in outdoor environments, and obtain real-time speed information and real-time attitude information through integration operations; GPS positioning sensors are used to collect satellite data and real-time position projection coordinate map information in outdoor environments; magnetometers are used to collect real-time geomagnetic information in outdoor environments.

[0089] In the sensor acquisition module, the inertial navigation sensor includes an accelerometer and a gyroscope.

[0090] The data processing module is used to generate feature point parameters and landmark parameters for image information in outdoor environments using image processing algorithms; and to convert the real-time position projection coordinate map information into the WGS84 geodetic coordinate system using the Gaussian inverse calculation algorithm; and to calculate the body posture using the real-time speed information and real-time attitude information to generate state variables.

[0091] In the data processing module, the image processing algorithm includes the following steps:

[0092] Step 1: Edge detection: Use the Canny edge detection algorithm to perform edge detection on the collected outdoor environment image and generate edge features.

[0093] Step 2: Feature extraction: Using the SIFT feature extraction algorithm, feature points for direction determination are extracted from edge features and corresponding feature point parameters are generated.

[0094] Step 3: Image matching: Match the extracted feature points with the feature points in other images collected in real time, find the most similar feature point pairs by comparing the similarity of the SIFT descriptors of the feature points, and generate landmark parameters.

[0095] In the data processing module, Gaussian backcalculation includes the following calculation formula:

[0096]

[0097]

[0098] Where: B is latitude, I is longitude, B f is the latitude of the base point, η f is e′cosB f , V f is (1+η f 2 ) 1 / 2 , t f tanB f , N f for a(1-e 2 sin 2 B f ) 1 / 2 , y represents one of the source coordinate system longitude and the y coordinate system.

[0099] The inverse calculation formula of meridian arc length is:

[0100]

[0101] Dividing both sides by a0 gives:

[0102]

[0103] make:

[0104] So: β=B+P2sin2B+P4sin4B+P6sin6B (6).

[0105] Substituting the trigonometric series back into the formula we get:

[0106]

[0107] q4=-P4+P2 2 -2P2P6+4P2 2 P4 (8).

[0108]

[0109] B f =β+q2sin2β+q4sin4β+q6sin6β (10).

[0110] η f =e′cosB f (11).

[0111] V f =(1+η f 2 ) 1 / 2 (12).

[0112] t f =tanB f (13).

[0113] N f =a(1-e 2 sin 2 B f ) 1 / 2 (14).

[0114] In the data processing module, the body posture calculation includes the following calculation formula:

[0115]

[0116] Where Si is the state variable, Vi is the real-time speed information, Ti is the real-time time information, b i g is the gyroscope bias, b i a is the accelerometer bias.

[0117] The preliminary calibration module is used to use the ORB-SLAM3 framework to initialize the IMU using real-time velocity information and real-time attitude information to generate initial attitude information, initial velocity information, initial position information and initial IMU bias; and then combine the geomagnetic information with the initial attitude information, initial velocity information, initial position information and initial IMU bias to generate preliminary north calibration data.

[0118] In the preliminary calibration module, the initialization operation of the IMU includes the following steps:

[0119] Step 1: Estimate the noise parameters of real-time velocity information and real-time attitude information.

[0120] Step 2: Estimate and correct the deviation between the real-time velocity information and the real-time attitude information.

[0121] Step three, calibrate the scale factors of the accelerometer and gyroscope.

[0122] Step 4: Perform IMU alignment to determine the initial posture of the IMU.

[0123] Step 5: Initialize the IMU using the estimated noise parameters, bias, scale factor, and initial pose.

[0124] Specifically, first, use a device equipped with the present north-compassing calibration system in a complex outdoor environment to collect image information, motion status information, satellite data, real-time position projection coordinate map information and geomagnetic information in the outdoor environment through visual sensors, GPS positioning sensors and magnetometers.

[0125] After receiving all the data collected by the sensor acquisition module, the data processing module will first process the image information. The edge detection step in the image processing algorithm and the Canny edge detection algorithm are used to perform edge detection on the collected outdoor environment image, thereby identifying the edge features in the image. The edge features are the basis for subsequent feature extraction.

[0126] Next, the data processing module uses the SIFT feature extraction algorithm to extract key points from edge features and generate corresponding feature point parameters. Feature point parameters include the location, direction and scale information of feature points.

[0127] After feature extraction, the data processing module will match the feature points extracted from the current image with the feature points in other previously collected images. By comparing the similarity of the SIFT descriptors of the feature points, the module can find the most similar feature point pairs and generate landmark parameters based on these feature point pairs. The landmark parameters describe the relative position and posture relationship between different images and are the basis for building environmental maps and performing navigation positioning.

[0128] In addition to processing image information, the data processing module also converts the real-time position projection coordinate map information. Using the Gaussian inverse calculation formula, the module converts the real-time position projection coordinates into coordinates in the WGS84 geodetic coordinate system; this conversion process is crucial for fusing and calibrating the north calibration data with the Global Positioning System (GPS).

[0129] At the same time, the data processing module will also use the real-time velocity information and real-time attitude information to calculate the body posture; by combining the gyroscope bias and accelerometer bias, the data processing module can generate the state variables of the current position and attitude of the equipment of the north calibration system in complex outdoor environments.

[0130] In the preliminary calibration module, the real-time velocity and attitude information are initialized to the IMU using the ORB-SLAM3 framework. During this process, the preliminary calibration module estimates and corrects the noise parameters and deviations of the real-time velocity information and real-time attitude information, and corrects the scale factors of the accelerometer and gyroscope. After completing these steps, the preliminary calibration module determines the initial attitude of the IMU and initializes the IMU using the estimated noise parameters, deviations, scale factors, and initial attitude.

[0131] Finally, the preliminary calibration module combines the geomagnetic information with the initial attitude information, initial velocity information, initial position information and initial IMU bias to generate preliminary north calibration data, which will serve as the basis for the subsequent comprehensive calibration north module to perform secondary north calibration.

[0132] The visual-inertial navigation joint module is used to utilize the visual-inertial navigation joint function of the ORB-SLAM3 framework to combine feature point parameters, landmark parameters, real-time speed information and real-time attitude information to generate direction information estimated by the ORB-SLAM3 framework.

[0133] The integrated calibration north module projects the direction information estimated by the ORB-SLAM3 framework with the WGS84 geodetic coordinate system to generate secondary north calibration data; the preliminary north calibration data and the secondary north calibration data are combined to generate the final north calibration data.

[0134] The communication module is used to communicate with the PDA using the IEEE802.11b communication protocol, and sends the state variables, the final north calibration data and the satellite's north calibration data to the PDA. The PDA uses the state variables, the final north calibration data and the PDA's own posture information to perform north calibration.

[0135] Specifically, first, when the device equipped with the present north calibration system in a complex outdoor environment generates preliminary north calibration data, the data can be sent to the PDA through the communication module for north calibration in an outdoor environment with less interference. When high-precision and complex outdoor environment north calibration is required, the system will further utilize the functions of the visual-inertial navigation joint module and the comprehensive calibration north guidance module.

[0136] For example, when a device equipped with this complex outdoor environment north calibration system is in an environment such as an urban canyon, densely packed buildings, or complex terrain, the GPS signal may be blocked or interfered with, resulting in a decrease in the accuracy of the satellite north guidance data. At this time, the visual-inertial navigation joint module will play an important role. The visual-inertial navigation joint module will use the visual-inertial navigation joint function of the ORB-SLAM3 framework to closely combine image features, landmarks, real-time speed and attitude information to generate more accurate and stable direction information estimated by the ORB-SLAM3 framework. The direction information can make up for the accuracy loss caused by insufficient GPS signals and improve the reliability of north calibration.

[0137] Next, the integrated calibration north module will project the direction information estimated by the ORB-SLAM3 framework with the WGS84 geodetic coordinate system to generate secondary north calibration data. This step further integrates visual and inertial navigation information to improve the accuracy and robustness of the north calibration data. At the same time, the integrated calibration north module will also combine the preliminary north calibration data with the secondary north calibration data, and generate the final north calibration data through the data processing process. This data not only contains the current position and attitude information of the device, but also has undergone multiple verifications and corrections to ensure its high accuracy and reliability.

[0138] Finally, the communication module sends the state variables and the final north calibration data to the PDA. After receiving this data, the PDA will use its own position information and algorithms for further fusion and processing to achieve more accurate north calibration.

[0139] The satellite north pointing module is used to calculate the satellite data and generate the satellite north pointing calibration data using one of the RTK mode and CORS differential positioning mode.

[0140] In the satellite north guidance module, the RTK mode includes the following steps:

[0141] Step 1: receiving satellite orbit information and time information from satellite data.

[0142] Step 2: Use the distance between two adjacent satellites and the known coordinates of each satellite to calculate the current precise coordinates of RTK using the real-time difference method.

[0143] Step three: Combine the precise coordinates with geomagnetic information to generate the satellite's north calibration data.

[0144] The CORS differential positioning mode includes the following steps:

[0145] Step 1: Receive differential information and position information from several continuously operating reference stations through the CORS network.

[0146] Step 2: Use the differential information and position information of the reference station to calculate the positioning result.

[0147] Step three: Combine the positioning results with geomagnetic information to generate the satellite's north reference calibration data.

[0148] When the inertial navigation sensor fails, the satellite north guidance module in the device equipped with the complex outdoor environment north guidance calibration system starts to start. For example, assuming that the inertial navigation sensor fails due to physical damage, electromagnetic interference or data abnormality and can no longer provide reliable motion status information, the system will identify the failure of the inertial navigation sensor and automatically switch to the satellite north guidance module as the main north guidance calibration method.

[0149] The satellite north guidance module uses the GPS positioning sensor to receive data from satellites; when using RTK mode, it receives satellite orbit information and time information and uses real-time differential method to calculate the current precise coordinates of the device. This process takes into account the precise distance between the satellite and the device and the known coordinates of each satellite to obtain a high-precision positioning result. The satellite north guidance module then combines this precise coordinate with the geomagnetic information to generate the satellite's north guidance calibration data.

[0150] When using CORS differential positioning mode, differential information and position information from several continuously operating reference stations are received through the CORS network. This information will be used to calculate the positioning result of the device, which will also be combined with geomagnetic information to generate satellite north calibration data.

[0151] During an inertial navigation sensor failure, the satellite north guidance module will continue to provide stable north guidance calibration data to ensure that the device can maintain the correct navigation direction. At the same time, the system will continue to monitor the status of the inertial navigation sensor. Once the failure is repaired or the sensor resumes normal operation, the system can seamlessly switch back to the inertial navigation sensor and the visual-inertial navigation joint module to provide more comprehensive and accurate navigation information.

[0152] In addition, even in the absence of inertial navigation sensor failure, the satellite north guidance module can also serve as a backup or auxiliary means to provide additional north guidance calibration data to enhance the robustness and reliability of the entire system. For example, in an environment where the GPS signal is weak or interfered, the satellite north guidance module can provide more reliable north guidance information to help the device maintain the correct navigation direction.

[0153] Embodiment 2:

[0154] The north pointing calibration system and north pointing calibration method in the above-mentioned complex outdoor environment are applied to the SAR line inspection system based on ORB-SLAM3 and augmented reality to realize high-precision positioning of power pipeline data.

[0155] The core of the SAR line patrol based on ORB-SLAM3 and augmented reality to achieve high-precision positioning of power pipeline data is mainly divided into two aspects. The first is to integrate various types of non-uniform data, including CAD data, surveying and mapping data, paper drawing data, high-precision single-point positioning data, etc., into GIS data of unified specifications; the second is to apply for cable pipeline GIS data through high-precision positioning of mobile terminals, and convert the data into SAR spatial data for 3D projection in real time, and use ORB-SLAM3 to make real-time corrections to SAR spatial binding data during line patrol. In this way, line patrol personnel can intuitively see the direction, burial depth, status, location and other information of underground cables through the real-world images and pipeline data projection of the equipment camera, thereby greatly improving the efficiency of cable pipeline patrol, maintenance, emergency rescue and other work. In order to achieve the ultimate design goal, the core technical difficulties of the platform are mainly in two aspects: data standardization and SAR spatial stability.

[0156] The first is data normalization: after selecting a standardized data and data standard, this embodiment intends to select the GIS data format and use the WGS84 geodetic coordinate system as the data standard. It is necessary to perform version control, data reading, identification and conversion on various types of data with huge differences, and then integrate and unify the data, and finally build a set of technical GIS data that can meet the needs of SAR augmented reality, so as to ensure the uniformity and reliability of the data.

[0157] Secondly, the stability of SAR space: Although standard SAR technologies such as Google's AR Core and iOS's AR Kit are feature-rich and easy to use, there are certain problems in terms of stable performance. For example, after SLAM initializes the SAR space, continuous movement in the space usually causes displacement of the initially bound physical space, which is a fatal problem for operations such as cable pipelines that require excavation and addressing. Every point of deviation represents a possible increase in risk. Therefore, how to use the visual inertial navigation and hybrid map SLAM system (ie ORB-SLAM3) to achieve the best robustness in both large and small scenes, and achieve real-time operation accuracy that is several times higher than that of standard package tools such as AR Core.

[0158] At the basic data level, due to the incomplete data of some cable pipelines and the fact that the data of existing cables of most power supply units are managed manually, they are recorded and saved in the form of drawings and charts. At present, a large amount of GIS surveying and mapping data, construction CAD drawings and positioning data are generated in the laying, expansion and diversion of wires and cables. Whether it is historical data or data generated by subsequent operations, the data types are various and the data formats are different, so there are huge problems in data standardization, accuracy, version reliability and other aspects.

[0159] Data version standardization is to ensure the reliability of cable pipeline data in the face of various data and sources, rather than simply summarizing the data after input, so as to avoid data confusion.

[0160] Establish a basic standardization system and implement data version management based on HDFS object storage.

[0161] The first is data stripping and version management: whether it is CAD data, scanned construction drawing data or various data versions of surveying and mapping data, when importing any type of data source, it is necessary to use MD5 (Message Digest Algorithm 5) to identify the file, and identify and detect duplication with existing data. At the same time, the summary data and source file, creation time, name, etc. are stored in a unified data version. If it is CAD or GIS data, it is also necessary to read the internal editing time of the file to expand the file identification. Finally, the data version is controlled with the internal editing time record of CAD and GIS as the first priority, and the creation and editing time of the file as the second priority.

[0162] The second is the refined management of data: that is, for all imported cable pipeline data, the data source file is the core, but the system data should be refined to a marked line, that is, every line. All data such as the name, range, mileage, etc. of each line are used as the target of refined management. The source data file is given priority, and the post-editing data is used as a supplement to carry out refined version management accurate to each line. In other words, if the line data of a specified cable is involved in multiple data sources, the data source time is used first; if a cable data is named differently in multiple data sources, but all parameters such as line direction, range, mileage, etc. are very similar, it can also be regarded as the same line for merged version management.

[0163] After completing basic data version management, the text selects WGS84 as the standard to establish a unified standard data center to provide data support for SAR terminals. However, the core issue is how to standardize and unify a wide variety of data such as other standard GIS data, projection map data, CAD drawing data, etc.

[0164] Projection coordinate map to WGS84 geodetic coordinate system: WGS84 geodetic coordinate system is also called geocentric coordinate system. Geocentric coordinate system is based on the earth ellipsoid, and its origin is determined as a specific point on the earth ellipsoid. Under the premise of knowing the latitude, longitude and altitude of the point, the earth ellipsoid centered on the point can be determined, and the required standard system can be obtained based on the conversion relationship between it and the earth coordinate system.

[0165] When processing such data, accurate projection coordinate system conversion can be performed through Gaussian inverse calculation. The establishment of the Gaussian projection coordinate system requires the determination of the basic parameters of the conical surface. First, it is necessary to determine the central meridian where it is located, and then determine the longitude on the meridian as the zero point, and use the longitude of a point on the meridian and the height of the point to determine the cone where the point is located. The base of the cone includes all circular surfaces tangent to the earth's ellipsoid surface. By adjusting these circular surfaces, the cone can be projected on the central meridian after rotation. On this basis, a Gaussian projection coordinate system is established, in which the projection surface is a conical surface, and the intersection of the central meridian and the projection surface is called the center of the projection coordinate system. The upper and lower endpoints of the projection surface correspond to the due north and due south directions, respectively.

[0166] Table 1 is a reference table of ellipsoid coefficients of China's 1980 Xi'an coordinate system (GRS75 ellipsoid); Table 2 is an error table of actual calculation results and fixed solution measurement results based on the surveying and mapping data and Gaussian plane data in Table 1.

[0167] Coordinate system (ellipsoid name) a b f China 1980 Xi'an Coordinate System 6378140m 6356755.2882m 0.00335281317790

[0168] Table 1

[0169]

[0170] Table 2

[0171] Data conversion between geodetic coordinate system data of other standards and WGS84 standard: Usually, the geodetic coordinate system standards generated by different surveying and mapping standards in different places will also be different. For the GIS data obtained by such surveying and mapping, this embodiment can directly call the Arc GIS encapsulated static library toolkit to perform data standard conversion. The supported standard types include but are not limited to CGCS2000, BG54, XA80 and data conversion of other local coordinate systems.

[0172] Convert CAD drawing coordinate system to WGS84 coordinate system: In this embodiment, the conversion of all plane drawing data needs to be completed in two steps.

[0173] First, the drawing content data is projected into the projection coordinate system map described by the drawing parameters, and the multi-vertex alignment and binding of the drawing and the map base map are performed.

[0174] Then, the positions and parameter information of each point, line, surface, etc. in the binding data are uniformly converted with the binding point as the core.

[0175] Finally, the transformed projection coordinate data is brought into the Gaussian inverse calculation mentioned above to obtain the final standard WGS84 cable pipeline data.

[0176] Stable SAR application through ORB-SLAM3:

[0177] The ORB-SLAM3 framework has been upgraded to fully support the IMU (inertial navigation) mechanism. The core method is to obtain data through the IMU unit, and use the attitude, velocity, and position differential equations to derive solutions and convert them into approximate forms under discrete time. This makes it possible to solve navigation information under discrete time sampling. Using the angular velocity and acceleration measured by the IMU, the navigation information at the current moment is calculated based on the navigation information at the previous moment, including attitude solution, velocity solution, position solution, etc., to achieve SLAM more accurately fit the real world space, thereby realizing accurate display, navigation, line patrol and other operations of SAR cable pipelines.

[0178] ORB-SLAM3 is a system that integrates and upgrades ORB-SLAM2 and ORB-SLAM-VI. It can efficiently provide visual inertial navigation mode support. It is mainly composed of four parts:

[0179] Atlas component: A collection of multiple discontinuous maps that form a multi-map representation. The active map is used to track incoming frames and continuously optimize and extend keyframes through local mapping threads. Other inactive maps exist in Atlas. The system builds a unique DBoW2 database based on keyframes for loop detection, map merging, and relocalization.

[0180] Tracking component: Processes sensor information, calculates the pose of the current frame relative to the active map in real time, and matches map features by minimizing the reprojection error. This not only determines whether the current frame is used as a new keyframe, but also involves the estimation of the body velocity and IMU bias in visual inertial mode. The tracking thread attempts to resume tracking after repositioning in the Atlas map and switches the active map when appropriate.

[0181] Local Mapping Component: Add keyframes and feature points to the active map, remove redundant elements, and optimize the map using vision or visual inertial guidance BA. The local window operates only within the keyframe range close to the current frame, and the IMU parameters are initialized and optimized through a novel maximum a posteriori estimation technique.

[0182] Loop detection and map merging component: detects the common area between the active map and the entire Atlas, and performs loop correction if the common part belongs to the active map; otherwise, the two maps are seamlessly merged into a new active map. After the loop correction is completed, global BA optimization will be started to further adjust all relevant parameters without affecting real-time performance.

[0183] In the standard SAR framework, the visual sensor only estimates the pose of the current camera. However, in the visual-inertial navigation joint module in Example 1, in addition to the pose of the camera, the pose of the body also needs to be calculated.

[0184] SAR terminal and high-precision positioning:

[0185] The charm of augmented reality technology lies in its ability to cleverly superimpose virtual scenes with real scenes, allowing the information of interest to be presented in a more intuitive way, bringing users an unprecedented efficient and convenient cable pipeline inspection experience.

[0186] After the device equipped with this complex outdoor environment north calibration system calibrates the PDA and determines the true north direction and high-precision positioning data, the PDA mobile terminal can request the cable pipeline GIS data within the authority of the data center based on these data. Then, by converting the geodetic coordinate system data of WGS84 into plane data, and combining the buried depth, ellipsoid coefficient and other parameters of the cable pipeline as the value of the Z axis of the stereo coordinate system, the plane coordinate system is converted to the stereo coordinate system. This means that the point P (Px, Py) in the plane coordinate system will be expressed as (Px, Py, Pz) in the stereo coordinate system, and the formula is:

[0187]

[0188] Among them, N is the radius of the ellipse, a is the major axis of the earth ellipsoid, and b is the minor axis of the earth ellipsoid. The calculation formula is as follows:

[0189]

[0190]

[0191] Through this conversion, two-dimensional GIS data can be converted into three-dimensional space data, and dynamic cable pipeline models can be drawn in three-dimensional space, thus completing the construction of SAR space.

[0192] Final application integration and implementation effect:

[0193] SSAR augmented reality is mainly presented visually through PDA mobile terminals equipped with high-performance processors, and is also used to coordinate and control the various units of the entire system.

[0194] The communication unit communicates with the remote service center through a wireless network and communication protocol, enabling the PDA to send packet transmission requests to the service center and obtain pipeline attribute data. The display unit is composed of a PDA screen, which is used to display the AR effect of the virtual underground cable model superimposed on the real scene. The transmission unit is composed of a Bluetooth module, which is used to obtain the positioning data of the GNSS receiver of the high-precision positioning module, as well as the navigation information composed of continuous positioning data and forward the processed navigation signal acquisition.

[0195] The location information, moving speed, pitch rotation and other angle information of the mobile terminal are crucial for the pipeline SAR system to achieve real-time positioning and underground cable model tracking.

[0196] This information is collected by a variety of sensors, and the signal acquisition unit mainly includes PDA cameras, gyroscopes, gravity accelerometers, GPS (Global Positioning System) and other sensors.

[0197] The PDA camera is used to obtain images of the real environment. The high-precision positioning module obtains the spatial position information of the current position and integrates it with the spatial information of the underground cables to find the marking points of the underground cables, convert the virtual marking point information into a cable pipeline model, and superimpose the rendered image on the image obtained by the camera.

[0198] The gyroscope and gravity accelerometer are used to collect the six-axis attitude information of the PDA, and the position of the rendered cable model on the display is adjusted in real time according to the attitude and direction information.

[0199] The underground power pipeline visualization data application platform based on augmented reality technology integrates the high-precision positioning information of mobile terminals with the cable location information of the system database, realizing functions such as the display of cable location information with the environment as the background, improving the level of comprehensive data utilization, giving full play to the value of enterprise data resources, providing strong information support for the company's intensive management and operation, playing a good promoting role, establishing a good social image for the company, and achieving good social benefits.

[0200] Obviously, the above embodiments are merely examples for the purpose of clear explanation, and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived therefrom are still within the scope of protection of the invention.

Claims

1. A north calibration system in a complex outdoor environment, characterized in that: include: Sensor acquisition module, including visual sensor, inertial navigation sensor, GPS positioning sensor and magnetometer; Visual sensors are used to collect image information in outdoor environments; Inertial navigation sensors are used to collect motion state information in outdoor environments, and obtain real-time speed information and real-time attitude information through integration operations; GPS positioning sensors are used to collect satellite data and real-time position projection coordinate map information in outdoor environments; The magnetometer is used to collect real-time geomagnetic information in outdoor environments; The data processing module is used to generate feature point parameters and landmark parameters for image information in outdoor environments using image processing algorithms; and to convert the real-time position projection coordinate map information into the WGS84 geodetic coordinate system using Gaussian inverse calculation; Use real-time speed information and real-time posture information to calculate the body posture and generate state variables; The initial calibration module is used to use the ORB-SLAM3 framework to initialize the IMU using the real-time velocity information and real-time attitude information to generate initial attitude information, initial velocity information, initial position information and initial IMU bias; and then combine the geomagnetic information with the initial attitude information, initial velocity information, initial position information and initial IMU bias to generate preliminary north calibration data; The satellite north pointing module is used to calculate the satellite data and generate the satellite north pointing calibration data using one of the RTK mode and the CORS differential positioning mode; The visual-inertial navigation joint module is used to use the visual-inertial navigation joint function of the ORB-SLAM3 framework to combine feature point parameters, landmark parameters, real-time speed information and real-time attitude information to generate the direction information estimated by the ORB-SLAM3 framework; The integrated calibration north module projects the direction information estimated by the ORB-SLAM3 framework into the WGS84 geodetic coordinate system to generate secondary north calibration data. The preliminary northing calibration data and the secondary northing calibration data are combined to generate the final northing calibration data; The communication module is used to communicate with the PDA using the IEEE802.11b communication protocol, and sends the state variables, the final north calibration data and the satellite's north calibration data to the PDA. The PDA uses the state variables, the final north calibration data and the PDA's own posture information to perform north calibration.

2. The north calibration system in complex outdoor environment according to claim 1 is characterized in that: In the sensor acquisition module, the inertial navigation sensor includes an accelerometer and a gyroscope.

3. The north reference calibration system in complex outdoor environments according to claim 2, characterized in that: In the data processing module, the image processing algorithm includes the following steps: Step 1: Edge detection: Use the Canny edge detection algorithm to perform edge detection on the collected outdoor environment image and generate edge features; Step 2, feature extraction: Using the SIFT feature extraction algorithm, the edge features are extracted into feature points for direction determination, and the corresponding feature point parameters are generated; Step 3: Image matching: Match the extracted feature points with the feature points in other images collected in real time, find the most similar feature point pairs by comparing the similarity of the SIFT descriptors of the feature points, and generate landmark parameters.

4. The north calibration system in complex outdoor environment according to claim 3 is characterized in that: In the data processing module, Gaussian backcalculation includes the following calculation formula: Where: B is latitude, I is longitude, B f is the latitude of the base point, η f for V f is (1+η f 2 ) 1 / 2 , t f tanB f , N f for a(1-e 2 sin 2 B f ) 1 / 2 , y represents one of the source coordinate system longitude and the y coordinate system; The inverse calculation formula of meridian arc length is: Dividing both sides by a0 gives: make: So: β = B + P2sin2B + P4sin4B + P6sin6B (6); Back-substituting the trigonometric series into the formula yields: q4=-P4+P2 2 -2P2P6+4P2 2 P4 (8); B f =β+q2sin2β+q4sin4β+q6sin6β (10); V f =(1+η f 2 ) 1 / 2 (12); t f =tanB f (13); N f =a(1-e 2 that 2 B f ) -1 / 2 (14).

5. The north reference calibration system in complex outdoor environment according to claim 4, characterized in that: In the data processing module, the body posture calculation includes the following calculation formula: Where Si is the state variable, Vi is the real-time speed information, Ti is the time information, b i g is the gyroscope bias, b i a is the accelerometer bias.

6. The north reference calibration system in complex outdoor environments according to claim 5, characterized in that: In the preliminary calibration module, the initialization operation of the IMU includes the following steps: Step 1, estimating the noise parameters of the real-time speed information and the real-time attitude information; Step 2: Estimating and correcting the deviation between the real-time speed information and the real-time attitude information; Step three, calibrate the scale factors of the accelerometer and gyroscope; Step 4: Perform IMU alignment to determine the initial posture of the IMU; Step 5: Initialize the IMU using the estimated noise parameters, bias, scale factor, and initial pose.

7. The north reference calibration system in complex outdoor environments according to claim 6, characterized in that: In the satellite north guidance module, the RTK mode includes the following steps: Step 1, by receiving satellite orbit information and time information from satellite data; Step 2: Use the distance between two adjacent satellites and the known coordinates of each satellite to calculate the current precise coordinates of RTK using the real-time difference method; Step three: Combine the precise coordinates with geomagnetic information to generate the satellite's north calibration data.

8. The north reference calibration system in complex outdoor environments according to claim 7, characterized in that: In the satellite north guidance module, the CORS differential positioning mode includes the following steps: Step 1: receiving differential information and position information from several continuously operating reference stations through the CORS network; Step 2: Using the differential information and position information of the reference station, the positioning result is calculated; Step three: Combine the positioning results with geomagnetic information to generate the satellite's north reference calibration data.

9. A method for calibrating north in a complex outdoor environment, based on the operation of the north calibration system in a complex outdoor environment according to any one of claims 1 to 8, characterized in that: The following steps are involved: Step 1: Data collection: Use visual sensors, inertial navigation sensors, GPS positioning sensors and magnetometers to collect image information, motion status information, real-time position projection coordinate map information and real-time geomagnetic information in the outdoor environment; Step 2, data processing: Apply image processing algorithms to the collected image information to generate feature point parameters and landmark parameters; convert the real-time position projection coordinate map information into the WGS84 geodetic coordinate system; The real-time speed information and real-time posture information are used to calculate the body posture and generate state variables; Step 3: Preliminary calibration: Use the ORB-SLAM3 framework to initialize the IMU for real-time velocity information and real-time attitude information, generate initial attitude information, velocity information, position information and IMU bias; then combine the geomagnetic information to generate preliminary north calibration data; Step 4: Satellite north calibration: Select one of the RTK mode and CORS differential positioning mode, calculate the satellite data, and generate the satellite north calibration data; Step 5: Visual-Inertial Navigation Joint Calibration: Using the visual-inertial navigation joint function of the ORB-SLAM3 framework, combined with feature point parameters, landmark parameters, real-time speed information and real-time attitude information, the direction information estimated by the ORB-SLAM3 framework is generated; Step 6: Comprehensive calibration: Project the direction information estimated by the ORB-SLAM3 framework with the WGS84 geodetic coordinate system to generate secondary north calibration data; The preliminary north calibration data and the secondary north calibration data are combined to generate the final north calibration data; Step 7, data communication: Use the IEEE 802.11b communication protocol to send the state variables, final northing calibration data and satellite northing calibration data to the PDA for the PDA to perform northing calibration.

10. The method for calibrating north compass in a complex outdoor environment according to claim 9, characterized in that: In step 7, the satellite's north calibration data is used to perform multi-source data fusion on the PDA.

Images