Multi-source data fusion autonomous positioning method and system

Abstract

The application provides a multi-source data fusion autonomous positioning method and system, wherein the method comprises the following steps: collecting GPS positioning data, visual image data and laser radar point cloud data of an unmanned aerial vehicle; adopting an unscented Kalman filtering algorithm to carry out noise reduction pretreatment on the GPS positioning data; performing sensor joint calibration based on the visual image and the laser radar point cloud, searching a pre-set high-precision tower station account database, solving a PnP problem, and establishing a geometric mapping relationship between a camera coordinate system and a world coordinate system; inputting image target detection data, GPS state estimation values and geometric mapping data into a pre-trained autoencoder regression network; extracting anti-interference potential features through an encoder of the network; and outputting target position estimation values of a target space position of the unmanned aerial vehicle by using a regression head. The application effectively solves the positioning drift problem caused by strong electromagnetic interference and complex topography in power inspection.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of unmanned aerial vehicle navigation and positioning, and particularly relates to a multi-source data fusion autonomous positioning method and system. BACKGROUND

[0002] At present, power inspection unmanned aerial vehicles mainly rely on global satellite navigation systems (GNSS) for positioning. However, in special scenarios such as ultra-high voltage transmission line inspection, the existing positioning scheme faces severe challenges: first, there is a strong power frequency electromagnetic field around the high-voltage line tower, which will seriously interfere with the magnetometer and GPS receiver of the unmanned aerial vehicle, resulting in signal loss or non-Gaussian distribution of positioning jumps. Second, the terrain in the wild mountainous area is complex, and the visual background is mostly repetitive texture (vegetation) or weak texture (sky), which causes the feature point-based visual SLAM algorithm to easily lose tracking. The existing extended Kalman filter (EKF) fusion algorithm is difficult to handle such nonlinear strong noise.

[0003] Therefore, there is an urgent need for an autonomous positioning method that can deeply fuse geometric constraints and deep learning features and has anti-electromagnetic interference capability. SUMMARY

[0004] The present application aims to provide a multi-source data fusion autonomous positioning method and system to solve the problems pointed out in the background.

[0005] In a first aspect, the present application provides a multi-source data fusion autonomous positioning method, which is applied to a power inspection unmanned aerial vehicle and includes the following steps:

[0006] Collecting multi-source sensor data of the unmanned aerial vehicle, wherein the multi-source sensor data includes GPS positioning data, visual image data, laser radar point cloud data and inertial measurement unit data;

[0007] Using an unscented Kalman filter algorithm to perform noise reduction preprocessing on the GPS positioning data to obtain a GPS state estimate value;

[0008] Based on the GPS state estimate value or the inertial measurement unit data, indexing a high-precision tower database, extracting image key points from the visual image data, combining the laser radar point cloud data to solve multi-dimensional geometric features, establishing a mapping relationship between the camera coordinate system and the world coordinate system by solving the PnP problem, and obtaining geometric pose feature data;

[0009] Extracting image target detection data from the visual image data, inputting the image target detection data, the GPS state estimate value and the geometric pose feature data into a pre-trained autoencoder regression network;

[0010] The online fusion filtering is performed through the auto-encoder regression network, and a target spatial position estimation value of the unmanned aerial vehicle is output.

[0011] Optionally, the GPS positioning data is preprocessed by using the unscented Kalman filtering algorithm, and the preprocessing specifically includes:

[0012] The filter state is initialized, the initial state estimation value and variance of a state vector are determined, and Sigma sampling is performed to obtain 2n+1 Sigma sampling points, wherein n is the dimension of the state vector;

[0013] Time updating is performed according to the Sigma sampling points, a state prediction value is calculated, and Sigma sampling is performed again in the state prediction value;

[0014] Measurement updating is performed according to a measurement value, and a latitude and longitude state estimation value is calculated as the GPS state estimation value.

[0015] Optionally, the mapping relationship between the camera coordinate system and the world coordinate system is established by solving the PnP problem, and the establishing specifically includes:

[0016] Four image key points are selected as control points in the camera coordinate system, and the four control points are not coplanar;

[0017] The corresponding three-dimensional coordinates of the four control points in the world coordinate system are analyzed, and a mapping relationship between the four control points in the world coordinate system, the camera coordinate system and the laser radar point cloud coordinate system is constructed;

[0018] The camera external parameters are calculated by analyzing the coordinates of the four control points, and the camera pose information is solved, and geometric pose calculation is completed.

[0019] Optionally, the auto-encoder regression network adopts a mechanism combining unsupervised learning and self-supervised learning; the image target detection data, the GPS state estimation value and the geometric pose feature data are input into the pre-trained auto-encoder regression network, and the inputting specifically includes:

[0020] The image target detection data is aligned and spliced, and the GPS state estimation value and the geometric pose feature data are standardized;

[0021] The processed data is subjected to online fusion filtering in the auto-encoder regression network, and a target spatial position is calculated.

[0022] Optionally, the method further includes a relative positioning step for a power tower, and the relative positioning step is realized by one of the following methods:

[0023] extracting power tower structure features from the visual image data by using a key point detection algorithm, calculating the scale of the actual object and the image pixel based on the principle of parallax measurement, and solving the relative distance and attitude of the unmanned aerial vehicle and the power tower;

[0024] Alternatively, mapping the point cloud and the image pixel through sensor coordinate transformation of the radar and the camera, refining the relative distance and attitude of the unmanned aerial vehicle and the power tower according to the average distance and inclination angle data of the point cloud.

[0025] Optionally, the method further comprises an enhanced positioning step:

[0026] obtaining an external positioning scheme by using visual inertial odometry technology, and enhancing the geographical positioning accuracy in combination with the inertial navigation system of the unmanned aerial vehicle;

[0027] obtaining height data of the barometric pressure sensor, and using radar height finding or visual height finding data as an auxiliary to correct the height error caused by barometric pressure fluctuation, to obtain relative ground height positioning data.

[0028] Optionally, the method further comprises an adaptive weighting step based on electromagnetic field environment perception, specifically comprising:

[0029] monitoring the carrier phase noise variance of the GPS positioning data and the magnetometer fluctuation variance of the inertial measurement unit in real time, and constructing an electromagnetic interference intensity index in the power inspection scene;

[0030] establishing a nonlinear mapping relationship between the electromagnetic interference intensity index and the measurement noise covariance matrix in the unscented Kalman filter algorithm;

[0031] when the electromagnetic interference intensity index exceeds a preset safety threshold, adaptively expanding the measurement noise covariance matrix, reducing the confidence weight of the GPS positioning data in state updating, and simultaneously increasing the fusion weight of the visual image data and the laser radar point cloud data in the autoencoder regression network, to suppress the positioning drift caused by strong electromagnetic field.

[0032] Optionally, the method further comprises a visual anti-interference step based on radar reflection intensity guidance, specifically comprising:

[0033] obtaining reflection intensity information in the laser radar point cloud data, setting a reflection intensity threshold according to the high reflectivity characteristic of the power tower metal material, and separating out tower point cloud with high reflection intensity and background point cloud with low reflection intensity;

[0034] projecting the tower point cloud with high reflection intensity to the imaging plane of the visual image data to generate a region of interest mask containing the tower metal structure;

[0035] In the process of extracting image target detection data from the visual image data, the visual image is spatially filtered by using the region of interest mask, only the pixel features within the region of interest mask are reserved as valid inputs, and the tree vegetation and mountain texture interference in the power inspection background are filtered out.

[0036] Optionally, in the autoencoder regression network, the self-supervised learning mechanism comprises an online updating step based on geometric-depth mutual supervision:

[0037] In the process of flight of the unmanned aerial vehicle, an Euclidean distance error between a geometric pose solved by a PnP problem in the geometric pose feature data and a target space position estimation value output by the autoencoder regression network is calculated in real time;

[0038] When the Euclidean distance error is smaller than a preset confidence interval threshold and the number of feature points of the visual image data meets the robustness requirement of PnP solving, the current geometric pose is determined as a high-confidence label;

[0039] The high-confidence label is used as a self-supervised signal to construct a loss function, and the autoencoder regression network is trained by online back propagation to update network weight parameters to adapt to feature distribution deviation caused by light changes and tower corrosion in the inspection process.

[0040] In a second aspect, an embodiment of the present application provides a multi-source data fusion autonomous positioning system, comprising:

[0041] A data acquisition module is configured to acquire GPS positioning data, visual image data and laser radar point cloud data of the unmanned aerial vehicle.

[0042] A GPS preprocessing module is configured to perform noise reduction preprocessing on the GPS positioning data by using an unscented Kalman filter algorithm to obtain a GPS state estimation value.

[0043] A joint calibration module is configured to perform sensor joint calibration based on the visual image data and the laser radar point cloud data, to establish a mapping relationship between a camera coordinate system and a world coordinate system by solving a PnP problem, and to obtain geometric pose feature data.

[0044] A fusion positioning module is configured to extract image target detection data from the visual image data, and to input the image target detection data, the GPS state estimation value and the geometric pose feature data into a pre-trained autoencoder regression network to output a target space position estimation value of the unmanned aerial vehicle through online fusion filtering.

[0045] The present application has the following beneficial effects:

[0046] The application adopts an unscented Kalman filter algorithm to replace a traditional extended Kalman filter, and approximates a state probability density through a deterministic Sigma sampling technology, effectively avoiding Jacobian matrix calculation and truncation error in a linearization process, improving the processing capability of the system on non-Gaussian distribution positioning jump noise, and thus guaranteeing the filtering accuracy of satellite positioning data in a strong nonlinear noise environment such as an ultra-high voltage transmission line. Meanwhile, the application constructs an adaptive weighting mechanism based on electromagnetic field environment perception, monitors the fluctuation characteristics of a carrier phase and a magnetometer in real time, uses a mapping network to establish a nonlinear mapping relationship between an interference index and a measurement noise covariance matrix and a feature weight, and can automatically block the propagation of electromagnetic interference noise at a physical level, effectively suppressing the positioning drift caused by strong electromagnetic fields.

[0047] The application uses the cross-modal complementary characteristics of laser radar reflection intensity and visual image to construct an anti-interference filter based on physical material characteristics. By extracting the high reflection point cloud of the metal material of the power tower, and combining the convex hull generation and the morphological dilation algorithm to generate a continuous and closed effective region mask in the image plane, the system can forcibly filter out the interference of complex background textures such as trees, vegetation and mountains at the image pixel level. This mechanism ensures that the visual algorithm only focuses on the core power facility features, improves the robustness and signal-to-noise ratio of visual perception in complex terrain and topography in the wild, and solves the problem that the traditional visual SLAM algorithm easily loses tracking in a weak texture or repetitive texture environment.

[0048] The application establishes a robust index mechanism based on inertial topological recursion, which can still guarantee accurate retrieval of the tower table under the extreme condition of complete satellite signal failure through inertial navigation calculation, avoid the logical dead loop that the traditional positioning logic cannot retrieve the table and cannot perform geometric solution when GPS loses lock, and ensure the continuity of the inspection task. In addition, the application has environment adaptive capability under low algorithm consumption, and is suitable for deployment of a machine-mounted edge computing platform. The system uses an online updating strategy based on geometric and depth mutual supervision, uses a high-confidence geometric pose as a self-supervised signal, and uses a frozen backbone network to update only the parameters of the regression head optimization strategy, reduces the calculation amount of back propagation to a very low level of full parameter update, so that the unmanned aerial vehicle can adapt to the feature distribution shift caused by light changes and tower corrosion in real time during the execution of the task, realizes continuous precision optimization under the premise of ensuring the real-time of the system.

[0049] Other features and advantages of the present application will be set forth in the following description, and in part will become apparent to those skilled in the art from the description, or can be learned by practice of the application. The objects and other advantages of the present application can be achieved and obtained by the structure particularly pointed out in the written description and the accompanying drawings.

[0050] The technical solutions of the present application will be further described in detail below through the drawings and examples. BRIEF DESCRIPTION OF DRAWINGS

[0051] The accompanying drawings are included to provide a further understanding of the application and are incorporated in and constitute a part of this specification, illustrate embodiments of the application and are meant to explain the application and are not intended to limit the application. In the drawings:

[0052] Figure 1 is a flow chart of a multi-source data fusion autonomous positioning method in an embodiment of the application;

[0053] Figure 2 is a schematic diagram of a multi-source data fusion autonomous positioning system in an embodiment of the application. DETAILED DESCRIPTION

[0054] The preferred embodiments of the application will be described hereinafter with reference to the accompanying drawings, in which the preferred embodiments of the application will be presented to explain and illustrate the application, but not to limit the application.

[0055] Embodiment one:

[0056] The embodiment of the application provides a hardware architecture of a multi-source data fusion autonomous positioning system. The system is configured as a set of on-board edge computing and perception platforms, and is applied to power inspection unmanned aerial vehicles in complex electromagnetic environments and terrains. In view of strong power frequency electromagnetic interference generated by high-voltage transmission lines in the power inspection scene, satellite signal multipath effects brought by mountain and valley environments, and unstructured environment characteristics, the system is optimized and unified in time and space on the aspects of hardware selection, electromagnetic shielding design and bottom driver.

[0057] The multi-source data fusion autonomous positioning system is divided into a sensor data acquisition subsystem, an edge computing processing subsystem and a flight control interaction subsystem.

[0058] The sensor data acquisition subsystem is a perception front end, and the selection and configuration of the core hardware components thereof need to be designed and redundantly configured in electromagnetic compatibility (EMC). Specifically, the following is included:

[0059] High-precision satellite positioning receiver. The receiver adopts an industrial-grade anti-interference multi-mode multi-frequency RTK (Real-Time Kinematic) differential positioning module. The module supports receiving global navigation satellite signals of GPS L1 / L2 / L5 bands, Beidou B1I / B1C / B2a / B2b / B3I bands, GLONASS G1 / G2 bands, and Galileo E1 / E5a / E5b bands. In order to suppress the out-of-band interference of 50Hz power frequency and its high-order harmonics generated by power lines, the receiver RF front-end is connected in series with a high-Q SAW surface acoustic wave filter, the center frequency bandwidth is controlled within ±10MHz, and the out-of-band suppression ratio is greater than 40dB. The raw observation data output frequency of the receiver is set to 20Hz to meet the dynamic capture requirements of the unmanned aerial vehicle at a high speed of 15m / s. The output data interface adopts LVTTL level UART serial port, and the baud rate is set to 460800bps to ensure low delay of data transmission. The output data content includes longitude, latitude and ellipsoidal height after positioning calculation, as well as original pseudo-range observation value, carrier phase observation value, Doppler frequency shift value and satellite ephemeris data, and the covariance matrix of positioning calculation is also output , the diagonal elements of the covariance matrix directly reflect the uncertainty variance of the current positioning in the north, east and sky directions.

[0060] Visual perception unit. The unit is composed of a group of binocular stereo vision cameras and a long-focus industrial camera, forming a heterogeneous visual perception array. The baseline length of the binocular camera is set to 220mm, which is used for depth perception and obstacle avoidance at close range (0.5m to 15m); the long-focus camera is equipped with a large-aperture fixed-focus lens with a focal length f=25mm (aperture value F1.8), which is used to capture fine textures of power towers, insulator strings and wires at long distances (15m to 80m). The camera sensor selects a global shutter CMOS sensor with a pixel size of 3.45μm×3.45μm to eliminate the rolling shutter jelly effect caused by high-speed flight vibration of the unmanned aerial vehicle. Preferably, the resolution of image acquisition is set to 1920×1080 pixels, the pixel bit depth is 12 bits, and the acquisition frame rate is set to 30fps. The camera receives a synchronization signal through a hardware trigger pin, and its exposure time is adaptively adjusted between 50μs and 2000μs. The camera internal parameters (including focal length , , principal point coordinates , ) and distortion coefficients (radial distortion , , , tangential distortion , ) are calibrated by high-precision calibration board before leaving factory, and the re-projection error is controlled within 0.15 pixels. The calibration results are stored in the EEPROM of the on-board computer in binary format.

[0061] Three-dimensional laser radar. A mechanical rotating laser radar with 32 lines or more is selected, the laser wavelength is 905 nm, and the eye safety level is Class 1. The ranging radius of the radar is not less than 150 m (at 10% reflectivity), the ranging accuracy is controlled within ±2 cm, the horizontal field of view is 360 degrees, and the vertical field of view is not less than 30 degrees. The radar not only outputs the spatial coordinates of the point cloud , but also is configured to output the reflection intensity of each laser point, which is quantized as a gray value of 0 to 255, for subsequent algorithms to distinguish between metal power equipment and natural vegetation background. The radar communicates with the computing platform through a gigabit Ethernet interface and transmits point cloud data packets using the UDP protocol, which includes microsecond-level timestamp information.

[0062] Inertial measurement unit (IMU). The unit integrates a three-axis micro-electromechanical system (MEMS) accelerometer, a three-axis MEMS gyroscope, and a three-axis magnetometer. The IMU is rigidly mounted at the center of the unmanned aerial vehicle body (center of gravity) and is shock-absorbed to filter out high-frequency vibrations of the rack. The data output frequency of the IMU is set to 200 Hz to capture high-frequency motion of the body. Its zero offset stability is better than 10 degrees / h, and the angle random walk is better than 0.5 degrees / sqrt(h). The data output by the IMU includes three-axis specific force, three-axis angular velocity, and three-axis magnetic field intensity.

[0063] To ensure the accuracy of multi-source data fusion, the system is equipped with a space-time synchronization processing mechanism.

[0064] In terms of time synchronization, the system uses FPGA (Field Programmable Gate Array) as a hardware synchronization controller. The input end of the FPGA is connected to the PPS (Pulse Per Second) signal pin of the GPS receiver, and the rising edge of the PPS signal is aligned with the UTC second time with an accuracy of better than 20 ns. A high-precision counter is maintained inside the FPGA for calibration based on the PPS signal. The output end of the FPGA is connected to the hardware trigger pins of the camera and the laser radar. Whenever the PPS signal arrives or within the interval of the PPS signal, a trigger pulse is generated at a fixed frequency (such as 10 Hz), and the FPGA sends a trigger signal to all sensors at the same time, so that the camera starts exposure at the same time and the radar records the azimuth zero position at the same time. All sensor data packets arriving at the on-board computer are stamped with a unified GPS timestamp based on the FPGA trigger time, so that the time synchronization error is strictly controlled within 1 ms.

[0065] In terms of spatial alignment, the system defines the center of the unmanned aerial vehicle body as the origin of the reference coordinate system (Body Frame, denoted as b system). After the system integration is completed, joint external parameter calibration is performed through calibration. The calibration process calculates the rotation matrix and translation vector of the camera optical center coordinate system (c system) relative to the body coordinate system , the rotation matrix and translation vector of the radar coordinate system (l system) relative to the body coordinate system, and the lever effect vector of the GPS antenna phase center relative to the body coordinate system. These calibration parameters are fixed in the system parameters and used for rigid body coordinate transformation in subsequent algorithms.

[0066] Embodiment two:

[0067] Embodiment two: The embodiment of the application details a multi-source data fusion autonomous positioning method. The method relies on the hardware platform described in embodiment one and is run in real time on the onboard edge computing module. The method solves the problems of GPS positioning jump and visual feature interference in power inspection through deep coupling of unscented Kalman filtering and self-encoder regression network.

[0068] As shown in Figure 1 , the method includes the following detailed steps:

[0069] Step S1: Collecting multi-source sensor data of the unmanned aerial vehicle.

[0070] The onboard computing platform reads data streams from each hardware interface through a driver program. The system sets a ring buffer in the memory to store GPS data, image data, point cloud data, and IMU data. Using a timestamp index, multi-source data at the same time (or within a permissible threshold, such as 5ms) is packaged into a fusion frame.

[0071] The multi-source sensor data includes:

[0072] GPS positioning data: containing longitude , latitude , geodetic height , and positioning covariance matrix .

[0073] Visual image data: containing left eye image and right eye image (or monocular long-focus image ), the image has been distortion-corrected according to the intrinsic parameters.

[0074] Laser radar point cloud data: containing a data set of laser points, each point containing coordinates and reflection intensity .

[0075] Step S2: The GPS positioning data is preprocessed by using an unscented Kalman filter algorithm to reduce noise and obtain the GPS state estimate.

[0076] Given that GPS signals in power line inspection scenarios often exhibit non-Gaussian abrupt noise due to pole obstruction, and that the system state equations are nonlinear, this embodiment employs unscented Kalman filtering (UKF) for state estimation. UKF approximates the probability density distribution of the state through deterministic Sigma point sampling, avoiding the Jacobian matrix calculation and truncation errors inherent in the linearization process of Extended Kalman Filtering (EKF), thus more accurately transmitting the mean and covariance.

[0077] This step specifically includes the following sub-steps:

[0078] Sub-step S2.1: Initialize the filter state.

[0079] Define the state vector of the UAV , dimension (In this embodiment) The state vector contains the UAV's three-dimensional position in the Northeastern Sky (ENU) navigation coordinate system. and three-dimensional velocity :

[0080] ;

[0081] in, The transpose operation represents a matrix or vector;

[0082] Initial state estimate Determined based on static alignment data before drone takeoff or initial high-confidence GPS data.

[0083] Initial error covariance matrix Set as a diagonal matrix, with diagonal elements defined based on the square of the GPS nominal accuracy (e.g., position error 1m, velocity error 0.1m / s):

[0084] ;

[0085] in, This represents the initial standard deviation of the positional state components (e.g., 1m). This represents the initial standard deviation of the velocity state component (e.g., 0.1 m / s).

[0086] Sub-step S2.2: Sigma point sampling.

[0087] According to the current time posterior state estimate and covariance matrix , generate Sigma points using unscented transformation.

[0088] Firstly, calculate scale parameter :

[0089] ；

[0090] wherein, is a constant to control the range of Sigma points distribution, usually taking 1e-3; is an auxiliary scaling parameter, usually taking 0; is the dimension of state vector (in this embodiment ) 。

[0091] Calculate Sigma points set :

[0092] ;

[0093] ;

[0094] ;

[0095] wherein, symbol represents a lower triangular matrix obtained by Cholesky decomposition of matrix , satisfying ; subscript represents taking the column vector of the lower triangular matrix .

[0096] Sub-step S2.3: time update (state prediction).

[0097] Pass each Sigma point using nonlinear state transition equation . In this embodiment, constant velocity (CV) motion model is adopted, i.e. considering that the velocity is constant within extremely short sampling time .

[0098] ;

[0099] State transition matrix F is defined as:

[0100] ;

[0101] wherein, represents a 3x3 unit matrix, represents a 3x3 zero matrix; is the time interval (or sampling period) for filter update, in seconds (s), corresponding to the sampling frequency of GPS data.

[0102] Compute one-step prediction state mean :

[0103] ;

[0104] Compute one-step prediction covariance matrix :

[0105] ；

[0106] where, is the process noise covariance matrix of dimension , used to compensate the deviation of constant velocity model from actual motion.

[0107] Weights and are computed as follows:

[0108] ;

[0109] ;

[0110] ;

[0111] is the non-negative weight coefficient used to incorporate the prior distribution information (e.g. higher order moment information) of state vector . For Gaussian distribution, .

[0112] To improve the accuracy, based on the predicted value and , the Sigma sampling is performed again to generate a new set of Sigma points for measurement update.

[0113] Sub-step S2.4: Measurement update.

[0114] Establish the GPS measurement equation . Although GPS directly observes the position, the measurement equation is linear, but it is processed uniformly under the UKF framework.

[0115] The measurement vector .

[0116] Substitute the new Sigma points into the measurement equation to obtain the measurement predicted point set :

[0117] ;

[0118] Computing the measurement prediction mean :

[0119] ;

[0120] Computing the measurement auto-covariance matrix and the state-measurement cross-covariance matrix :

[0121] ;

[0122] ;

[0123] wherein, is the measurement noise covariance matrix. In subsequent embodiments, will be adaptively adjusted according to the electromagnetic interference index.

[0124] Computing the Kalman gain :

[0125] ;

[0126] Updating the state estimate and the posterior covariance :

[0127] ;

[0128] ;

[0129] wherein, is the three-dimensional position observation vector in the ENU navigation coordinate system after the latitude and longitude data output by the GPS receiver at the current time is converted by Gauss projection (or UTM projection), and the unit of the position component in the state vector is consistent (meters).

[0130] The position component in the final is the GPS state estimate value after de-noising processing. This value effectively smooths the random jumps of the GPS signal and provides a stable and reliable absolute position reference for subsequent data fusion.

[0131] Embodiment Three:

[0132] This embodiment describes in detail the specific implementation process of sensor joint calibration and PnP problem solving. This step aims to build an accurate mapping of visual features and physical world coordinates, providing geometric priors for subsequent deep learning fusion.

[0133] Step S3: Sensor joint calibration based on the visual image data and the lidar point cloud data, mapping relationship between the camera coordinate system and the world coordinate system is established by solving the PnP problem, and geometric pose feature data is obtained.

[0134] This step utilizes the complementarity of visual features and radar depth to construct a geometric positioning result independent of GPS.

[0135] Sub-step S3.1: Key point extraction and robust indexing.

[0136] Before this step, the system pre-sets a high-precision tower geometry parameter library. This library is generated by pre-constructing a high-precision map of the transmission line using an airborne lidar, and then performing point cloud semantic segmentation and vectorization processing. The library stores the global unique identification code (GUID) of each tower on the entire line, as well as the absolute three-dimensional coordinates of the specific semantic key points of the tower body (such as tower base corner points, cross arm end points) in the world coordinate system The semantic key points of the power tower (such as tower base corner points) are extracted from the visual image plane as image control points using a deep learning key point detection algorithm (such as HRNet).

[0137] In view of the problem that strong electromagnetic interference may cause GPS failure, this step adopts an inertial topological recursive mechanism to lock the target tower ID: real-time monitoring of GPS covariance, when the covariance is less than the threshold, using the GPS state estimation value to retrieve the nearest tower in the account library; when the GPS covariance is out of limit (indicating that the interference is serious), automatically switching to the inertial navigation mode, using the displacement vector calculated by IMU integration, combining the tower ID confirmed at the last time and the topology structure of the inspection line to recursively lock the target tower ID in the current field of view. The specific logic is: when the GPS covariance is less than the preset threshold, the nearest neighbor tower is retrieved using the GPS position; when strong interference is detected (the GPS covariance is out of limit), the system automatically switches to the inertial navigation mode, and performs dead reckoning based on the acceleration and angular velocity output by the IMU, and predicts the target tower ID at the current time based on the tower ID confirmed at the last time and the line topology structure, so as to realize continuous indexing under GPS failure. Specifically, the inertial topological recursive mechanism adopts a node prediction algorithm based on a directed graph: the system defines the inspection line as a directed topological graph wherein the node stores the absolute coordinates and structure fingerprint of the tower, and the edge represents the line corridor direction vector. When the intensity index of electromagnetic interference is detected to exceed the threshold (i.e. the GPS is not reliable), the system locks the tower ID confirmed at the last time as an anchor point, and performs double integration using the high-frequency acceleration and angular velocity of the IMU to real-time solve the displacement increment vector of the UAV relative to the anchor point . The projection scalar (in (The unit vector representing the route direction). When When the current tower span exceeds a threshold (e.g., 90% of the span), it is determined that the drone is about to enter the line of sight of the next tower, and the target index is automatically switched to the next-hop node ID in the topology map. Simultaneously, the tower outline scanned by the LiDAR is matched and verified against the structural fingerprints (e.g., goblet-shaped, T-shaped) in the ledger database to prevent index misalignment caused by IMU integration drift. Specifically, the inertial topology recursion mechanism is key to solving the problem of GPS lockout failure leading to the inability to index the ledger under strong electromagnetic interference. The system pre-constructs a directed topology map of the lines. When the electromagnetic interference intensity index exceeds the threshold, the system immediately locks the tower ID confirmed in the previous moment (when GPS is reliable) as the topology anchor point. The system then uses the high-frequency acceleration and angular velocity output from the IMU for dual integration to calculate the displacement increment of the UAV relative to the anchor point. When the projection of the displacement onto the topological path exceeds a span threshold (e.g., 90%), the index target is automatically switched to the next hop node ID. Simultaneously, the tower height and crossarm layer number fingerprint features obtained from LiDAR scanning are compared and verified with the ledger data, thus achieving continuous and accurate ledger indexing and geometric benchmark acquisition even without satellite signals.

[0138] To prevent inertial recursion from drifting over time and causing incorrect tower IDs to be locked, the system immediately performs a structural fingerprint consistency check after locking an ID: It retrieves the structural fingerprint data (including tower height, number of crossarm layers, and number of insulator strings) of the tower with that ID from the ledger database. The physical features obtained from the current lidar point cloud clustering analysis are compared with this fingerprint. If the matching error exceeds a preset threshold (e.g., 15%), the current index is deemed invalid, and the system automatically triggers a local topology search, traversing the fingerprint features of adjacent nodes until the correct tower ID is locked again.

[0139] Sub-step S3.2: 3D coordinate matching.

[0140] Based on the locked tower ID, retrieve the absolute three-dimensional coordinates of the corresponding key point of the tower in the world coordinate system from the pre-set high-precision tower ledger database. To ensure correct spatial correspondence between visually extracted 2D pixels and 3D physical points in the ledger database, this invention employs a semantic consistency matching strategy. When constructing the high-precision tower ledger database, each 3D keypoint is assigned a unique semantic category label (e.g., ID_01 is the left front corner of the tower base, ID_02 is the left endpoint of the first-layer crossarm). Accordingly, in sub-step S3.1, the deep learning keypoint detection algorithm is configured to output the pixel coordinates and semantic category probability of each detected point. When solving for PnP, the system selects only the semantic-coordinate pairs with the highest confidence, and 3D points with the same semantic labels as those in the ledger database. Matching pairs. If the number of valid matching point pairs is less than 4 (e.g. due to occlusion), mark this time of geometric solving as invalid, and trigger the fault-tolerant mechanism of the subsequent network.

[0141] Sub-step S3.3: PnP problem solving.

[0142] Construct the PnP projection equation based on 4 control points. According to the pinhole camera model:

[0143] ；

[0144] wherein, is the rotation matrix and translation vector of the camera coordinate system relative to the world coordinate system to be solved; is the homogeneous coordinate representation of the control points in the world coordinate system, is the scale factor of the image point.

[0145] In order to ensure the uniqueness and robustness of the solution, the EPnP (Efficient PnP) algorithm is adopted. The EPnP algorithm represents the 3D points as the weighted sum of 4 virtual control points , converts the nonlinear problem into a linear equation system for solving, and the computational complexity is :

[0146] ;

[0147] ;

[0148] In the formula, is the barycentric coordinate coefficient of the th 3D point relative to the th virtual control point, and satisfies the constraint condition ; represents the coordinate of the th virtual control point in the world coordinate system, represents the coordinate of the th virtual control point in the camera coordinate system.

[0149] First, solve the linear equation system to obtain the initial value of and .

[0150] Then, construct the reprojection error objective function :

[0151] ;

[0152] wherein, is the observed pixel coordinate; For the projection mapping function based on the aforementioned pinhole camera model, for projecting a point in the world coordinate system to the pixel plane; Euclidean norm (L2 norm) of a vector;

[0153] The Levenberg-Marquardt algorithm is used to perform nonlinear iterative optimization on the objective function, and the and .

[0154] The calculated and are the pose information of the camera, representing the current attitude and position of the unmanned aerial vehicle in the world coordinate system. We call this set of data the geometric pose feature data, which combines the semantic feature positioning ability of vision and the accurate ranging ability of radar, and is one of the important input features of the subsequent deep learning network.

[0155] Embodiment Four:

[0156] This embodiment describes in detail the fusion positioning step based on the autoencoder regression network. This network uses a combination of unsupervised and self-supervised mechanisms to solve the problem of traditional Kalman filtering that is difficult to handle high-dimensional image features and nonlinear error models.

[0157] Step S4: Extract image target detection data from the visual image data, and input the image target detection data, the GPS state estimate value, and the geometric pose feature data into the pre-trained autoencoder regression network.

[0158] Sub-step S4.0: Construction and offline pre-training of the autoencoder regression network.

[0159] Before the unmanned aerial vehicle performs a task, the network needs to be trained offline:

[0160] Collect a large amount of inspection data in an interference-free environment, and use the fixed solution of RTK differential positioning as the position true value label . At the same time, artificially superimpose simulated electromagnetic interference noise and GPS multipath drift noise on the input data to form the training input . Specifically, in order to support the strong electromagnetic interference resistance characteristic of the present application, the simulated electromagnetic interference noise is generated using a heavy-tailed distribution (Heavy-tailed Distribution) model. On the GPS original observation data, a Cauchy distribution (Cauchy Distribution) impulse noise is randomly superimposed to simulate the non-Gaussian large amplitude jump caused by the loss of lock of the GPS phase-locked loop in a strong electromagnetic field. Such training data forces the network to learn robust features that filter out long-tail noise, rather than just dealing with Gaussian white noise.

[0161] The network comprises an encoder, a decoder and a regression head. The encoder is used to map multi-source heterogeneous data into a low-dimensional latent vector , forcing the network to learn the intrinsic manifold distribution of the data; the regression head directly maps the vector to obtain a position correction amount.

[0162] A composite loss function is constructed. Wherein is the reconstruction loss , which is used to learn the geometric consistency of each sensor data unsupervisedly; Smooth L1 Loss is adopted. Because the training data is superimposed with heavy-tailed impulse noise obeying Cauchy Distribution, the traditional Mean Square Error (MSE) loss function is too sensitive to abnormal values, which easily leads to network gradient explosion. Smooth L1 Loss grows linearly when the residual is large, which can effectively suppress the damage of long-tail noise to gradient and ensure the robustness of model convergence: .

[0163] Sub-step S4.1: data preprocessing and feature encoding.

[0164] Image target detection data: a lightweight convolutional neural network (such as YOLOv5-Nano) is used to infer visual images to detect power towers, insulators and other targets. The center coordinates , width and height and confidence of the detection box are output. These geometric information is normalized to the interval [0, 1] to form a visual feature vector .

[0165] GPS state estimation value: in order to ensure the dimensional consistency (metric units) of the input data, and avoid the nonlinear distortion caused by spherical coordinates, this step first projects the longitude, latitude and elevation data output by step S2 to the East-North-Up (ENU) local navigation coordinate system to obtain a three-dimensional relative coordinate vector relative to the inspection base station. Then, the ENU coordinate data is subjected to Z-Score standardization processing:

[0166] ;

[0167] Wherein, is the UKF filtered state value output by step S2, and The mean and standard deviation are obtained from the statistical analysis of the offline-built power inspection scenario dataset; the subtraction and division in the above formula are element-wise operations performed on the corresponding dimensions of the vector.

[0168] Geometric pose feature data: the pose obtained in step S3 and quaternions Similarly, Z-Score standardization is performed to form geometric feature vectors. .

[0169] Feature splicing: , , spliced ​​together to form a high-dimensional one-dimensional tensor To meet the requirement of fixed input dimensions in fully connected neural networks, the system employs a primary objective optimization strategy to process visual data: selecting only the tower detection box with the highest confidence level, including normalized center coordinates. Width and height and confidence level The data consists of 5 dimensions, padded with zeros if no target is found. Combined with 6-dimensional GPS status values ​​(position + velocity) and 7-dimensional joint calibration geometric data (3-dimensional position + 4-dimensional quaternions), the resulting data is stitched together. The total dimension is locked at 18. This embodiment explicitly defines the 18-dimensional input feature vector. The physical composition ensures the alignment of multi-source data in tensor space: dimensions 1-3 are the normalized GPS ENU relative coordinates; dimensions 4-6 are the normalized GPS three-axis velocities; dimensions 7-9 are the camera's three-dimensional position in the world coordinate system obtained through PnP calculation; dimensions 10-13 are the quaternion attitude parameters obtained through PnP calculation. ); Dimensions 14-18 represent the geometric features of the visual detection box (including normalized center coordinates). Width and height And detection confidence level). The channel arrangement order strictly corresponds to the output weight channels of MappingMLP in Example 7. It should be noted that: if the annotation geometry solution in step S3 is invalid (PnP fails), then it will be forced to The seven dimensions are set to all zeros, and the network automatically ignores this part of the input through the Dropout mechanism learned during training, instead relying on... and Degradation regression is performed to ensure the robustness of the system under extreme conditions.

[0170] Sub-step S4.2: Autoencoder regression network inference.

[0171] The network consists of an encoder, a decoder, and a regression head.

[0172] The embodiment discloses a network topology parameter verified by experiments: the encoder adopts a three-layer full connection structure, and the number of nodes is designed as , respectively , a Swish activation function and a BatchNormalization layer are connected after each layer, and a 16-dimensional latent vector is finally output . The decoder structure is symmetrical , and is used for unsupervised reconstruction. The regression head is composed of two full connection layers

[0173] Before the spliced feature tensor is input into the encoder, in order to realize the anti-interference in the physical layer, the system performs an attention gating operation.

[0174] Specifically, the weight vector (outputted by the MappingMLP in the seventh embodiment) is introduced (the dimension is consistent with ), and a Hadamard product (element-wise multiplication) operation is performed:

[0175]

[0176] Through the operation, when the environmental electromagnetic interference index increases, the MappingMLP automatically outputs a weight coefficient close to 0 for the GPS channel, so that the propagation of electromagnetic interference noise can be automatically suppressed at the feature extraction level before the data enters the neural network for feature extraction. The weighted is then sent to the first full connection layer of the encoder.

[0177] The encoder maps the high-dimensional to a low-dimensional latent space to obtain a latent vector .

[0178]

[0179] This process enables the network to learn the internal consistency between GPS, visual and geometric features. When the GPS is disturbed and jumps greatly, the consistency with the visual feature is destroyed, and the encoder will automatically suppress the expression weight of the GPS feature in the latent vector Z.

[0180] The regression head directly maps the latent vector to the position correction :

[0181] ;

[0182] The original design of the autoencoder network is to pre-train the feature extraction capability through the unsupervised reconstruction task (Decoder reconstructs the input), and then fine-tune the positioning output through the supervised regression task.

[0183] Step S5: online fusion filtering is performed through the autoencoder regression network, and a target spatial position estimation value of the unmanned aerial vehicle is output.

[0184] Calculate the final positioning result :

[0185] ；

[0186] wherein, represents element-wise multiplication, and this step is used to restore the normalized GPS data to physical coordinates using the aforementioned mean and variance; that is, on the basis of rough GPS positioning, the correction amount predicted by the network is superimposed. This result fuses the absolute reference of satellite navigation, the environmental semantics of visual perception, and the rigid constraint of geometric solution, achieving all-weather, anti-interference centimeter-level positioning.

[0187] Embodiment five:

[0188] This embodiment details the relative positioning steps of the power tower. In fine inspection (such as pin defect detection), the unmanned aerial vehicle needs to maintain an accurate relative distance from the tower.

[0189] The method further includes a relative positioning step for the power tower, which is achieved by one of the following methods:

[0190] Method one: using a key point detection algorithm to extract power tower structure features from the visual image data, calculating the scale of the actual object and the image pixel based on the principle of parallax measurement, and solving the relative distance and attitude of the unmanned aerial vehicle and the power tower.

[0191] Using a key point detection network (such as HRNet) to extract tower body key points. The distance is calculated based on the principle of similar triangles.

[0192] Let the camera focal length be (unit: millimeter), the actual physical width of a component (such as a ground wire support) of the power tower be (unit: meter, obtained from the account database), the pixel width of the component on the image be (unit: pixel), and the pixel size be (unit: millimeter / pixel).

[0193] Then the vertical distance D between the unmanned aerial vehicle and the component is calculated as follows:

[0194] ;

[0195] wherein, focal length (mm), actual object width (mm), pixel width (pixel), pixel physical size (mm / pixel).

[0196] In terms of attitude solution, by analyzing the slope of the tower center axis on the image plane , the roll angle of the UAV can be calculated :

[0197] ;

[0198] wherein, is the slope of the fitted straight line of the power tower center axis in the two-dimensional image pixel coordinate system (u-v coordinate system).

[0199] Method two: map the point cloud and image pixels through the sensor coordinate transformation of the radar and camera, and refine the relative distance and attitude of the UAV and the power tower according to the average distance and inclination data of the point cloud.

[0200] When insufficient light causes blurred edges in visual detection, radar data is introduced.

[0201] Transform the lidar point cloud to the camera coordinate system:

[0202] ;

[0203] Project onto the image plane to generate a depth map. Use the visual target detection box as a mask (Mask) to extract the point cloud set inside the box.

[0204] Calculate the average distance of the point cloud in the set :

[0205] ;

[0206] wherein, wherein, is the three-dimensional coordinate vector of the th point in the set in the camera coordinate system; represents the Euclidean norm operation of the vector (calculates the distance from the point to the origin); is the total number of valid point clouds in the set .

[0207] Use the least squares method to fit the point cloud plane to obtain the plane normal vector , and then the relative angle between the optical axis of the UAV and the tower plane is calculated to achieve high-precision relative attitude estimation.

[0208] Embodiment Six

[0209] This embodiment is directed to the enhancement positioning step. This step aims to solve the problem of height drift of a single barometric pressure sensor in complex terrain (such as a valley wind gap) and variable temperature environment.

[0210] The method further comprises an enhancement positioning step:

[0211] First, an external positioning scheme is obtained using visual inertial odometry technology, and the inertial navigation system of the UAV is combined to enhance the geographical positioning accuracy.

[0212] When the GPS signal is completely lost, the system switches to VIO mode. A nonlinear optimization method based on a sliding window is used.

[0213] State quantity IMU state (pose, velocity, zero offset) and inverse depth of visual landmarks within the sliding window are included.

[0214] The optimization objective is to minimize the weighted sum of IMU pre-integration residuals and visual re-projection residuals :

[0215] ;

[0216] In the formula, is the prior residual vector generated by the marginalization operation; is the Jacobian matrix corresponding to the prior information; is the state vector to be optimized within the sliding window; is the set of all IMU pre-integration constraints within the sliding window, is the index variable in this set, represents the IMU pre-integration measurement value between the frame and the frame body coordinate system, is the IMU pre-integration residual term; is the set of all visual observation frames within the sliding window, is the traversal index in this set, represents the pixel coordinate observation value of the th landmark point in the th frame image, represents the camera frame index, represents the feature point index; represents the covariance matrix Mahalanobis distance square of , and are the noise covariance matrices of prior information, IMU pre-integrated measurement and visual observation, respectively; represents the visual re-projection residual function, which is used to calculate the deviation between the actual observed pixel coordinates of the road point and the theoretical pixel coordinates obtained by projecting the state quantity ; the symbol represents the Mahalanobis distance square based on the covariance matrix , which is defined as .

[0217] The high-frequency pose increments output by the VIO system are used to recursively update the position of the UAV.

[0218] Secondly, the height data of the barometric sensor is obtained, and the radar height or visual height data is used as an auxiliary to correct the height error caused by the fluctuation of the barometric sensor, so as to obtain the relative ground height positioning data.

[0219] The height of the barometric sensor is easily affected by the airflow. A one-dimensional Kalman filter is constructed to fuse the barometric height, radar height and GPS height. . .

[0220] In the measurement equation, the barometric deviation state is introduced . The high-precision ranging of the radar is used as the observed true value to estimate : ; and the corrected relative ground height is: ;

[0221] This ensures that the UAV can maintain a constant ground clearance distance in mountainous areas with severe terrain changes, ensuring flight safety.

[0222] Embodiment Seven

[0223] This embodiment describes in detail the adaptive weighting technology based on electromagnetic field environment perception, which is the core means of the present application to solve the positioning drift caused by strong electromagnetic interference.

[0224] The method further comprises an adaptive weighting step based on electromagnetic field environment perception, specifically comprising:

[0225] Firstly, the carrier phase noise variance of the GPS positioning data and the fluctuation variance of the magnetometer of the inertial measurement unit are monitored in real time, and an electromagnetic interference intensity index in the power inspection scene is constructed.

[0226] The sliding window is calculated in real time (the window length is denoted as , for example, the window length is taken as ​The variance of the GPS carrier phase residual (epoch-making unit) The variance of the magnetometer modulus Define the electromagnetic interference intensity index. :

[0227] ;

[0228] in, and The baseline variance is the variance under interference-free conditions. These are the weighting coefficients for the carrier phase noise term and the magnetometer fluctuation term, respectively, with values ​​ranging from [value range missing]. And satisfy (For example, all values ​​are 0.5).

[0229] The second step involves establishing a nonlinear mapping relationship between the electromagnetic interference intensity index and the measurement noise covariance matrix in the unscented Kalman filter algorithm. An exponential expansion function is designed to dynamically adjust this mapping. Let the reference measurement noise matrix be... The preset security threshold is The coefficient of thermal expansion is ( A constant scalar greater than 0, used to control the magnitude of the noise covariance as the disturbance increases (e.g., a value between 5.0 and 10.0). Adaptive measurement noise matrix. The calculation formula is:

[0230] when hour, ;

[0231] when hour, ;in, Represented by natural constant An exponential function with base 0.

[0232] Thirdly, when the electromagnetic interference intensity index exceeds a preset safety threshold, the measurement noise covariance matrix is ​​adaptively expanded to reduce the confidence weight of the GPS positioning data in state updates, while simultaneously increasing the fusion weight of the visual image data and the lidar point cloud data in the autoencoder regression network to suppress positioning drift caused by strong electromagnetic fields. Specifically, a lightweight mapping network (MappingMLP) is constructed to convert the scalar... Mapped to a weight vector with the same dimensions as the input tensor The specific network structure of the MappingMLP is designed as a three-layer fully connected perceptron: the input layer contains one neuron (receiving scalar signals). ); the hidden layer contains 8 neurons with ReLU activation function; the output layer contains 18 neurons (corresponding to the total dimension of the input tensor) with Sigmoid activation function to strictly limit the output weight in the interval (0, 1). The network is jointly trained with the autoencoder in an end-to-end manner, and automatically learns the nonlinear mapping relationship between the interference intensity and the weight of each sensor channel.

[0233] In order to make the network have real anti-interference ability, the training set adopts the true value + synthetic noise strategy. Based on Cauchy distribution (Cauchy Distribution), the simulated electromagnetic pulse noise is generated , which has significant heavy-tailed characteristics and can accurately simulate the non-Gaussian jump caused by high-voltage line corona discharge. The noise is superimposed on the pure GPS data as an input sample.

[0234] The MappingMLP network is jointly trained with the autoencoder in an end-to-end manner. In the backpropagation process, the gradient of the total loss function to the weighted input feature is passed to the MappingMLP through Hadamard product operation . When the GPS channel causes the regression error to increase dramatically due to the superposition of heavy-tailed noise, in order to minimize , the gradient descent algorithm will force the MappingMLP to adjust the parameters so that the weight output value of the channel tends to 0. This mechanism realizes the automatic gating and isolation of untrusted signals at the physical level.

[0235] The training mechanism is as follows: since the Hadamard product operation of the input layer is differentiable, during joint training, the gradient of the total position regression loss (Loss) of the autoencoder regression network to the input feature is backpropagated to the MappingMLP through the chain rule. If the MappingMLP fails to correctly suppress the GPS channel with high noise, the high variance of this channel will cause the final regression error to increase dramatically, resulting in a large gradient to force the MappingMLP to adjust the parameters so that it outputs a weight close to 0 for the high interference index . This mechanism realizes unsupervised anti-interference strategy learning. The Hadamard product operation is performed at the input layer: . Through this physical mask mechanism, when the interference index increases, the The weight corresponding to the GPS channel is reduced to zero, and the physical propagation of noise is blocked. The deep training mechanism is that in the offline joint training phase, the GPS channel in the input data is artificially superimposed with simulated electromagnetic pulse noise obeying Cauchy distribution. Because Cauchy distribution has a significant heavy-tailed characteristic (there are outliers with a large probability), if the MappingMLP does not suppress this channel, the huge input noise will directly cause the gradient explosion of the regression loss function (Loss). In order to minimize the total loss, the back propagation algorithm will produce a strong negative gradient, forcing the MappingMLP to adjust the parameters so that it automatically outputs a weight close to 0 for high-variance input features. This mechanism uses the mathematical gradient competition principle to achieve automatic learning and gating of interference signal features. When strong interference is monitored (the interference index is greater than the safety threshold) , the network automatically reduces the numerical weight of the GPS data channel in the input feature tensor, so that the encoder relies more on visual and geometric features to generate latent vectors , thereby physically cutting off the propagation of interference in the feature extraction stage.

[0236] In the update step of the UKF, the Kalman gain . As increases exponentially, significantly decreases. This means that the filter automatically reduces the degree of trust in the current GPS measurement value, and the state update will mainly rely on the state prediction value (i.e., rely on the short-time calculation of the inertial navigation system), thereby effectively suppressing the GPS positioning jump caused by electromagnetic interference. At the same time, in the autoencoder regression network, an attention module is designed. Take as input, dynamically adjust the weight of the feature channel.

[0237] ;

[0238] ;

[0239] , where is the sensitivity coefficient of the Sigmoid function (must be greater than 0), used to adjust the decay rate of the weight with the interference (for example, take ); is a safety threshold, which is the same as the preset safety threshold set in the second step of the embodiment; wherein The mathematical expression of the function is , which is used to map the input to the interval (0, 1), is a natural constant, whose value is about 2.71828.

[0240] When the interference index is detected to be raised, the weight of the GPS feature channel is automatically lowered, and the weights of the visual and radar features are simultaneously raised.

[0241] Embodiment Eight:

[0242] This embodiment describes a visual anti-interference step based on radar reflection intensity guidance, which solves the problem of background texture interference.

[0243] The method further comprises a visual anti-interference step based on radar reflection intensity guidance, specifically comprising:

[0244] First, obtain the reflection intensity information in the laser radar point cloud data, set a reflection intensity threshold according to the high reflectivity characteristics of the metal material of the power tower, and separate the high reflection intensity tower point cloud and the low reflection intensity background point cloud.

[0245] Set the reflection intensity threshold (e.g., take 200). Traverse the laser radar point cloud , and screen out the point set that satisfies .

[0246] Second, project the high reflection intensity tower point cloud to the imaging plane of the visual image data to generate a mask of the region of interest containing the tower metal structure.

[0247] Use the joint calibration matrix to transform any point in the separated metal point set to the camera coordinate system to obtain the coordinates in the camera coordinate system ; then project to the image imaging plane according to the following pinhole camera model formula:

[0248] ;

[0249] ;

[0250] wherein are the X, Y, and Z axis coordinate components of the point in the camera coordinate system, respectively; is the normalized focal length of the camera in the x and y directions, is the coordinate of the principal point (optical center) of the camera in the image coordinate system.

[0251] On the image plane, the directly projected point cloud only forms a sparse discrete noise point. In order to generate a closed and effective region of interest mask, this step performs morphological processing on the projected point set: first, the convex hull algorithm is used to calculate the minimum circumscribed polygon of the projected point set on the image plane; second, the morphological dilation operation is performed on the polygon region (the dilation kernel size is set to ), filling the sparse gaps, and finally generating a continuous, solid binary mask image Mask. It needs to be specially pointed out that in order to solve the technical contradiction that the mask may be misaligned due to the projection using the disturbed pose, this step introduces a dynamic dilation strategy based on uncertainty. Specifically: read the covariance matrix output by the unscented Kalman filter , calculate its trace to quantify the position uncertainty. Set the morphological dilation radius of the mask . When strong interference causes the positioning covariance to diverge, the system automatically generates a tolerant mask, ensuring that even if the projection center has a deviation, the real tower is still included; when the positioning is accurate, a compact mask is generated. This mechanism solves the technical paradox of using a noisy pose for projection filtering.

[0252] Real-time acquisition of the posterior error covariance matrix output by the unscented Kalman filter, and calculate the uncertainty radius of the current position estimate . Set the physical size of the dilation kernel proportional to .

[0253] When electromagnetic interference causes the positioning covariance to increase, the system automatically generates a large range of tolerant masks, ensuring that even if the projection center has a deviation, the real tower structure is still included in the mask range; when the signal is good, a compact mask is generated. This mechanism realizes robust filtering in an uncertain environment.

[0254] Third, when extracting image target detection data from the visual image data, the region of interest mask is used for spatial filtering of the visual image, only retaining the pixel features within the region of interest mask as valid input, and filtering out the tree vegetation and mountain texture interference in the power inspection background.

[0255] The original visual image is pixel-by-pixel bitwise ANDed with the mask Mask:

[0256] ;

[0257] Wherein, the symbol Hadamard product operation of two matrices, i.e. multiplying the elements in the corresponding row and column positions, is used here to only retain the image pixel values corresponding to the region where the mask value is 1.

[0258] Result image In this case, only the pixel information of the power tower region is retained, and the texture of the background mountains and trees is forced to zero. Using As input for feature extraction, it can filter out the interference of complex background and improve the signal-to-noise ratio of visual positioning.

[0259] Example nine:

[0260] This embodiment describes a self-supervised online updating mechanism of the autoencoder network, which gives the system the ability to continuously optimize in an unknown environment.

[0261] The self-supervised learning mechanism includes an online updating step based on geometric-depth mutual supervision:

[0262] First, during the flight of the unmanned aerial vehicle, the Euclidean distance error between the geometric pose obtained by solving the PnP problem in the geometric pose feature data and the target space position estimate output by the autoencoder regression network is calculated in real time.

[0263] Calculate the Euclidean distance error :

[0264] ;

[0265] At the same time, the number of matching feature points of the current frame of visual feature points is counted .

[0266] Second, when the Euclidean distance error is less than a preset confidence interval threshold and the number of feature points of the visual image data meets the robustness requirement of PnP solving, the current geometric pose is determined as a high-confidence label.

[0267] If the current solution meets the reprojection error constraint and the number of feature matches meets , it is determined that the PnP solution has very high confidence. At this time, the system includes it as a pseudo-label, i.e. let the observed target value be used for subsequent model supervision training or state updating.

[0268] Third, the high-confidence label is used as a self-supervised signal to construct a loss function, and the autoencoder regression network is trained in reverse through online backpropagation to update the network weight parameters to adapt to the feature distribution shift caused by changes in light and tower corrosion during the inspection process.

[0269] Construct the loss function :

[0270] ;

[0271] wherein, is a L2 regularization coefficient (Weight Decay) for preventing overfitting of the network (e.g. taking value ); wherein, denotes a forward inference mapping function of the autoencoder regression network; denotes a network weight parameter matrix; denotes a square of the Frobenius Norm of the weight parameters (i.e. the sum of squares of all weight elements); is a feature tensor input to the network.

[0272] Considering the computing power limitation of the UAV on-board edge computing platform, in the online updating step, the strategy includes: freezing all weight parameters of the encoder (Encoder) and the decoder (Decoder) in the autoencoder network, and only updating the fully connected layer parameters of the regression head (RegressionHead) by back propagation. This strategy reduces the computation to within 5% of full parameter update, ensuring the real-time performance of the system.

[0273] Embodiment Ten:

[0274] The embodiment provides a multi-source data fusion autonomous positioning system, which is a physical entity for implementing the above method.

[0275] A multi-source data fusion autonomous positioning system, as shown in Figure 2 , comprising:

[0276] A data acquisition module for acquiring GPS positioning data, visual image data and laser radar point cloud data of a UAV;

[0277] A GPS preprocessing module for denoising and preprocessing the GPS positioning data by using an unscented Kalman filter algorithm to obtain a GPS state estimate value;

[0278] A joint calibration module for performing sensor joint calibration based on the visual image data and the laser radar point cloud data, establishing a mapping relationship between the camera coordinate system and the world coordinate system by solving the PnP problem, and obtaining geometric pose feature data;

[0279] A fusion positioning module for extracting image target detection data from the visual image data, and inputting the image target detection data, the GPS state estimate value and the geometric pose feature data into a pre-trained autoencoder regression network, and outputting a target spatial position estimate value of the UAV through online fusion filtering.

[0280] It will be apparent to those skilled in the art that various modifications and variations can be made to the present application without departing from the spirit or scope of the application. Thus, it is intended that the present application cover modifications and variations of this application provided they come within the scope of the appended claims and their equivalents.

Claims

1. A multi-source data fusion autonomous positioning method, characterized in that, The method is applied to a power inspection unmanned aerial vehicle and comprises the following steps: collecting multi-source sensor data of the unmanned aerial vehicle, wherein the multi-source sensor data comprises GPS positioning data, visual image data, laser radar point cloud data and inertial measurement unit data; performing noise reduction preprocessing on the GPS positioning data by using an unscented Kalman filtering algorithm to obtain a GPS state estimation value; monitoring covariance of the GPS positioning data in real time, and performing an indexing step according to whether the covariance exceeds a preset threshold value: when the covariance is less than the preset threshold value, searching for a nearest neighbor tower in a preset high-precision tower database based on the GPS state estimation value; when the covariance exceeds the preset threshold value, starting an inertial topology recursion mechanism: locking a tower ID confirmed at a previous time as an anchor point, performing double integration by using acceleration and angular velocity in the inertial measurement unit data to solve a displacement increment of the unmanned aerial vehicle relative to the anchor point, projecting the displacement increment in a direction of a preset line directional topological graph, and switching an indexing target to a next hop node ID when a projection distance exceeds a span threshold value; extracting tower profile physical features from the laser radar point cloud data, and comparing and verifying the tower profile physical features with a structure fingerprint of a corresponding ID in the high-precision tower database to determine a target tower ID; extracting image key points from the visual image data based on the target tower ID, combining the laser radar point cloud data to solve multi-dimensional geometric features, solving a PnP problem to establish a mapping relationship between a camera coordinate system and a world coordinate system, and obtaining geometric pose feature data; extracting image target detection data from the visual image data, and inputting the image target detection data, the GPS state estimation value and the geometric pose feature data into a pre-trained autoencoder regression network; performing online fusion filtering by using the autoencoder regression network, and outputting a target spatial position estimation value of the unmanned aerial vehicle.

2. The multi-source data fusion autonomous positioning method according to claim 1, characterized in that, The noise reduction preprocessing on the GPS positioning data by using the unscented Kalman filtering algorithm specifically comprises: initializing a filtering state, determining an initial state estimation value and a variance of a state vector, and performing Sigma sampling to obtain 2n+1 Sigma sampling points, wherein n is a dimension of the state vector; performing time updating according to the Sigma sampling points, calculating a state prediction value, and performing Sigma sampling again in the state prediction value; performing measurement updating according to a measurement value, and calculating a latitude and longitude state estimation value as the GPS state estimation value.

3. The multi-source data fusion autonomous positioning method according to claim 1, characterized in that, The solving of the PnP problem to establish the mapping relationship between the camera coordinate system and the world coordinate system specifically comprises: selecting four image key points as control points in the camera coordinate system, and the four control points are not coplanar; analyzing corresponding three-dimensional coordinates of the four control points in the world coordinate system, and constructing a mapping relationship between the four control points in the world coordinate system, the camera coordinate system and a laser radar point cloud coordinate system; calculating external parameters of the camera by analyzing the coordinates of the four control points, and then solving pose information of the camera to complete geometric pose solving.

4. The multi-source data fusion autonomous positioning method according to claim 1, characterized in that, The self-encoder regression network adopts a mechanism combining unsupervised learning and self-supervised learning; the image target detection data, the GPS state estimation value and the geometric pose feature data are input into the pre-trained self-encoder regression network, and specifically include: Obtaining the center coordinates, width and height and confidence of the detection box in the image target detection data, and constructing a visual feature vector; The GPS state estimation value and the geometric pose feature data are subjected to Z-Score standardization processing; The visual feature vector, the standardized GPS state estimation value and the standardized geometric pose feature data are subjected to feature dimension splicing to form an input tensor; The input tensor is subjected to online fusion filtering in the self-encoder regression network to solve the target spatial position.

5. The multi-source data fusion autonomous positioning method according to claim 1, characterized in that, The method further includes a relative positioning step for the power tower, which is realized by one of the following ways: Key point detection algorithm is used to extract power tower structure features from the visual image data, the scale of the actual object and the image pixel is calculated based on the parallax measurement principle, and the relative distance and attitude of the unmanned aerial vehicle and the power tower are solved; Or, the point cloud and the image pixel are mapped through sensor coordinate transformation of the radar and the camera, and the relative distance and attitude of the unmanned aerial vehicle and the power tower are refined according to the average distance and inclination data of the point cloud.

6. The multi-source data fusion autonomous positioning method according to claim 1, characterized in that, The method further includes an enhanced positioning step: An external positioning scheme is obtained by using visual inertial ranging technology, and the geographic positioning accuracy is enhanced in combination with the inertial navigation system of the unmanned aerial vehicle; The height data of the barometric pressure sensor are obtained, and the height error caused by barometric pressure fluctuation is corrected by using radar height finding or visual height finding data as an auxiliary, to obtain relative ground height positioning data.

7. The multi-source data fusion autonomous positioning method according to claim 2, characterized in that, The method further includes an adaptive weighting step based on electromagnetic field environment perception, specifically including: The carrier phase noise variance of the GPS positioning data and the magnetometer fluctuation variance of the inertial measurement unit are monitored in real time, and an electromagnetic interference intensity index in the power inspection scene is constructed; A nonlinear mapping relationship between the electromagnetic interference intensity index and the measurement noise covariance matrix in the unscented Kalman filtering algorithm is established; When the electromagnetic interference intensity index exceeds a preset safety threshold, the measurement noise covariance matrix is adaptively inflated, the confidence weight of the GPS positioning data in state updating is reduced, and the fusion weight of the visual image data and the laser radar point cloud data in the self-encoder regression network is increased at the same time, so as to suppress the positioning drift caused by strong electromagnetic field.

8. The multi-source data fusion autonomous positioning method according to claim 1, characterized in that, The method further includes a visual anti-interference step based on radar reflection intensity guidance, specifically including: The reflection intensity information in the laser radar point cloud data is obtained, the reflection intensity threshold is set according to the high reflectivity characteristic of the power tower metal material, and the tower point cloud with high reflection intensity and the background point cloud with low reflection intensity are separated; The tower point cloud with high reflection intensity is projected onto the imaging plane of the visual image data to generate a region of interest mask containing the tower metal structure; In the extracting image target detection data from the visual image data, the visual image is spatially filtered by using the region of interest mask, only the pixel features within the region of interest mask are reserved as valid inputs, and the tree vegetation and mountain texture interference in the power inspection background are filtered out.

9. The multi-source data fusion autonomous positioning method according to claim 4, characterized in that, In the autoencoder regression network, the self-supervised learning mechanism includes an online updating step based on geometric-depth mutual supervision: In the unmanned aerial vehicle flight process, the Euclidean distance error between the geometric pose solved by the PnP problem in the geometric pose feature data and the target space position estimation value output by the autoencoder regression network is calculated in real time; When the Euclidean distance error is less than a preset confidence interval threshold and the number of feature points of the visual image data meets the robustness requirement of PnP solving, the current geometric pose is determined as a high confidence label; The high confidence label is used as a self-supervised signal to construct a loss function, and the autoencoder regression network is trained by online back propagation to update the network weight parameters to adapt to the feature distribution shift caused by light changes and tower corrosion in the inspection process.

10. A multi-source data fusion autonomous positioning system, characterized in that, Comprise: A data acquisition module for acquiring GPS positioning data, visual image data and laser radar point cloud data of an unmanned aerial vehicle; A GPS preprocessing module for denoising and preprocessing the GPS positioning data by using an unscented Kalman filter algorithm to obtain a GPS state estimation value; An intelligent indexing module for monitoring the covariance of the GPS positioning data in real time; when the covariance exceeds a preset threshold, an inertial topological recursion is performed by using inertial measurement unit data to lock the tower ID confirmed at the last time as an anchor point and calculate a displacement increment, predict a target tower ID based on a line directed topological graph, and perform fingerprint verification by using the structural features of the laser radar point cloud data; A joint calibration module for performing sensor joint calibration based on the determined target tower ID, the visual image data and the laser radar point cloud data, establishing a mapping relationship between a camera coordinate system and a world coordinate system by solving a PnP problem, and obtaining geometric pose feature data; A fusion positioning module for extracting image target detection data from the visual image data, and inputting the image target detection data, the GPS state estimation value and the geometric pose feature data into a pre-trained autoencoder regression network to output a target space position estimation value of the unmanned aerial vehicle through online fusion filtering.

Images