Transformer internal inspection robot navigation method based on drawing and internal anchor point constraint

Abstract

The present application relates to electric power robot navigation positioning technical field, specifically to a kind of transformer internal inspection robot navigation method based on drawing and internal anchor point constraint, comprising: the real-time image of robot collected to the oil tank inside is repaired with image enhancement processing to the environment feature under oil;The visual semantic features extracted are matched with multi-modal navigation anchor point knowledge base, and navigation anchor point is identified;Based on the navigation anchor point identified by feature matching, the real-time pose of robot relative to navigation anchor point is calculated;The current dead reckoning data of robot is obtained, and robot global pose correction is carried out;With the global pose after correction as reference, path planning based on structural design drawing constraint is carried out.The method continuously fuses the dead reckoning data of inertial navigation unit and high-confidence anchor point absolute coordinate observation value, drives robot to autonomously, safely and accurately reach preset inspection point, and completes close observation task.

Description

Technical Field

[0001] This invention relates to the field of navigation and positioning technology for power robots, specifically a navigation method for a transformer internal inspection robot based on drawings and internal anchor point constraints. Background Technology

[0002] Currently, transformer internal inspections mainly rely on power outage core lifting or endoscopic inspection, which are time-consuming, costly, and have limited visibility, making it difficult to meet the needs of intelligent transformer operation and maintenance. To improve inspection efficiency and reduce human risks, in recent years, technical solutions have emerged that utilize submersible internal inspection robots to complete transformer internal inspections.

[0003] However, the interior of a transformer oil tank is a closed metal space where satellite navigation signals are unavailable. The high-voltage electromagnetic environment interferes with sensors such as magnetometers, and the refraction and turbidity changes in transformer oil weaken the stability of visual perception, making autonomous navigation and precise positioning of the internal inspection robot quite difficult. Existing navigation methods in GPS-free environments mostly rely on inertial navigation or vision-based simultaneous localization and mapping (SLAM) techniques. These methods are prone to cumulative drift during long-term operation, and in the highly repetitive structural textures of the transformer's windings and pressure plates, visual features are easily confused, leading to the loss of positioning references.

[0004] To improve positioning performance, some technologies have introduced multi-source sensor fusion or semantic recognition methods. However, since they do not form effective constraints with the spatial information in the original structural design drawings of the transformer, the existing methods still mainly achieve relative positioning, making it difficult to achieve high-precision absolute pose reset in the complex and obstructed environment inside the oil tank.

[0005] For example, CN120949126A proposes a method for an internal inspection robot to operate in the working state of a large transformer, but its navigation method mainly relies on manual intervention or simple motion control; CN111984005B discloses an internal inspection robot and control system for an oil-immersed transformer, which solves the problem of motion and posture control of the robot in the oil-immersed environment, but does not involve an autonomous positioning method based on structural priors.

[0006] In summary, the existing navigation technology for transformer internal inspection robots generally has the following shortcomings: (1) positioning errors are easy to accumulate under conditions of no GPS and strong electromagnetic interference; (2) the internal structure is regular and repetitive, lacking a stable absolute positioning reference; (3) prior information such as structural design drawings is not effectively utilized, making it difficult to achieve fine correction and global reset of the pose. Summary of the Invention

[0007] To address the aforementioned problems, this invention provides a navigation method for a transformer internal inspection robot based on drawings and internal anchor point constraints, comprising: S1. Perform underwater environment feature restoration and image enhancement processing on real-time images collected by the robot submerged inside the oil tank; perform real-time visual recognition on the processed images to determine candidate components, extract visual semantic features of the candidate components, and perform feature matching between the extracted visual semantic features and the multimodal navigation anchor point knowledge base to identify navigation anchor points. The method for constructing the multimodal navigation anchor point knowledge base is as follows: based on the transformer structural design drawings, the absolute spatial coordinates of the internal structural components are extracted, and the geometric constraint models of each structural component are established; a multidimensional visual sample set of the structural components is established, and visual semantic features are extracted; the absolute spatial coordinates, geometric constraint models and visual semantic features are mapped and associated to establish three-dimensional key-value pair index data of each structural component, and the three-dimensional key-value pair index data of all structural components are saved to construct the multimodal navigation anchor point knowledge base; S2. Based on the navigation anchor points identified by feature matching, calculate the robot's real-time pose relative to the navigation anchor points; and construct a confidence evaluation function to determine the valid navigation anchor points with high confidence, and retain the pose parameters calculated from the anchor points. S3. Obtain the robot's current dead reckoning data, and perform global pose correction of the robot based on the coordinates of the high-confidence valid navigation anchor points and their calculated pose parameters. S4. Based on the corrected global pose, perform path planning based on the constraints of the structural design drawings.

[0008] The specific operations described in S1 for establishing a multidimensional visual sample set of structural components and extracting visual semantic features are as follows: Visual image samples of transformer internal structural components under different operating conditions are collected, and the visual images are uniformly labeled and preprocessed to construct a multi-dimensional visual sample set. The preprocessing includes image scale normalization, brightness and contrast adjustment and noise suppression. Based on a multi-dimensional visual sample set, visual semantic features of each structural component are extracted using a deep learning neural network, specifically: All visual image samples from the multidimensional visual sample set are input into a pre-trained convolutional neural network. The 2048-dimensional knowledge base feature vector output from the penultimate fully connected layer of the convolutional neural network is extracted. The feature vector contains high-level semantic and appearance information of the structural components, which is the visual semantic feature of the structural components.

[0009] The specific operations described in S1 for performing submersible environment feature restoration and image enhancement processing on real-time images collected by the robot submerged inside the oil tank are as follows: Obtain the intrinsic parameter matrix K of the camera when it is in air and the radial distortion coefficient of the lens. k 1 , k 2 、k3 and tangential distortion coefficient p 1 , p 2 ; The intrinsic parameter matrix and distortion coefficients are corrected based on the refractive index of transformer oil, and a distortion model is used to perform inverse mapping on each frame of the original image acquired by the camera for geometric distortion correction; the distortion model formula is: in, These are the pixel coordinates before distortion. These are the pixel coordinates after distortion. , The radial distortion coefficient is... The tangential distortion coefficient; Color correction and texture enhancement are performed on the geometrically distorted image. The distorted image is converted from the RGB color space to the Lab color space. The mean values ​​of the a and b channels of the entire image are calculated and linearly offset compensation is performed on the neutral point to remove the color cast caused by oil aging. The CLAHE algorithm is applied to the L channel to enhance the contrast between the component surface and the background. The Gaussian-Laplacian pyramid fusion algorithm is used to sharpen the image edges.

[0010] The operation described in S1, which involves real-time visual recognition of the processed image to determine candidate components, specifically includes: Each frame of the image is pixel-scaled, and the scaled image is input into a deep learning model. The deep learning model uses the YOLOv8s object detection network, which is configured with a confidence threshold of 0.5 and a non-maximum suppression threshold of 0.45. The deep learning model outputs bounding boxes and class probabilities of suspected structural components in the image; candidate components are confirmed based on the bounding boxes, and the region image is cropped. The cropped region image is preprocessed, and real-time feature vectors of candidate components are extracted using a deep learning neural network. .

[0011] The process described in S1, which involves feature matching the extracted visual semantic features with a multimodal navigation anchor knowledge base, includes: Calculate real-time feature vectors All feature vectors in the multimodal navigation anchor knowledge base index cosine similarity Perform a first-level visual feature similarity matching, retrieve and return the top N candidate anchor points with the highest cosine similarity, and compare the cosine similarity of the candidate anchor points with a preset visual matching threshold. If the highest cosine similarity is found to be the highest, the candidate anchor point is selected. Less than the set threshold If the visual screening fails, the candidate anchor point is discarded. For each candidate anchor point that passes the initial visual screening, geometric verification is performed based on the established geometric constraint model. This is achieved by measuring the real-time dimensions of the candidate components in the real-time image and calculating the relative error between the real-time measured dimensions and the standard geometric dimensions. Perform double geometric constraint matching; set a geometric matching threshold. If the relative error Greater than the geometric matching threshold If so, the double geometric constraint matching is deemed to have failed; Only when a single candidate anchor point satisfies And its corresponding If the candidate anchor point is confirmed to be valid, it will be used as the navigation anchor point.

[0012] The calculation of the robot's real-time pose relative to the navigation anchor points, as described in S2 based on feature matching identification, is specifically performed as follows: The robot's real-time 3D distance and deflection angle relative to the navigation anchor point are calculated. A local coordinate system is established with the anchor point center as the origin. The robot's pose parameters are then calculated. satisfy: in, These are the pixel coordinates of the image; This is the camera intrinsic parameter matrix; This is the translation vector, corresponding to the three-dimensional distance deviation between the robot and the navigation anchor point; These are the robot's coordinates in the global right-handed coordinate system. It is a 3×3 rotation matrix, determined by the attitude angles. It is derived from the Euler angle rotation formula, that is: ; in, Roll angle: the rotation angle of the robot around the X-axis of the global coordinate system; Pitch angle: the rotation angle of the robot about the Y-axis of the global coordinate system; Yaw angle: the rotation angle of the robot around the Z-axis of the global coordinate system.

[0013] S3 describes acquiring the robot's current dead reckoning data and performing global pose correction based on the coordinates of high-confidence valid navigation anchor points and their calculated pose parameters. The specific operation is as follows: S301. Obtain dead reckoning data output by the robot's inertial navigation unit, wherein the dead reckoning data includes predicted pose. and its covariance matrix ; S302. The absolute coordinates corresponding to the high-confidence valid navigation anchor points obtained by the determination are used as observation values. Based on the observation values ​​and the pose parameters calculated by the anchor points, the extended Kalman filter is used to update the state, so as to realize the accurate reset of the robot dead reckoning data in the global coordinate system.

[0014] The specific method for state updating using extended Kalman filtering as described in S302 is as follows: Define the system state vector ,in For the robot's position coordinates, For attitude angle, Linear velocity; Establish prediction equations ,in, Here is the state transition matrix. To control the input matrix, X k-1 It is the global pose state vector from the previous moment. This is a motor speed control variable. For process noise, obey distributed, Here is the process noise covariance matrix; k represents the current time. Establish observation equations ,in, These are the absolute coordinates of the anchor point observations. For the observation matrix, To observe noise, obey distributed, To observe the noise covariance matrix; Based on the predicted pose and predict covariance matrix Calculate Kalman gain ; Using anchor point observations The state is updated to obtain the corrected optimal estimated state, as shown in the following formula: in, To predict pose; This is the global pose state vector corrected at the previous moment; The covariance updated at the previous moment; This is the corrected global pose state vector at the current moment; This is the covariance updated at the current moment; It is an identity matrix.

[0015] S4 describes path planning based on the corrected global pose and constraints from the structural design drawings. The specific operation is as follows: S401. Based on the target location inspection command, record the prior environmental information in a 3D map, and mark the prohibited areas, recommended cooling oil passages, and the locations of all key anchor points; using the latest output global pose as the starting point and the target location coordinates of the multimodal navigation anchor point knowledge base as the target point, plan the optimal path based on the discretized network map and construct the cost function: in, The actual cost from the starting point to node n. The heuristic cost from node n to the target point; S402. Compare the target pose output by the path planning with the actual global pose of the robot, calculate the pose deviation, and calculate the desired body speed control command and thruster control command based on the pose deviation to perform closed-loop motion control of the robot.

[0016] The method described in S402 for calculating the attitude deviation and calculating the desired body speed control command and thruster adjustment command based on the attitude deviation is as follows: The formula for calculating pose deviation is: Wherein, ΔX represents the robot pose deviation vector; Δx represents the robot's position deviation in the X-axis direction of the global coordinate system; Δy represents the robot's position deviation in the Y-axis direction of the global coordinate system; Δz represents the robot's position deviation in the Z-axis direction of the global coordinate system; Δα represents the robot's roll angle deviation; Δβ represents the robot's pitch angle deviation; and Δγ represents the robot's yaw angle deviation. The attitude deviation is input into the cascade PID controller: the outer loop position PID calculates the desired body speed command based on the position error; the inner loop position PID calculates the thrust or speed command of each of the four thrusters based on the desired speed, the current speed, and the attitude error.

[0017] Compared with the prior art, the beneficial effects of the present invention are: (1) This invention builds a multimodal anchor point knowledge base based on transformer structural design drawings and uses internal fixed structural components as absolute spatial references. It breaks through the limitation of traditional visual SLAM in the repetitive texture environment where the positioning reference is easily lost, and realizes the technical leap from relative positioning to absolute coordinate navigation. The positioning accuracy can reach the centimeter level.

[0018] (2) The present invention designs a special physical compensation scheme for the optical properties of transformer oil. Through the combination of refractive index correction, spectral compensation and texture enhancement algorithms, it effectively solves the visual degradation problem caused by oil refraction, turbidity and color deviation, ensuring that the robot can stably perceive the environment under different oil conditions and operating conditions, and significantly improving the robustness of visual recognition.

[0019] (3) This invention proposes a Kalman filter data fusion mechanism of inertial navigation position estimation + anchor point absolute constraint, which overcomes the defects of single navigation means being susceptible to electromagnetic interference and cumulative drift. Combined with the prior space constraints of structural design drawings, even in deep and narrow flow channels with limited vision, the robot can achieve safe passage and accurate posture reset.

[0020] (4) The present invention introduces a multi-criteria confidence evaluation model, which can adaptively balance the weights of each sensor according to the environmental visibility and feature matching quality under dynamic flow field, effectively suppressing the positioning jump caused by sensor observation anomalies, and greatly improving the continuity and positioning reliability of inspection tasks under complex working conditions. Attached Figure Description

[0021] The solutions and advantages of this application will become clear to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of the invention.

[0022] In the attached diagram: Figure 1 This is a flowchart illustrating the steps of the transformer internal inspection robot navigation method described in this invention. Figure 2 This is a flowchart of the navigation anchor point identification and confidence assessment process of the present invention; Figure 3 This is a schematic diagram of the navigation for a transformer internal inspection robot. Detailed Implementation

[0023] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0024] Example This embodiment provides a navigation method for a transformer internal inspection robot based on drawings and internal anchor point constraints, including the construction of a multimodal anchor point knowledge base, underwater image enhancement, real-time anchor point recognition, dual matching verification, global pose reset, path planning, and closed-loop control execution.

[0025] This embodiment uses a 220kV oil-immersed power transformer (model SFP-240000 / 220) as an example to implement the navigation method described in this invention. The internal structure of the oil-immersed power transformer's tank is complex, including typical structures such as high-voltage windings, lead wire supports, and tap changers. The robot platform performing the inspection task is a pressure-resistant submersible robot equipped with a Hikvision MV-CH200-10UM industrial camera, a nine-axis IMU (integrating a three-axis accelerometer, a three-axis gyroscope, and a three-axis magnetometer, model: TDKICM-20948), and four brushless DC thrusters. Figure 1 As shown, the transformer internal inspection robot navigation method includes the following steps: S1. Perform underwater environment feature restoration and image enhancement processing on real-time images collected by the robot submerged inside the oil tank; perform real-time visual recognition on the processed images to determine candidate components, extract visual semantic features of the candidate components, and perform feature matching between the extracted visual semantic features and the multimodal navigation anchor point knowledge base to identify navigation anchor points. Before performing S1 feature matching, this navigation method also includes constructing a multimodal navigation anchor point knowledge base. The method for constructing the multimodal navigation anchor point knowledge base is as follows: Based on the transformer structural design drawings, the absolute spatial coordinates of the internal structural components are extracted, and geometric constraint models of each structural component are established; a multidimensional visual sample set of the structural components is established, and visual semantic features are extracted; the absolute spatial coordinates, geometric constraint models, and visual semantic features are mapped and associated to establish three-dimensional key-value pair index data for each structural component; and the three-dimensional key-value pair index data of all structural components are saved to construct the multimodal navigation anchor point knowledge base. The specific operations include: S101. Establish a global coordinate system based on the transformer's structural design drawings; calibrate the absolute spatial coordinates of the transformer's internal structural components based on the global coordinate system; Based on the structural design drawings of the transformer, a global right-handed coordinate system is established with the geometric center of the bottom surface of the transformer tank as the origin O, the major axis of the transformer as the X-axis of the world coordinate system, the minor axis as the Y-axis, and the vertical upward direction as the Z-axis.

[0026] Using the assembly constraints and dimensions in the structural design drawings, the absolute spatial coordinates of key internal structural components of the transformer in the global right-handed coordinate system are accurately determined. The coordinate accuracy is at the millimeter level.

[0027] The key structural components include A / B / C three-phase high-voltage bushings, on-load tap changer operating mechanism, diagonal bolts on the high-voltage winding upper pressure plate, and three-phase lead brackets.

[0028] The absolute spatial coordinates of the internal structural components of the calibration transformer include the center points of the flange faces at the root of the A / B / C three-phase high-voltage bushings, the corner points of the on-load tap changer operating mechanism housing, the center points of the four diagonal bolts on the upper pressure plate of the high-voltage winding, the center point of the fixed base of the three-phase lead bracket, and typical bending points.

[0029] S102. Obtain the standard geometric dimension parameters of the structural components based on the transformer structural design drawings, and establish the geometric constraint model of each structural component; This embodiment takes the upper pressure plate bolt of the high-voltage winding as an example. Its standard parameters are: nominal bolt diameter mm, center distance between adjacent bolts mm, hexagonal distance between sides of bolt head mm .

[0030] Based on a pinhole camera model, a projection relationship between physical dimensions and image pixel dimensions is established. For a given physical dimension... (Unit: mm), its projected length on the image (Unit: pixels) depends primarily on the camera's focal length. (Unit: pixels) and the distance of the component from the camera (Unit: mm), the relationship can be approximated as: .

[0031] For each structural component, three key measurable dimensions—diameter, spacing, and aspect ratio—are selected. The aforementioned projection relationship equations are constructed to form the geometric constraint model G of that component, which is used for subsequent calculations of dimensional relative errors. .

[0032] S103. Collect visual image samples of the internal structural components of the transformer under different operating conditions, perform unified annotation and preprocessing on the visual images, and construct a multi-dimensional visual sample set. For the above four types of key structural components, samples were collected or generated under multiple working conditions, with a total of 1,000 samples per component, of which 500 were collected in real-world scenarios and 500 were generated through simulation.

[0033] Operating conditions covered: Five shooting distances were selected, including 5cm, 10cm, 20cm, 30cm, and 50cm. Five shooting angles were selected, including -60°, -30°, 0°, 30°, and 60°; Five illuminance levels were selected for the lighting conditions: 5 lux, 50 lux, 100 lux, 300 lux, and 500 lux. Four turbidity levels were selected for oil transparency: 5 NTU, 20 NTU, 35 NTU, and 50 NTU.

[0034] Then, the visual image samples are uniformly labeled and preprocessed for subsequent learning and matching of visual semantic features. The preprocessing includes image scale normalization, brightness and contrast adjustment and noise suppression.

[0035] The unified annotation and preprocessing of visual image samples specifically includes: Labelling method: Using Labelling, the regions containing key structural components in each image are precisely labeled with rectangular boxes, and two labels are associated with each label box, including the component category and the standard geometric dimension parameters in S102; Scale normalization: Crops out all labeled areas and scales them to 224×224 pixels, scaling the shorter side proportionally and filling the longer side with black pixels. Brightness and contrast adjustment: Convert the image to the CIE Lab color space and apply the Limit Contrast Adaptive Histogram Equalization (CLAHE) algorithm to the L (brightness) channel with the following parameters: block size of 8×8 and contrast limit of 2.0. Noise suppression: A bilateral filter is applied for smoothing and denoising, with the parameters set as follows: spatial domain standard deviation. Color gamut standard deviation It suppresses oil particles and shooting noise while preserving the surface texture details of the components.

[0036] S104. Based on a multi-dimensional visual sample set, visual semantic features of each structural component are extracted using a deep learning neural network. The specific method is as follows: All visual image samples from the multidimensional visual sample set are input into a pre-trained ResNet-50 convolutional neural network. The 2048-dimensional knowledge base feature vector output from the penultimate fully connected layer of the convolutional neural network is extracted. The feature vector contains high-level semantic and appearance information of the structural components, which is the visual semantic feature of the structural components.

[0037] S105. Map and associate absolute spatial coordinates, geometric constraint models and visual semantic features to establish three-dimensional key-value pair index data for each structural component, and save the three-dimensional key-value pair index data of all structural components to build a multimodal navigation anchor point knowledge base. Visual semantic feature vectors of key structural components Its absolute spatial coordinates P and its geometric constraint model G are linked one-to-one to form a three-dimensional key-value pair index data: The system uses the FAISS vector database to store all 3D key-value pair index data, supports fast retrieval based on cosine similarity, has a retrieval response time of ≤50ms, and can quickly match corresponding anchor point information through real-time feature vectors, providing data support for subsequent real-time matching.

[0038] After constructing the multimodal navigation anchor point knowledge base, the S1 operation is performed to repair underwater environmental features and enhance images of the real-time images collected by the robot inside the oil tank. The specific operation is as follows: Obtain the intrinsic parameter matrix K of the camera when it is in air and the radial distortion coefficient of the lens. k 1 , k 2 and tangential distortion coefficient p 1 , p 2 ; The intrinsic parameter matrix and distortion coefficients are corrected based on the refractive index of transformer oil, and the Brown-Conrady distortion model is used to reverse-map each frame of the original image acquired by the camera for geometric distortion correction; the distortion model formula is as follows: in, These are the pixel coordinates before distortion. These are the pixel coordinates after distortion. , The radial distortion coefficient is... denoted as the tangential distortion coefficient.

[0039] Color correction and texture enhancement are performed on images with geometric distortion correction, specifically including: The distortion-corrected image was converted from the RGB color space to the Lab color space. To address the yellowish-brown tone caused by transformer oil aging, the mean values ​​of channels a and b of the entire image were calculated and linearly offset towards the neutral point to effectively remove color cast. Subsequently, the CLAHE algorithm was applied to the L channel again, significantly enhancing the contrast between the component surface and the background, highlighting subtle textures such as winding insulation paper wrinkles, bolt corrosion, and metal scratches. Finally, a lightweight Gaussian-Laplacian pyramid fusion algorithm was used to further sharpen edges while suppressing any residual local noise after correction.

[0040] The specific operation of performing real-time visual recognition on the processed image to determine candidate components is as follows: Each frame of the image is pixel-scaled, and the scaled image is input into a deep learning model. The deep learning model outputs bounding boxes and class probabilities of suspected structural components in the image. Candidate components are confirmed based on the bounding boxes, and region image cropping is performed. The cropped region image is then preprocessed, and real-time feature vectors of the candidate components are extracted using the deep learning neural network in S104 above. .

[0041] The deep learning model uses the YOLOv8s object detection network, which is configured with a confidence threshold of 0.5 and a non-maximum suppression (NMS) threshold of 0.45.

[0042] The specific steps for matching the extracted visual semantic features with the multimodal navigation anchor point knowledge base are as follows: Calculate real-time feature vectors All feature vectors in the multimodal navigation anchor knowledge base index cosine similarity A first-level visual feature similarity matching is performed, and its calculation formula is as follows: Set the visual matching threshold T v The system retrieves and returns the top 5 candidate anchors with the highest similarity. These candidate anchors are then compared to a visual matching threshold. The highest cosine similarity is selected as the most similar anchor. Less than the set threshold If the visual screening fails, the candidate anchor point is discarded.

[0043] For each candidate anchor point that passes the initial visual screening, geometric verification is performed based on the established geometric constraint model. This is achieved by measuring the real-time dimensions of the candidate components in the real-time image and calculating the relative error between the real-time measured dimensions and the standard geometric dimensions. Perform double geometric constraint matching; the relative error calculation formula is: in, To measure dimensions in real time, These are standard geometric dimensions.

[0044] Similarly, set a geometric matching threshold. That is, a 15% error. If the relative error is greater than the geometric matching threshold, the double geometric constraint matching is considered to have failed.

[0045] Only when a single candidate anchor point satisfies And its corresponding If the candidate anchor point is confirmed to be valid, it will be used as the navigation anchor point, and its matching information will be recorded.

[0046] S2. Based on the navigation anchor points identified through feature matching, calculate the robot's real-time pose relative to the navigation anchor points; and construct a confidence evaluation function to determine valid navigation anchor points with high confidence, retaining the pose parameters calculated from these anchor points; such as Figure 2 As shown, the specific operation is as follows: S201. The EPnP algorithm is used to calculate the real-time 3D distance and deflection angle of the robot relative to the navigation anchor point. A local coordinate system is established with the center of the anchor point as the origin. The robot's pose parameters are... satisfy: in These are the pixel coordinates of the image; This is the camera intrinsic parameter matrix; This is the translation vector, corresponding to the three-dimensional distance deviation between the robot and the navigation anchor point; These are the robot's coordinates in the global coordinate system. It is a 3×3 rotation matrix, determined by the attitude angles. It is derived from the Euler angle rotation formula, that is: ; in Roll angle: The rotation angle of the robot around the X-axis of the global coordinate system, with a rotation range of ±180°, and clockwise rotation is defined as positive; Pitch angle: The rotation angle of the robot around the Y-axis of the global coordinate system, with a rotation range of ±90°, and upward pitch is defined as positive; Yaw angle: The rotation angle of the robot around the Z-axis of the global coordinate system, with a rotation range of ±180°, and clockwise rotation is defined as positive.

[0047] S202. Construct a multi-criteria confidence evaluation function that integrates visual matching quality, geometric matching accuracy, and anchor point occlusion. The formula for the confidence evaluation function is as follows: in, Confidence level (0≤ ≤1), These are weighting coefficients, set to 0.5, 0.3, and 0.2 respectively in this example. Anchor point occlusion rate (0≤ ≤1).

[0048] The proportion of occluded regions is calculated using an image segmentation algorithm, and a confidence threshold is set. =0.75, if the calculated If the observation data of the anchor point is valid, the pose parameters it provides will be used for global correction; otherwise, it is considered a low-quality observation, and the robot's relative pose parameters calculated from it will be deleted.

[0049] S3. Obtain the robot's current dead reckoning data, and perform global pose correction of the robot based on the coordinates of the high-confidence valid navigation anchor points and their calculated pose parameters. The specific operation is as follows: S301. Obtain dead reckoning data output by the robot's inertial navigation unit, wherein the dead reckoning data includes predicted pose. and its covariance matrix ; During the period when the robot fails to acquire a valid navigation anchor point, its own system relies on an inertial measurement unit (IMU) for pose prediction. The IMU provides angular velocity. and acceleration Attitude calculation is performed using complementary filtering, and velocity and position changes are obtained by integration.

[0050] Meanwhile, the motor encoder provides feedback on the thruster speed, and combined with the pre-calibrated robot hydrodynamic coefficient model, the robot's speed is estimated.

[0051] By fusing the velocity calculated by the IMU with the velocity estimated by the dynamic model using Kalman filtering, a more reliable estimate of the organism's velocity is obtained.

[0052] Finally, by integrating the fused velocity and combining it with the calculated pose, the robot's predicted pose is obtained. and its covariance matrix .

[0053] S302. Using the absolute coordinates corresponding to the high-confidence valid navigation anchor points as observations, and based on the observations and the pose parameters calculated from the anchor points, an extended Kalman filter is used for state updates to achieve accurate resetting of the robot dead reckoning data in the global coordinate system; the specific operation is as follows: Define the system state vector ,in For the robot's position coordinates, For attitude angle, Linear velocity; Establish prediction equations ,in, Here is the state transition matrix. To control the input matrix, X k-1 It is the global pose state vector from the previous moment. This is a motor speed control variable. For process noise, obey distributed, Here is the process noise covariance matrix; k represents the current time. Establish observation equations ,in, These are the absolute coordinates of the anchor point observations. For the observation matrix, To observe noise, obey distributed, To observe the noise covariance matrix; Based on the predicted pose and predict covariance matrix Calculate Kalman gain ; Using anchor point observations Perform a state update to obtain the corrected optimal estimated state, which is the robot's corrected global pose state vector. and the updated covariance The specific formula is as follows: in, To predict pose; This is the global pose state vector corrected at the previous moment; The covariance updated at the previous moment; This is the corrected global pose state vector at the current moment; The covariance updated at the current moment; It is an identity matrix.

[0054] S4. Using the corrected global pose as a reference, perform path planning based on the constraints of the structural design drawings, such as... Figure 3 As shown, the specific operation is as follows: S401. Based on the target location inspection command, record the prior environmental information in a 3D map, and mark the prohibited areas, recommended cooling oil passages, and the locations of all key anchor points; using the latest output global pose as the starting point and the target location coordinates of the multimodal navigation anchor point knowledge base as the target point, plan the optimal path based on the discretized network map and construct the cost function: in, The actual cost from the starting point to node n. Let n be the heuristic cost from node n to the target point.

[0055] Strictly adhere to the constraints during planning: the route must be entirely within the permitted traffic area and maintain a safe distance of at least 0.5m from high-voltage hazard areas.

[0056] S402. Compare the target pose output by the path planning with the actual global pose of the robot, calculate the pose deviation, and calculate the desired body speed control command and thruster control command based on the pose deviation to perform closed-loop motion control of the robot. The formula for calculating the pose deviation is: Where ΔX represents the robot pose deviation vector; Δx represents the robot's position deviation in the X-axis direction of the global coordinate system; Δy represents the robot's position deviation in the Y-axis direction of the global coordinate system; Δz represents the robot's position deviation in the Z-axis direction of the global coordinate system; Δα represents the robot's roll angle deviation; Δβ represents the robot's pitch angle deviation; and Δγ represents the robot's yaw angle deviation.

[0057] The attitude deviation is input into a cascaded PID controller: the outer loop position PID calculates the desired body speed command based on the position error; the inner loop position PID calculates the thrust or rotational speed command for each of the four thrusters based on the desired speed, current speed, and attitude error. : in, , , The PID control parameters are identified and tuned through robot system simulation.

[0058] Driven by the thruster control commands, the robot moves toward the target point, forming a closed loop of perception-localization-planning-control. Ultimately, the robot is guided to approach the target position stably and accurately for high-definition photography and status checks.

[0059] Working principle: First, in the offline stage, the value of the transformer's three-dimensional structural design drawings is deeply explored, the absolute spatial coordinates and standard geometric dimensions of the internal fixed components are extracted, and their visual features under multiple working conditions in real oil are integrated to build a multimodal navigation anchor point knowledge base that associates absolute coordinates, geometric models and visual semantic features. This transforms the static information in the drawings into dynamic spatial landmarks that can be recognized and used by the robot.

[0060] During the online inspection phase, after the robot enters the oil tank, its vision system first performs real-time distortion correction and enhancement processing on the original image based on the optical and physical properties of the oil to suppress interference caused by refraction, turbidity, and color cast. Subsequently, a deep learning model performs real-time semantic recognition on the enhanced image, extracting the visual features of candidate components. This is then rigorously matched against a knowledge base for both visual feature similarity and geometric size consistency to ensure the reliability of anchor point recognition. After a successful match, a perspective geometry algorithm is used to calculate the robot's precise relative pose to the anchor point, and the confidence level is assessed based on the matching quality.

[0061] The core innovation of this navigation system lies in its closed-loop pose correction mechanism based on absolute coordinate constraints. This method continuously integrates dead reckoning data from the inertial navigation unit with high-confidence anchor point absolute coordinate observations obtained through the aforementioned process. By employing a Kalman filter algorithm, the anchor point coordinates are injected as strong constraints into the state estimation, achieving periodic and precise resetting of the robot's global pose, thereby completely suppressing the cumulative drift error of inertial navigation during long-term operation.

[0062] Finally, based on the corrected global pose, and combined with the prior environment map extracted from the same structural design drawings, online path planning and closed-loop motion control are performed to drive the robot to autonomously, safely, and accurately reach the preset inspection point and complete the close-range observation task.

[0063] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the rights involved.

Claims

1. A navigation method for a transformer internal inspection robot based on drawings and internal anchor point constraints, characterized in that, include: S1. Perform submerged environment feature restoration and image enhancement processing on real-time images collected by the robot that has entered the oil tank; The processed image is subjected to real-time visual recognition to determine candidate components, the visual semantic features of the candidate components are extracted, and the extracted visual semantic features are matched with the multimodal navigation anchor point knowledge base to identify navigation anchor points. The method for constructing the multimodal navigation anchor point knowledge base is as follows: based on the transformer structural design drawings, the absolute spatial coordinates of the internal structural components are extracted, and the geometric constraint models of each structural component are established; a multidimensional visual sample set of the structural components is established, and visual semantic features are extracted; the absolute spatial coordinates, geometric constraint models and visual semantic features are mapped and associated to establish three-dimensional key-value pair index data of each structural component, and the three-dimensional key-value pair index data of all structural components are saved to construct the multimodal navigation anchor point knowledge base; The step of matching the extracted visual semantic features with the multimodal navigation anchor point knowledge base includes: Calculate real-time feature vectors All feature vectors in the multimodal navigation anchor knowledge base index cosine similarity Perform a first-level visual feature similarity matching, retrieve and return the top N candidate anchor points with the highest cosine similarity, and compare the cosine similarity of the candidate anchor points with a preset visual matching threshold. If the highest cosine similarity is found to be the highest, the candidate anchor point is selected. Less than the set threshold If the visual screening fails, the candidate anchor point is discarded. For each candidate anchor point that passes the initial visual screening, geometric verification is performed based on the established geometric constraint model. This is achieved by measuring the real-time dimensions of the candidate components in the real-time image and calculating the relative error between the real-time measured dimensions and the standard geometric dimensions. Perform double geometric constraint matching; set a geometric matching threshold. If the relative error Greater than the geometric matching threshold If so, the double geometric constraint matching is deemed to have failed; Only when a single candidate anchor point satisfies And its corresponding If the candidate anchor point is confirmed to be valid, it will be used as the navigation anchor point. S2. Based on the navigation anchor points identified by feature matching, calculate the robot's real-time pose relative to the navigation anchor points; and construct a confidence evaluation function to determine the valid navigation anchor points with high confidence, and retain the pose parameters calculated from the anchor points. S3. Obtain the robot's current dead reckoning data, and perform global pose correction of the robot based on the coordinates of the high-confidence valid navigation anchor points and their calculated pose parameters. S4. Based on the corrected global pose, perform path planning based on the constraints of the structural design drawings.

2. The transformer internal inspection robot navigation method based on drawings and internal anchor point constraints according to claim 1, characterized in that, The specific operations described in S1 for establishing a multidimensional visual sample set of structural components and extracting visual semantic features are as follows: Visual image samples of transformer internal structural components under different operating conditions are collected, and the visual images are uniformly labeled and preprocessed to construct a multi-dimensional visual sample set. The preprocessing includes image scale normalization, brightness and contrast adjustment and noise suppression. Based on a multi-dimensional visual sample set, visual semantic features of each structural component are extracted using a deep learning neural network, specifically: All visual image samples from the multidimensional visual sample set are input into a pre-trained convolutional neural network. The 2048-dimensional knowledge base feature vector output from the penultimate fully connected layer of the convolutional neural network is extracted. The feature vector contains high-level semantic and appearance information of the structural components, which is the visual semantic feature of the structural components.

3. The transformer internal inspection robot navigation method based on drawings and internal anchor point constraints according to claim 1, characterized in that, The specific operations described in S1 for performing submersible environment feature restoration and image enhancement processing on real-time images collected by the robot submerged inside the oil tank are as follows: Obtain the intrinsic parameter matrix K of the camera when it is in air and the radial distortion coefficient of the lens. k 1 , k 2 、k 3 and tangential distortion coefficient p 1 , p 2 ; The intrinsic parameter matrix and distortion coefficients are corrected based on the refractive index of transformer oil, and a distortion model is used to perform reverse mapping on each frame of the original image captured by the camera to correct geometric distortion. The distortion model formula is: in, These are the pixel coordinates before distortion. These are the pixel coordinates after distortion. , The radial distortion coefficient is... The tangential distortion coefficient; Color correction and texture enhancement are performed on the geometrically distorted image. The distorted image is converted from the RGB color space to the Lab color space. The mean values ​​of the a and b channels of the entire image are calculated and linear offset compensation is performed on the neutral point to remove the color cast caused by oil aging. The CLAHE algorithm is applied to the L channel to enhance the contrast between the component surface and the background. The Gaussian-Laplacian pyramid fusion algorithm is used to sharpen image edges.

4. The transformer internal inspection robot navigation method based on drawings and internal anchor point constraints according to claim 1, characterized in that, The operation described in S1, which involves real-time visual recognition of the processed image to determine candidate components, specifically includes: Each frame of the image is pixel-scaled, and the scaled image is input into a deep learning model. The deep learning model uses the YOLOv8s object detection network, which is configured with a confidence threshold of 0.5 and a non-maximum suppression threshold of 0.

45. The deep learning model outputs bounding boxes and class probabilities of suspected structural components in the image; candidate components are confirmed based on the bounding boxes, and the region image is cropped. The cropped region image is preprocessed, and real-time feature vectors of candidate components are extracted using a deep learning neural network. .

5. The transformer internal inspection robot navigation method based on drawings and internal anchor point constraints according to claim 1, characterized in that, The calculation of the robot's real-time pose relative to the navigation anchor points, as described in S2 based on feature matching identification, is specifically performed as follows: The robot's real-time 3D distance and deflection angle relative to the navigation anchor point are calculated. A local coordinate system is established with the anchor point center as the origin. The robot's pose parameters are then calculated. satisfy: in, These are the pixel coordinates of the image; This is the camera intrinsic parameter matrix; This is the translation vector, corresponding to the three-dimensional distance deviation between the robot and the navigation anchor point; These are the robot's coordinates in the global right-handed coordinate system. It is a 3×3 rotation matrix, determined by the attitude angles. It is derived from the Euler angle rotation formula, that is: ； in, Roll angle: the rotation angle of the robot around the X-axis of the global coordinate system; Pitch angle: the rotation angle of the robot about the Y-axis of the global coordinate system; Yaw angle: the rotation angle of the robot around the Z-axis of the global coordinate system.

6. The transformer internal inspection robot navigation method based on drawings and internal anchor point constraints according to claim 1, characterized in that, S3 describes acquiring the robot's current dead reckoning data and performing global pose correction based on the coordinates of high-confidence valid navigation anchor points and their calculated pose parameters. The specific operation is as follows: S301. Obtain dead reckoning data output by the robot's inertial navigation unit, wherein the dead reckoning data includes predicted pose. and its covariance matrix ; S302. The absolute coordinates corresponding to the high-confidence valid navigation anchor points obtained by the determination are used as observation values. Based on the observation values ​​and the pose parameters calculated by the anchor points, the extended Kalman filter is used to update the state, so as to realize the accurate reset of the robot dead reckoning data in the global coordinate system.

7. The transformer internal inspection robot navigation method based on drawings and internal anchor point constraints according to claim 6, characterized in that, The specific method for state updating using extended Kalman filtering as described in S302 is as follows: Define the system state vector ,in For the robot's position coordinates, For attitude angle, Linear velocity; Establish prediction equations ,in, Here is the state transition matrix. To control the input matrix, X k-1 It is the global pose state vector from the previous moment. This is a motor speed control variable. For process noise, obey distributed, Here is the process noise covariance matrix; k represents the current time. Establish observation equations ,in, These are the absolute coordinates of the anchor point observations. For the observation matrix, To observe noise, obey distributed, To observe the noise covariance matrix; Based on the predicted pose and predict covariance matrix Calculate Kalman gain ; Using anchor point observations The state is updated to obtain the corrected optimal estimated state, as shown in the following formula: in, To predict pose; This is the global pose state vector corrected at the previous moment; The covariance updated at the previous moment; This is the corrected global pose state vector at the current moment; This is the covariance updated at the current moment; It is an identity matrix.

8. The transformer internal inspection robot navigation method based on drawings and internal anchor point constraints according to claim 1, characterized in that, S4 describes path planning based on the corrected global pose and constraints from the structural design drawings. The specific operation is as follows: S401. Based on the target location inspection command, record the prior environmental information in a 3D map, and mark the prohibited areas, recommended cooling oil passages, and the locations of all key anchor points; using the latest output global pose as the starting point and the target location coordinates of the multimodal navigation anchor point knowledge base as the target point, plan the optimal path based on the discretized network map and construct the cost function: in, The actual cost from the starting point to node n. The heuristic cost from node n to the target point; S402. Compare the target pose output by the path planning with the actual global pose of the robot, calculate the pose deviation, and calculate the desired body speed control command and thruster control command based on the pose deviation to perform closed-loop motion control of the robot.

9. The transformer internal inspection robot navigation method based on drawings and internal anchor point constraints according to claim 8, characterized in that, The method described in S402 for calculating the attitude deviation and calculating the desired body speed control command and thruster adjustment command based on the attitude deviation is as follows: The formula for calculating pose deviation is: Wherein, ΔX represents the robot pose deviation vector; Δx represents the robot's position deviation in the X-axis direction of the global coordinate system; Δy represents the robot's position deviation in the Y-axis direction of the global coordinate system; Δz represents the robot's position deviation in the Z-axis direction of the global coordinate system; Δα represents the robot's roll angle deviation; Δβ represents the robot's pitch angle deviation; and Δγ represents the robot's yaw angle deviation. The attitude deviation is input into the cascade PID controller: the outer loop position PID calculates the desired body speed command based on the position error; the inner loop position PID calculates the thrust or speed command of each of the four thrusters based on the desired speed, the current speed, and the attitude error.

Images