Vision-imu-based power line ranging method
By combining the SuperGlue deep learning matching network of monocular vision and IMU devices, the problems of high equipment cost, complex operation and insufficient accuracy in power line ranging technology are solved, realizing high-precision and low-cost power line ranging, which is suitable for power inspection in complex environments.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- QING DAO ZHONG QI TE ZHONG QI CHE YOU XIAN GONG SI
- Filing Date
- 2026-03-26
- Publication Date
- 2026-06-05
AI Technical Summary
Existing power line ranging technologies suffer from high equipment costs, complex operation, and insufficient accuracy, especially in complex urban environments where it is difficult to accurately identify power lines.
Combining monocular vision ranging technology with IMU equipment, and employing the SuperGlue deep learning matching network, feature points of wire images are acquired through a monocular camera and an inertial measurement unit. High-precision ranging is then achieved by combining the principles of pinhole imaging and triangle similarity.
It achieves high-precision, low-cost power line ranging, supports multiple mobile platforms, is suitable for power line inspection in complex environments, avoids equipment collisions, and improves the accuracy and stability of ranging.
Smart Images
Figure CN122149399A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of computer vision and distance measurement technology, specifically a power line distance measurement method based on Vision-IMU, which is particularly suitable for non-contact distance measurement of power lines using a monocular camera mounted on a mobile platform such as a bucket truck during power line inspection. Background Technology
[0002] With the acceleration of urbanization, the layout of power lines in urban infrastructure has become increasingly complex, with power, communication, and other infrastructure wires intertwined to form a vast network. In this environment, the demand for power line inspection and maintenance using boom trucks and other intelligent equipment is increasing. If power lines are not effectively identified and avoided during equipment operation, collisions, damage, or even safety accidents may occur. Therefore, reliable power line ranging technology is particularly important.
[0003] Traditional power line distance measurement methods often employ laser ranging or manual measurement, which suffer from drawbacks such as high equipment costs and complex operation. Vision-based power line sensing technologies include monocular vision and binocular vision. Monocular vision ranging technology has attracted attention due to its simple equipment and low cost. Although monocular vision systems have the advantages of simple information acquisition and fast calculation speed, they often struggle to accurately identify power lines in complex urban environments. This invention combines vision technology and IMU equipment to propose a high-precision, low-cost monocular power line ranging solution. Summary of the Invention
[0004] This invention addresses the problems of high equipment cost, complex operation, and insufficient accuracy in existing power line distance measurement technologies by providing a Vision-IMU-based power line distance measurement method. This method innovatively combines monocular vision ranging technology, an IMU device, and the SuperGlue deep learning matching network, achieving high-precision and low-cost power line distance measurement.
[0005] This invention is a wire ranging method based on Vision-IMU, comprising a mobile platform, a monocular camera and an inertial measurement unit mounted on the mobile platform, and the following steps: S1: The monocular camera and inertial measurement unit are moved from the first position to the second position via a mobile platform; S2: The monocular camera acquires wire patterns at the first and second positions respectively, and uses an inertial measurement unit (IMU) to obtain the precise displacement Δm of the camera from the first position to the second position; S3: A monocular camera acquires a sequence of images of the power lines, including a first position image I1 and a second position image I2. The SuperGlue deep learning matching network is used to process the image pair (I1, I2). The SuperGlue neural network model consists of two parts: an attention neural network and an optimal matching layer. The feature point sets P1 and P2 of the power lines in the two images are extracted and the matching relationship between the feature points M={(p1,p2)|p1∈P1,p2∈P2} is established. The confidence score matrix S of the matching point pair is output. S4: Calculate the wire imaging diameter based on the matching results, and obtain the change in wire imaging diameter H2-H1 between the two images; S5: The formula for calculating the distance between wires is obtained by using the principles of pinhole imaging and triangle similarity.
[0006] Based on the above technical solution, the wire ranging method based on Vision-IMU of the present invention can be further improved as follows: Furthermore, the SuperGlue deep learning matching network in S3 adopts the following optimized configuration: The feature extraction backbone network is an improved ResNet-34 architecture; The graph neural network uses a 6-layer message-passing structure; The confidence threshold for keypoint detection is set to 0.7; The matching score matrix S is subjected to bidirectional softmax normalization.
[0007] Furthermore, the application of the SuperGlue deep learning matching network in S3 includes the following steps: The S31 attention map neural network part encodes the feature points and descriptors of the two images at the first and second positions into a vector through a feature point encoder. S32: After multiple iterations of self-attention and cross-attention, the two images obtain their respective matching descriptors; S33: The optimal matching layer obtains feature point descriptors for the two images through an attention GNN. and And perform an inner product of the descriptors to calculate their score matrix:
[0008] S34: Add a new row and column to the score matrix, then use the Sinkhorn algorithm to solve the optimal transfer problem to obtain the allocation matrix. The allocation matrix P can be obtained by calculating the score matrix and maximizing the total score.
[0009] S35: Unmatched feature points are thrown into the dustbin channel to filter out mismatched point pairs. Finally, maximizing the total score is transformed into an optimal transmission problem. This method mainly uses the Sinkhorn algorithm to handle the optimal transmission problem.
[0010] Furthermore, S4 specifically includes: S41: Filter reliable matching pairs with a confidence score > 0.85; S42: Calculate the imaging diameters H1 and H2 of the wire in the image based on the matching point pairs; S43: Calculate the change in imaging diameter ΔH = H2 - H1 and verify its rationality.
[0011] Furthermore, the calculation formula in S5 is as follows:
[0012] After simplification, we get: ; Combining the above two equations, we can obtain the distance between the power line and the camera: ; For the camera's focal length, , The image diameter of the wire in two images taken from two different locations. , The distances between the camera and the actual power line at different locations are represented by φ, where φ is the actual diameter of the power line. This is the distance between two different locations, i.e., the displacement measured by the IMU.
[0013] Through the above design scheme, the present invention can bring the following beneficial effects: 1. This invention innovatively applies the SuperGlue deep learning matching network to the field of wire ranging, which significantly improves the accuracy and stability of feature matching and solves the matching problem of traditional visual ranging methods in complex environments.
[0014] 2. This invention establishes an accurate ranging model by combining IMU motion data and the pinhole imaging principle, achieving high-precision ranging without prior knowledge of the actual diameter of the wire.
[0015] 3. The system architecture of this invention is simple, efficient, and low-cost, providing an economical and reliable solution for power inspection.
[0016] 4. This invention supports various mobile platforms such as bucket trucks and drones, which can meet the power inspection needs in different scenarios. It mainly infers the two distances between the camera and the actual power line by the longitudinal movement of the platform, that is, the distance between the camera and the actual power line at different positions, which has broad application prospects.
[0017] 5. Wire distance measurement measures the distance between the wire and the camera. A point is selected on the wire to obtain the distance between the point and the camera, thereby determining the distance between the moving platform and the wire, avoiding contact with the wire and maintaining a safe distance.
[0018] 6. It is not necessary to know the "actual diameter of the wire" in advance. In the final result of the formula derivation, the actual diameter of the wire is replaced by H1, H2, m1, m2, and f. Attached Figure Description
[0019] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.
[0020] Figure 1 This is a schematic diagram of the pinhole imaging model used in this invention; Figure 2 The diagram shows the principle of monocular ranging, where (a) is a long-distance power line image, (b) is a short-distance power line image, (c) is a long-distance power line imaging principle diagram, and (d) is a short-distance power line imaging principle diagram. Figure 3 This is a diagram of the SuperGlue architecture. Figure 4 This is a schematic diagram of the imaging diameter of the wire. Detailed Implementation
[0021] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0022] like Figure 1 As shown, this invention provides a wire ranging method based on Vision-IMU, comprising a mobile platform, a monocular camera and an inertial measurement unit mounted on the mobile platform, wherein the method includes the following steps: S1: The monocular camera and inertial measurement unit are moved from the first position to the second position via a mobile platform; S2: The monocular camera captures wire images at the first and second positions respectively; The precise displacement Δm of the camera from the first position to the second position is obtained using an inertial measurement unit (IMU); S3: A monocular camera acquires a sequence of images of the power lines, including a first position image I1 and a second position image I2. The SuperGlue deep learning matching network is used to process the image pair (I1, I2). The SuperGlue neural network model consists of two parts: an attention neural network and an optimal matching layer. The feature point sets P1 and P2 of the power lines in the two images are extracted and the matching relationship between the feature points M={(p1,p2)|p1∈P1,p2∈P2} is established. The confidence score matrix S of the matching point pair is output. S4: Calculate the wire imaging diameter based on the matching results, and obtain the change in wire imaging diameter H2-H1 between the two images; S5: The formula for calculating the distance between wires is obtained by using the principles of pinhole imaging and triangle similarity.
[0023] Optionally, in the above technical solution, the SuperGlue deep learning matching network in S3 adopts the following optimized configuration: The feature extraction backbone network is an improved ResNet-34 architecture; The graph neural network uses a 6-layer message-passing structure; The confidence threshold for keypoint detection is set to 0.7; The matching score matrix S is subjected to bidirectional softmax normalization.
[0024] Optionally, in the above technical solution, the application of the SuperGlue deep learning matching network in S3 includes the following steps: The S31 attention map neural network part encodes the feature points and descriptors of the two images at the first and second positions into a vector through a feature point encoder. S32: After multiple iterations of self-attention and cross-attention, the two images obtain their respective matching descriptors; S33: The optimal matching layer obtains feature point descriptors for the two images through an attention GNN. and And perform an inner product of the descriptors to calculate their score matrix:
[0025] S34: Add a new row and column to the score matrix, then use the Sinkhorn algorithm to solve the optimal transfer problem to obtain the allocation matrix. The allocation matrix P can be obtained by calculating the score matrix and maximizing the total score.
[0026] S35: Unmatched feature points are thrown into the dustbin channel to filter out mismatched point pairs. Finally, maximizing the total score is transformed into an optimal transmission problem. This method mainly uses the Sinkhorn algorithm to handle the optimal transmission problem.
[0027] Optionally, in the above technical solution, S4 specifically includes: S41: Filter reliable matching pairs with a confidence score > 0.85; S42: Calculate the imaging diameters H1 and H2 of the wire in the image based on the matching point pairs; S43: Calculate the change in imaging diameter ΔH = H2 - H1 and verify its rationality.
[0028] Optionally, in the above technical solution, the calculation formula in S5 is:
[0029] After simplification, we get: ; Combining the above two equations, we can obtain the distance between the power line and the camera: ; For the camera's focal length, , The image diameter of the wire in two images taken from two different locations. , The distances between the camera and the actual power line at different locations are represented by φ, where φ is the actual diameter of the power line. This is the distance between two different locations, i.e., the displacement measured by the IMU.
[0030] Image feature points can be extracted using the SuperPoint network.
[0031] The monocular camera and the IMU are both mounted on the same rigid component of the mobile platform, which ensures that the spatial relationship between the camera and the IMU remains unchanged during the vehicle's movement.
[0032] In this process, multiple first feature points (P1) are extracted from the first location image; and multiple first feature points (P2) are extracted from the second location image.
[0033] Among them, a point is selected from the edge of a wire (e.g.) Figure 4Draw a straight line parallel to the vertical axis of the image coordinate system along that point. The distance between the two points where the line intersects the edge of the wire is the imaging diameter H of the wire.
[0034] Feature point extraction involves identifying points with significant features (preferably points at edges) in the image through key point detection.
[0035] Because the power line is not straight (it cannot be parallel to the ground plane), even if the moving platform is moved horizontally, when the monocular camera takes another picture (from the second position), the position of the power line in the picture will still be different from the first one. Therefore, it is necessary to match the characteristics of the power line in the two pictures.
[0036] Where M = {(p1, p2) | p1 ∈ P1, p2 ∈ P2} is a mathematical set representation used to describe the "set of potential matching pairs" of feature points in two images, with the following specific meanings: Set M: Represents the set of all possible feature point matching pairs, where each element (p1, p2) is an "ordered pair" representing a matching candidate consisting of a feature point p1 in Figure 1 and a feature point p2 in Figure 2.
[0037] Symbol meaning: {}: represents a "set", and its contents are the elements of the set; (p1, p2): Represents a matching pair, where p1 is a feature point in the first image and p2 is a feature point in the second image; |: Read as "satisfy", and what follows is the "constraint condition" of the element; p1 ∈ P1: ∈ means "belongs to", that is, p1 is an element in the feature point set P1 of the first image; p2 ∈ P2: That is, p2 is an element in the feature point set P2 of the second image.
[0038] Suppose the first image has feature points P1 = {a, b, c} (e.g., 3 feature points), and the second image has feature points P2 = {x, y} (e.g., 2 feature points). Then the set M is all possible combinations of matching pairs: M = {(a,x),(a,y), (b,x), (b,y), (c,x), (c,y)}.
[0039] In SuperGlue's matching process, this set M represents all the initial potential matching possibilities, and the core task of the algorithm is to select the real matching feature point pairs from M (which can be done through GNN, attention mechanism, Hungarian algorithm to find the optimal matching result from all candidates).
[0040] in, The probability that the i-th feature point in Figure A (first position) matches the j-th feature point in Figure B (second position) is calculated using vector dot product or a fully connected network.
[0041] For each feature point p1 in P1, the feature point p2 with the most similar descriptor is found in P2 (by measuring Euclidean distance, cosine similarity, etc.), and only matching pairs with similarity higher than the threshold are retained.
Claims
1. A wire ranging method based on Vision-IMU, comprising a mobile platform, and a monocular camera and an inertial measurement unit mounted on the mobile platform, characterized in that, Includes the following steps: S1: The monocular camera and inertial measurement unit are moved from the first position to the second position via a mobile platform; S2: The monocular camera captures wire images at the first and second positions respectively; The precise displacement Δm of the camera from the first position to the second position is obtained using an inertial measurement unit (IMU); S3: A monocular camera acquires a sequence of images of the power lines, including a first position image I1 and a second position image I2. The SuperGlue deep learning matching network is used to process the image pair (I1, I2). The SuperGlue neural network model consists of two parts: an attention neural network and an optimal matching layer. The feature point sets P1 and P2 of the power lines in the two images are extracted and the matching relationship between the feature points M={(p1,p2)|p1∈P1,p2∈P2} is established. The confidence score matrix S of the matching point pair is output. S4: Calculate the wire imaging diameter based on the matching results, and obtain the change in wire imaging diameter H2-H1 between the two images; S5: The formula for calculating the distance between wires is obtained by using the principles of pinhole imaging and triangle similarity.
2. The wire ranging method based on Vision-IMU according to claim 1, characterized in that: The SuperGlue deep learning matching network in S3 adopts the following optimized configuration: The feature extraction backbone network is an improved ResNet-34 architecture; The graph neural network uses a 6-layer message-passing structure; The confidence threshold for keypoint detection is set to 0.7; The matching score matrix S is subjected to bidirectional softmax normalization.
3. The wire ranging method based on Vision-IMU according to claim 1, characterized in that: The application of the SuperGlue deep learning matching network in S3 includes the following steps: S31: The attention map neural network part encodes the feature points and descriptors of the two images at the first and second positions into a vector through the feature point encoder; S32: After multiple iterations of self-attention and cross-attention, the two images obtain their respective matching descriptors; S33: The optimal matching layer obtains feature point descriptors for the two images through an attention GNN. and And perform an inner product of the descriptors to calculate their score matrix: ； S34: Add a new row and column to the score matrix, then use the Sinkhorn algorithm to solve the optimal transfer problem to obtain the allocation matrix. The allocation matrix P can be obtained by calculating the score matrix and maximizing the total score. ； S35: Unmatched feature points are thrown into the dustbin channel to filter out mismatched point pairs. Finally, maximizing the total score is transformed into an optimal transmission problem. This method mainly uses the Sinkhorn algorithm to handle the optimal transmission problem.
4. The wire ranging method based on Vision-IMU according to claim 1, characterized in that: S4 specifically includes: S41: Filter reliable matching pairs with a confidence score > 0.85; S42: Calculate the imaging diameters H1 and H2 of the wire in the image based on the matching point pairs; S43: Calculate the change in imaging diameter ΔH = H2 - H1 and verify its rationality.
5. The wire ranging method based on Vision-IMU according to claim 1, characterized in that: The calculation formula in S5 is as follows: ； After simplification, we get: ； Combining the above two equations, we can obtain the distance between the power line and the camera: ； For the camera's focal length, , The image diameter of the wire in two images taken from two different locations. , The distances between the camera and the actual power line at different locations are represented by φ, where φ is the actual diameter of the power line. This is the distance between two different locations, i.e., the displacement measured by the IMU.