A power transmission tower pose estimation and attitude angle calculation method and system based on image processing

Abstract

The present application relates to the technical field of computer vision and image processing, and particularly relates to a power transmission tower pose estimation and attitude angle calculation method and system based on image processing, which comprises the following steps: acquiring a depth image and a color image, extracting two-dimensional image pixel coordinates, and reconstructing three-dimensional space coordinates; performing spatial structure consistency screening on the three-dimensional space coordinates to obtain a high-confidence inlier set; updating a three-dimensional space reference point; constructing a joint objective function and performing nonlinear iterative optimization to decouple the output target camera relative pose matrix; extracting a normal vector and a reference radial vector, and calculating a full-degree-of-freedom relative rotation angle instruction; the present application aims to provide a power transmission tower pose estimation and attitude angle calculation method and system based on image processing, which can improve the technical problems of pose iterative divergence, poor robustness to structural noise, and missing degrees of freedom caused by "rigid body assumption failure" of the existing pure visual pose estimation algorithm in the heavy-load micro-deformation scene.

Description

Technical Field

[0001] This invention relates to the field of computer vision and image processing technology, and in particular to a method and system for estimating the pose and calculating the attitude angle of power transmission towers based on image processing. Background Technology

[0002] In the automated assembly and hoisting of large-scale engineering equipment (such as the tower sections of ultra-high voltage transmission towers), accurately acquiring the relative pose and rotation angle of the components to be docked is the core data input support for achieving intelligent hole alignment and automated docking. Existing visual measurement solutions typically rely on depth cameras to acquire 3D point cloud data and use traditional PnP (Perspective-n-Point) algorithms combined with purely statistical outlier detection algorithms such as Z-score to remove noise points. Then, nonlinear optimization methods are used to solve for the camera's extrinsic parameter matrix.

[0003] However, when these conventional image processing algorithms are directly deployed in outdoor heavy-duty hoisting scenarios, the following problems arise: First, in complex lighting environments such as strong outdoor light and large-area obstruction, the mechanical rigid structure of the target object (such as flange hole array) often produces "structural mismatch" (for example, pseudo-feature points formed by reflection happen to be symmetrically distributed). The statistical elimination algorithm based on Euclidean distance alone cannot identify these pseudo-features that conform to the mechanical distribution law but are actually noise.

[0004] Secondly, existing mainstream PnP pose estimation algorithms are typically based on the "rigid body" assumption. In practical engineering, the tower end may undergo millimeter-level elastic or plastic micro-deformation under stress conditions, which can manifest as the hole array evolving from an ideal circular distribution to a non-ideal shape. This type of morphological change may produce similar behavior to camera perspective tilt in two-dimensional projection features, causing pose calculations relying solely on the rigid body assumption to suffer from model mismatch in some conditions, thus affecting the stability and convergence quality of the nonlinear optimization process. Furthermore, under strong outdoor light conditions, metal surface reflection and specular overflow may introduce spurious feature points, making it difficult for outlier removal strategies relying solely on statistical distance to simultaneously achieve robustness and effectiveness in certain scenarios.

[0005] Finally, when calculating relative rotation angles, some existing vision algorithms primarily use the angle between the normal vectors of two fitted planes to characterize attitude differences. This geometric calculation logic can usually obtain tilt information such as pitch and roll angles, but it is difficult to directly provide a stable solution with a directional sign for the phase deflection angle (yaw) around the normal axis. In scenarios where the pose results need to be used for automatic hole alignment and docking control, the lack of this attitude information may affect the downstream control strategy's determination and execution accuracy of the relative rotation direction. Summary of the Invention

[0006] To address the technical problems of existing pure vision pose estimation algorithms in heavy-load, micro-deformation scenarios, such as pose iteration divergence, poor robustness against structural noise, and lack of solution degrees of freedom caused by the "failure of rigid body assumption," the present invention aims to provide a method and system for power transmission tower pose estimation and attitude angle calculation based on image processing.

[0007] To achieve the above objectives, the core technical solution adopted by this invention is as follows: This invention provides a method for estimating the pose and calculating the attitude angle of a power transmission tower based on image processing, comprising: Step 1, acquiring depth and color images of an optical calibration target surface containing periodic array features, extracting the two-dimensional image pixel coordinates of the centroids of sub-pixel-level edge features on the target surface, and reconstructing the corresponding initial three-dimensional spatial coordinates; Step 2, combining preset topological geometric prior constraints and physical deformation tolerance thresholds, performing spatial structure consistency screening on the initial three-dimensional spatial coordinates to obtain a high-confidence set of interior points; Step 3, calculating the principal direction distribution of the spatial radial residual based on the high-confidence set of interior points, and establishing a parameterized distortion model to update the three-dimensional spatial reference points when the preset deformation triggering conditions are met; Step 4, constructing a joint objective function including the three-dimensional spatial reference points, camera pose matrix parameters, and distortion constraint boundaries, performing nonlinear iterative optimization to decouple and output the relative pose matrix of the target camera; Step 5, based on the optimized and converged relative pose matrix, extracting the normal vector of the spatial fitting plane and the reference radial vector of the reference feature points, and calculating the full-degree-of-freedom relative rotation command including the tilt angle and the phase deflection angle with direction sign.

[0008] Further, the specific screening process in step two includes: calculating the standardized score of the error set composed of the reconstructed initial three-dimensional spatial coordinates and performing statistical coarse elimination to obtain a coarse screening set; performing flange plane fitting and circular array projection fitting on the coarse screening set, and performing structural consistency screening based on the calculated point-to-plane distance residual, circular fitting residual, and polar angle spacing residual to obtain a structural screening set; calculating the spatial circumcircle radius and spatial chord length of locally adjacent feature points in the structural screening set as topological geometric prior constraints, constructing a topological penalty factor with dead zones based on the physical deformation tolerance threshold, reducing the weight or eliminating feature points that deviate from the tolerance dead zones, and outputting a high-confidence inlier set.

[0009] Furthermore, the formula for calculating the topology penalty factor with dead zones is as follows: in, For the first The topological penalty factor weights of each feature point and The penalty coefficient is... The radius of the spatial circumcircle is calculated in real time. This is a reference value for the theoretical flange circumscribed circle radius. For real-time calculation of spatial chord length, This is a reference value for the chord length of theoretically adjacent feature points. This is the physical deformation tolerance threshold.

[0010] Furthermore, the process of extracting the principal direction distribution and establishing a parameterized distortion model in step three is configured to be executed independently by default using only pure visual image features. The specific steps include: calculating the difference between the measured distance and the theoretical radius from each inlier in the high-confidence inlier set to the fitted circle center to obtain the radial geometric residual; performing principal direction analysis on the radial geometric residual distribution characteristics along the circumference; if the residual extreme value direction shows a bimodal or uniaxial principal direction trend, the deformation triggering condition is met, and a systematic physical deformation is determined to have occurred; replacing the preset rigid circle model with a parameterized distortion model, making the parameterized distortion model an elliptical model, and aligning the major axis direction of the elliptical model with the radial geometric residual extreme value direction, and estimating the semi-major and semi-minor axis parameters of the ellipse based on the radial residual extreme value magnitude.

[0011] Furthermore, the formula for the joint objective function in step four is: in, For total energy error, This is the reprojection error term. For planar structural residuals, For parametric ellipse constraint terms, Here are the parameters of the major and minor semi-axes of the ellipse. These are the plane weighting coefficients. It is an adaptive elastic relaxation factor. For two-dimensional image pixel coordinates, This is the intrinsic parameter matrix of the depth camera. The camera pose matrix parameters to be optimized are... The three-dimensional spatial reference point is dynamically generated.

[0012] Furthermore, the distortion constraint boundary in step four manifests as a deformation upper limit constraint barrier. The nonlinear iterative optimization includes: establishing the initial pose based on local curvature or weak perspective approximation to break the ambiguity of projective geometric coupling between perspective tilt and physical deformation; and when using the Levenberg-Marquardt algorithm to solve the joint objective function, forcibly restricting the parameters of the ellipse's major and minor semi-axis to satisfy the following: and ;in, This represents the semi-major axis parameter of the ellipse model in the parametric distortion model. This represents the minor semi-axis parameter of the ellipse model. This represents the theoretical reference value for the circumscribed circle radius of the flange. This represents the dimensionless physical deformation relative tolerance threshold. This represents the upper limit of the allowable offset of the ellipse's semi-axis relative to the reference value of the radius of the circumscribed circle of the theoretical flange; when the calculated parameter iteration gradient causes the ellipse parameters to exceed the deformation upper limit constraint barrier, the iteration gradient is truncated and forcibly pulled back to the limit boundary, thereby decoupling the perspective pose error from the physical micro-deformation.

[0013] Further, step five, calculating the relative rotation command with full degrees of freedom, includes: unifying the spatial features of the reference target surface and the target surface to be docked to the same global coordinate system using the relative pose matrix; performing dot product and cross product operations on the normal vectors of the two target surface spatial fitting planes after unifying the coordinate system to calculate the tilt angle that makes the two planes parallel, where the tilt angle includes pitch angle and roll angle; selecting a preset reference feature as the phase reference point, constructing a reference radial vector pointing from the geometric center to the phase reference point, and rotating and compensating the reference radial vector of the target surface to be docked to the reference plane of the reference target surface according to the tilt angle; calculating the absolute value of the angle between the two coplanar radial vectors, and extracting the positive and negative signs using the triple scalar product operation result of the two coplanar radial vectors and the normal vector of the reference target surface to obtain the phase deflection angle with a clear rotation direction.

[0014] Preferably, an image processing-based system for estimating the pose and calculating the attitude angle of a power transmission tower is also provided, comprising: an image data processing module for acquiring depth and color images of an optical calibration target surface containing periodic array features, extracting the two-dimensional image pixel coordinates of the centroids of sub-pixel-level edge features on the target surface, and reconstructing the corresponding initial three-dimensional spatial coordinates; a topological feature filtering module for performing spatial structure consistency filtering on the initial three-dimensional spatial coordinates by combining preset topological geometric prior constraints and physical deformation tolerance thresholds, and obtaining a set of high-confidence interior points; and a parametric distortion modeling module for calculating the principal direction distribution of the spatial radial residual based on the set of high-confidence interior points, and establishing a parametric distortion modeling system when preset deformation triggering conditions are met. The system includes a digitized distortion model to update the 3D spatial reference point; a joint decoupling optimization module to construct a joint objective function containing the 3D spatial reference point, camera pose matrix parameters, and distortion constraint boundaries, and to perform nonlinear iterative optimization to decouple and output the relative pose matrix of the target camera; and a full-degree-of-freedom solution module to extract the normal vector of the spatial fitting plane and the reference radial vector of the reference feature point based on the optimized and converged relative pose matrix, and to calculate the full-degree-of-freedom relative rotation command including the tilt angle and the phase deflection angle with direction sign. The image processing-based transmission tower pose estimation and attitude angle calculation system is used to execute any of the above-mentioned image processing-based transmission tower pose estimation and attitude angle calculation methods.

[0015] Furthermore, the topological feature screening module is used to: perform statistical coarse screening by calculating standardized scores on the reconstructed initial 3D spatial coordinates; then perform structural consistency screening using the distance residuals from points to the fitting plane and the polar angle spacing residuals; and calculate the spatial circumcircle radius and spatial chord length of local features as topological geometric prior constraints, construct a topological penalty factor matrix with tolerance dead zones based on the physical deformation tolerance threshold, thereby outputting a set of high-confidence interior points.

[0016] Furthermore, the full-degree-of-freedom solution module is further configured with a sign determination subunit, which is used to obtain the reference side coplanar radial vector and the target side coplanar radial vector after the reference radial vector of the target surface to be docked is rotated and compensated to the reference plane of the reference target surface; calculate the absolute value of the angle between the reference side coplanar radial vector and the target side coplanar radial vector, and calculate the dot product between the cross product of the reference side coplanar radial vector and the target side coplanar radial vector and the reference target surface normal vector to obtain the sign determination scalar; and determine the clockwise or counterclockwise rotation direction of the phase deflection angle according to the sign of the sign determination scalar.

[0017] Beneficial effects: 1. Reduced blind spots in pure statistical noise reduction and improved robustness of physical tolerance-level initial screening: This invention introduces "geometric topological invariants (circumcircle, chord length) with physical deformation tolerance dead zone", which can ensure that error points that conform to the symmetrical structure are eliminated in the bottom image pixel screening stage, and avoid mistakenly killing the real stress deformation points, thereby improving the noise resistance robustness under complex outdoor lighting conditions.

[0018] 2. Achieved pure visual deformation decoupling and boundary control: To a certain extent, this breaks the "absolutely rigid body" assumption of visual algorithms. Without relying on external hardware strain gauges, the "principal stress deformation axis" can be dynamically extracted using only the principal direction distribution of the radial residuals at points within the image. By constructing dynamically parameterized 3D reference points in the objective function and using the deformation upper limit constraint for gradient truncation, precise mathematical decoupling between the camera pose matrix and the target's physical micro-deformation is achieved.

[0019] 3. It fills in the gaps in spatial degrees of freedom and improves the output of full-attitude closed-loop data: It overcomes the defect of yaw information loss caused by the calculation of the angle between normal vectors in the existing technology. It separates the tilt angle (Pitch, Roll) and the phase deflection angle (Yaw) based on the feature hole reference point, and outputs a full-degree-of-freedom relative rotation command matrix, which provides an important spatial input source for the precise hole insertion of intelligent actuators. Attached Figure Description

[0020] Figure 1 This is a flowchart illustrating a method for estimating the pose and calculating the attitude angle of a power transmission tower based on image processing, provided in an embodiment of the present invention. Figure 2 This is a logic diagram of the hierarchical initial screening principle in an embodiment of the present invention, which includes a topological feature penalty mechanism with dead zones. Figure 3 This is a data flow diagram of the construction of the pure visual parametric distortion model and the dynamic update of the reprojection model in an embodiment of the present invention; Figure 4 This is the algorithm architecture and variable interaction diagram of the joint decoupling nonlinear optimization in this embodiment of the invention; Figure 5 This is a schematic diagram of the geometric relationship in the full-degree-of-freedom relative rotation space solution (including normal vector alignment and reference hole phase alignment) in an embodiment of the present invention; Figure 6 This is a structural block diagram of a power transmission tower pose estimation and attitude angle calculation system based on image processing, provided in an embodiment of the present invention. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0022] Terminology Definition To facilitate understanding and ensure consistency of terminology, the following terms have the following meanings in this application; where there is no conflict, equivalent expressions that can be understood by those skilled in the art may also be used: (1) Topological prior constraints: refers to the set of rules that use the geometric topological invariants of the target surface periodic array structure to constrain the structural consistency of the three-dimensional reconstruction feature points; in some embodiments, the topological invariants include, but are not limited to, the spatial circumcircle radius, spatial chord length and statistical distribution characteristics of local adjacent feature points.

[0023] (2) Physical deformation tolerance threshold This refers to the threshold value at which geometric deviations are permissible when the target surface or the part to be docked undergoes minute deformations within the allowable range for engineering applications; in some embodiments, the... The parameters used to construct the tolerance dead zone and serve as the source of the upper limit boundary constraint for distortion can be predetermined based on manufacturing / assembly tolerances, the allowable deformation range of materials (e.g., safety margins related to yield), and / or field calibration results.

[0024] (3) Tolerance Dead Zone: refers to the interval in which no penalty or only a weak penalty is applied to the feature point when the absolute value of the geometric deviation does not exceed the allowable interval corresponding to ε; when the deviation exceeds the interval, the feature point is downweighted or eliminated.

[0025] (4) Deformation triggering condition: refers to the judgment criteria used to determine whether "systematic physical deformation has occurred"; in some embodiments, the judgment can be based on the distribution of radial geometric residuals along the circumference showing a bimodal or single-axis main direction trend; for example, when performing main direction analysis / principal component analysis on radial geometric residuals, if the first principal component is significantly larger than the second principal component, it can be determined that there is a systematic deformation trend with a single dominant direction.

[0026] (5) Parametric distortion model: refers to a geometric model used to replace the rigid circle model and characterize the evolution of the target face array from an ideal circle to a non-ideal shape; in some embodiments, the parametric distortion model includes an elliptical model and is expressed by the semi-major axis parameter. With the short half-axis parameter Describe the degree of distortion.

[0027] (6) Distortion constraint boundary: refers to the upper limit imposed on the range of parameter variation of the parameterized distortion model; in some embodiments, the boundary is used to limit , Relative to the reference radius (or reference scale) in the range of Variations are allowed within a given range.

[0028] (7) Full-degree-of-freedom relative rotation command: refers to the set of angle outputs that characterize the relative attitude change between two target surfaces / two components; in some embodiments, the set includes at least pitch and roll angles for describing tilt, and phase deflection angle (yaw) with rotation direction sign.

[0029] Example 1 like Figure 1 As shown, this embodiment provides a method for estimating the pose and calculating the attitude angle of a power transmission tower based on image processing. It relies on the acquisition, conversion, topology calculation and matrix iteration of underlying image data to abstract the external heavy machinery structure as the data input source of the algorithm.

[0030] To improve the stability of pose estimation under abnormal point interference and target surface micro-deformation conditions, this embodiment unifies the geometric consistency constraints of point features, deformation model constraints, and camera pose estimation into the same optimization framework. Specifically, considering the characteristic that local topological geometric quantities of the aperture array structure (such as the radius of the spatial circumcircle and the chord length of adjacent points) should remain consistent under ideal rigid body conditions, their deviation is taken as the geometric consistency residual; when the residual does not exceed the physical deformation tolerance threshold... When the residual exceeds the defined allowable interval, no penalty is imposed on the feature point; when the residual exceeds the allowable interval, the point is weighted using an exponential decay method to obtain the topological penalty factor weight. This achieves soft removal / soft weighting of outliers. Furthermore, considering that two-dimensional observation noise can be approximated as random perturbation, pose estimation can be obtained by minimizing the reprojection error term under maximum likelihood. Simultaneously, to utilize the physical priors of the end-face structure and avoid absorbing micro-deformation errors into the camera pose, planar structure residual terms and parametric distortion model constraint terms are introduced. Weighting coefficients are used to balance the scales of different error terms, thereby constructing a joint objective function. And perform nonlinear iterative optimization. For the semi-major and semi-minor axis parameters of the ellipse in the parameterized distortion model... , Apply upper limit constraints to the boundary of deformation; when iterative updates cause parameters to exceed the constraint set, project the parameters back into the constraint set to ensure the physical feasibility and optimization stability of the solution.

[0031] The method in this embodiment mainly includes the following steps: First, a depth image and a color image of an optical calibration target surface containing periodic array features are acquired. The two-dimensional image pixel coordinates of the centroids of sub-pixel-level edge features on the target surface are extracted, and the corresponding initial three-dimensional spatial coordinates are reconstructed.

[0032] In this step, the system is configured with a dual-channel pre-calibrated depth vision sensor array at the bottom layer. After acquiring the image, the edge gradient operator and morphological opening and closing operations of computer graphics are used to locate the two-dimensional region of interest (ROI) of the target object (i.e., the highly reflective target surface formed by the array of holes in the tower flange) in the color image, and the two-dimensional image pixel coordinates of the geometric center of the holes are extracted. Construct a two-dimensional image pixel coordinate vector To address depth image noise, image pyramid downsampling and a local depth variance filter are introduced to remove flying points at abrupt depth changes. Subsequently, this is combined with the pre-calibrated intrinsic parameter matrix of the depth camera. And extrinsic parameters, combining two-dimensional pixel coordinates with depth values Mapping and backprojection reconstruct the initial 3D spatial coordinates in the camera's physical coordinate system. These three-dimensional point sets constitute the initial input source for subsequent algorithms.

[0033] Secondly, by combining the preset topological geometric prior constraints and physical deformation tolerance threshold, the initial three-dimensional spatial coordinates are screened for spatial structure consistency to obtain a set of interior points with high confidence.

[0034] like Figure 2 As shown, to overcome structural mismatches caused by complex outdoor environments, this embodiment adopts a hierarchical funnel-shaped screening architecture. The first layer calculates a standardized score (Z-score) on the error set composed of the reconstructed initial three-dimensional spatial coordinates, setting... With a threshold of 3.0, a coarse statistical scavenging process is performed to obtain a coarsely screened set. In the second layer, the RANSAC (Random Sampling Consensus) algorithm is used to fit the coarsely screened set to the plane normal vector and to the three-dimensional space circle. The Euclidean distance residual from each discrete point to the ideal plane, the radial residual of the circle fitting, and the polar angle distance residual are calculated to perform spatial structure consistency screening.

[0035] The core third-layer filtering introduces a soft penalty mechanism with a "dead zone": extracting the radius of the local spatial circumcircle formed by any adjacent feature points. With spatial chord length This serves as a topological geometric prior constraint for the algorithm, and is based on a preset physical deformation tolerance threshold. Construct the formula for the topological penalty factor with dead zones: In this mathematical expression, Indicates the first The weights of each feature point retained in subsequent matrix iterations; , This is a penalty scaling hyperparameter. The engineering mechanism of this formula is: when the absolute value of the geometric deviation reconstructed in the image space falls within the range specified by the scaling hyperparameter... Demarcated Within this "tolerance dead zone," the penalty item... The function outputs 0. The algorithm maintains a maximum value of 1 (i.e., does not penalize small physical deformations that actually occur); the penalty exponent only decays rapidly when the deviation exceeds the physical limit due to optical noise such as severe reflection, forcibly reducing its weight or eliminating it. The final algorithm stably outputs a highly pure set of high-confidence interior points. Among them, the It can be predetermined based on manufacturing / assembly tolerances, the allowable range of material deformation (e.g., safety margins related to yield), and / or field calibration results to characterize the range of permissible micro-deformation in engineering applications.

[0036] Next, as Figure 3 As shown, the main direction distribution of the spatial radial residual is calculated based on the high-confidence in-point set, and a parameterized distortion model is established to update the three-dimensional spatial reference point when the preset deformation triggering condition is met.

[0037] This embodiment strictly emphasizes completing deformation diagnosis purely based on low-level visual features without requiring external hardware sensors. Specifically, the algorithm calculates the measured Euclidean distance from each point in the aforementioned high-confidence inlier set to the center of the fitted circle, subtracts this distance from the theoretical standard CAD radius, and forms the radial geometric residual. Subsequently, the angles were compared in polar coordinates. Principal component analysis (PCA) or least squares fitting is used to extract its circumferential distribution characteristics. If the diagnostic analysis determines that the residual extreme values ​​show a highly symmetrical bimodal or uniaxial main direction trend along a certain straight line, this can be used mathematically and statistically to determine that the target structure has undergone systematic tensile deformation, thereby triggering the "deformation triggering condition".

[0038] At this point, the system dynamically replaces the rigid ideal circle model used for image reprojection calculations with a parametric distortion model, making this distortion model an ellipse model, with the major axis reference directly coinciding mathematically with the direction of the extracted radial residual extremum. Simultaneously, the maximum residual amplitude is directly used as the increment to estimate the semi-major axis of the ellipse. and short half shaft The initial floating-point parameters. As... and Dynamic updates, the underlying three-dimensional spatial reference points of the algorithm This also leads to geometric reconstruction, ensuring that the computer vision model can match the deformed physical truth.

[0039] like Figure 4 As shown, a joint objective function is then constructed, which includes the three-dimensional spatial reference point, camera pose matrix parameters, and distortion constraint boundary. Nonlinear iterative optimization is then performed to decouple and output the relative pose matrix of the target camera.

[0040] This is the core algorithmic step to ensure pose accuracy does not collapse. To resolve the ambiguity of projective geometric coupling between perspective tilt and physical elliptical deformation, and to prevent direct iteration from causing the system equations to become unobservable, the algorithm first establishes an initial pose based on local curvature features of the image or the weak perspective projection assumption. Based on this, the Levenberg-Marquardt algorithm is initiated to solve the joint objective function. .

[0041] Among them, the reprojection error term In the actual solution of this objective function, a Huber robust kernel function can be optionally introduced into the above reprojection error term. To resist abnormal flypoints, the algorithm enforces a deformation upper limit constraint barrier representing physical laws, that is, it forcibly restricts the major and minor semi-axes of the ellipse to satisfy: and . This represents the semi-major axis parameter of the elliptical model in the parametric distortion model. This represents the minor semi-axis parameter of the elliptical model. This represents the theoretical reference value for the circumscribed circle radius of the flange. This represents the dimensionless physical deformation relative tolerance threshold. This represents the upper limit of the allowable offset of the elliptical semi-axis relative to the reference value of the radius of the theoretical flange circumcircle. During optimization, if the iteration step size calculated by the Jacobian matrix attempts to exceed this material mechanical limit, the system immediately performs gradient truncation and forced pull-back operations, thereby decoupling the pose perspective error from the micro-deformation parameters. The system logic in this embodiment is strictly configured as a "pure vision-driven independent computational architecture." That is, the process of extracting the principal direction distribution and establishing a parameterized distortion model is, by default, completed independently using only the pure visual radial geometric residuals of a high-confidence inlier set, without the intervention of any external physical stress sensors. Pure vision is the default execution method; optionally, when visual features are insufficient, external sensors are introduced as redundant input. For example, in the face of extreme weather conditions (such as blizzards causing lens obstruction and visual feature points falling below a safe threshold), the system allows the use of a degraded redundancy scheme, wirelessly mounting and fusing the basic physical data of strain gauges embedded in the flange inner wall, and using a Bayesian filtering framework to replace the initial pure visual estimation. This value forms a degradation compensation control. This setting ensures the algorithm's ability to operate independently in closed loops under the vast majority of operating conditions.

[0042] like Figure 5 As shown, finally, based on the optimized and converged relative pose matrix, the normal vector of the spatial fitting plane and the reference radial vector of the reference feature point are extracted, and the full-degree-of-freedom relative rotation command including the tilt angle and the phase deflection angle with direction sign is calculated.

[0043] The algorithm unifies the spatial features of the reference target and the target to be docked to the same global coordinate system through the output relative pose matrix. First, the normal vectors of the spatial fitting planes of the reference target and the target to be docked are extracted respectively. and By performing dot product (to calculate the absolute angle) and cross product (to determine the axis of rotation) operations on three-dimensional vectors, the tilt angle (Pitch, Roll) that makes the two planes parallel can be calculated.

[0044] To avoid the loss of rotational direction caused by pure angle calculation, a preset reference feature is selected as the phase reference point when calculating the phase deflection angle (Yaw). A reference radial vector is constructed pointing from the geometric center to the phase reference point. The reference radial vector of the target surface to be docked is then rotated and compensated to fit into the reference plane of the reference target surface according to the aforementioned tilt angle. Subsequently, not only is the absolute value of the angle between the two coplanar radial vectors calculated, but a triple scalar product operation is also introduced: let the coplanar radial vector on the reference side be... The radial vector of the docking sides to be rotated to coplanarity is The reference target surface normal vector is The system calculates... The symbol is used to accurately extract the clockwise or counterclockwise yaw direction. The result of the triple scalar product operation is the symbol determination scalar, which is obtained by the cross product of the coplanar radial vectors of the reference side and the coplanar radial vectors of the side to be docked, and then the dot product of the normal vector of the reference target surface. The resulting phase deflection angle, including the direction symbol, constitutes the full-degree-of-freedom relative rotation command that can be safely executed by the downstream robotic arm.

[0045] Example 2 like Figure 6 As shown, based on the same technical concept, this embodiment provides a transmission tower pose estimation and attitude angle calculation system based on image processing, used to execute the above-described method. This system is presented in the form of a combination of hardware architecture and functional modules.

[0046] The system includes a processor, memory, a dual-channel depth vision sensor interface network, and a communication bus. The memory stores executable computer vision algorithm instructions. The processor uses the bus to invoke instructions to instantiate and execute the following functional modules: Image data processing module 1 is used to acquire depth and color images of an optical calibration target surface containing periodic array features through a dual-channel depth vision sensor interface network, extract sub-pixel level feature coordinates, and realize depth back-projection reconstruction from the two-dimensional array to the initial three-dimensional spatial coordinates through a GPU acceleration unit.

[0047] Topological feature filtering module 2, with an embedded tensor calculator, is used to perform the Z-score coarse screening and plane / cylindrical fitting consistency check as described in the previous embodiments, and focuses on... , and Construct a penalty matrix with tolerance dead zone, and output a high-confidence set of point clouds by Gaussian kernel decay.

[0048] Parametric distortion modeling module 3, as the core of the system's pure vision-based non-sensor diagnostic unit, is dedicated to performing radial geometric residual traversal on high-confidence interior points in the memory buffer pool. When performing principal component analysis on the radial geometric residuals, if the first principal component of the covariance matrix is ​​significantly greater than the second principal component, the deformation triggering condition is satisfied. This is used to determine the existence of a systematic deformation trend with a single dominant direction, and to dynamically update the reprojection function in the video memory. The parameter model.

[0049] The joint decoupling optimization module 4 loads a nonlinear optimization solver with distortion upper limit constraint barriers (such as a secondary development framework based on the Ceres Solver library), integrates the reprojection error and parameterized elliptic constraint terms into the same residual block, synchronously iterates the camera's extrinsic tensor and physical deformation floating-point numbers, and performs gradient pullback.

[0050] The full-degree-of-freedom solution module 5 decomposes the coordinate transformation based on the optimized matrix information and performs inverse trigonometric function Euler angles of the normal and radial vectors. This module internally includes a sign determination subunit, which, after rotating and compensating the reference radial vector of the target surface to be docked to the reference plane of the target surface, obtains the coplanar radial vectors of the reference side and the target side; calculates the absolute value of the angle between the coplanar radial vectors of the reference side and the target side, and calculates the cross product of the cross product of the cross product of the cross product of the cross product of the cross product of the cross product of the cross product of the cross product and the target side, and the dot product between the normal vector of the target surface, to obtain the sign determination scalar; determines the clockwise and counterclockwise rotation direction of the phase deflection angle based on the sign determination scalar, and finally outputs a full-3D spatial pose control message containing Pitch, Roll, and a clearly oriented Yaw to the lower-level mechanical servo actuator.

[0051] The aforementioned functional modules operate within the underlying computing hardware architecture, relying on a multi-threaded asynchronous processing architecture and a ring buffer queue. Image data processing module 1 directly pushes the acquired high-resolution depth and color maps into the video memory buffer pool, while topology feature filtering module 2 performs tensor-level dead-zone penalty factor calculations on multiple frames of historical data by calling GPU operators. This high-concurrency data flow mechanism, combined with the early truncation of nonlinear constraint iterations, significantly reduces the CPU computation latency for single pose estimation, meeting the real-time requirements of servo control for heavy components.

[0052] This invention extends the rigid body pose estimation algorithm to the non-rigid deformation domain through low-level image residual reconstruction and mathematical boundary control without relying on external force-bearing hardware environment. This not only ensures the extremely high reliability of heavy automated equipment data, but also completes the full-degree-of-freedom attitude information urgently needed for spatial control.

[0053] Finally, it should be emphasized that the above embodiments are merely illustrative of the technical concept and preferred implementation of the present invention, and are not intended to exhaustively describe or limit the scope of protection of the present invention. Any person skilled in the art, after understanding the spirit and core technical solutions of the present invention, may make various modifications, equivalent substitutions, or improvements based on the content disclosed in the present invention, without departing from the basic principles of the present invention. These obvious modifications or substitutions should all be considered to be included within the scope of protection claimed by the present invention.

Claims

1. A method for estimating the pose and calculating the attitude angle of a power transmission tower based on image processing, characterized in that, include: Step 1: Obtain depth and color images of the optical calibration target surface containing periodic array features, extract the two-dimensional image pixel coordinates of the centroid of the sub-pixel level edge features on the target surface, and reconstruct the corresponding initial three-dimensional spatial coordinates. Step 2: Combining the preset topological geometric prior constraints and physical deformation tolerance threshold, the initial three-dimensional spatial coordinates are screened for spatial structure consistency to obtain a set of high-confidence interior points; Step 3: Calculate the principal direction distribution of the spatial radial residual based on the high-confidence in-point set, and when the preset deformation triggering conditions are met, establish a parameterized distortion model to update the three-dimensional spatial reference point; Step 4: Construct a joint objective function that includes the three-dimensional spatial reference point, camera pose matrix parameters, and distortion constraint boundary, and perform nonlinear iterative optimization to decouple and output the relative pose matrix of the target camera; Step 5: Based on the optimized and converged relative pose matrix, extract the normal vector of the spatial fitting plane and the reference radial vector of the reference feature point, and calculate the full-degree-of-freedom relative rotation command including the tilt angle and the phase deflection angle with direction sign.

2. The method for estimating the pose and calculating the attitude angle of a power transmission tower based on image processing according to claim 1, characterized in that, The screening process in step two includes: The standardized scores of the error set formed by the reconstructed initial three-dimensional spatial coordinates are calculated and statistically coarsely removed to obtain a coarsely screened set; Flange plane fitting and circular array projection fitting are performed on the coarse sieve set. Based on the calculated point-to-plane distance residual, circular fitting residual, and polar angle spacing residual, structural consistency screening is performed to obtain the structural screening set. The spatial circumcircle radius and spatial chord length of locally adjacent feature points in the structure screening set are calculated as the topological geometric prior constraints. A topological penalty factor with a dead zone is constructed based on the physical deformation tolerance threshold. Feature points that deviate from the tolerance dead zone are weighted down or eliminated, and the high-confidence inlier set is output.

3. The method for estimating the pose and calculating the attitude angle of a power transmission tower based on image processing according to claim 2, characterized in that, The formula for calculating the topology penalty factor with dead zones is as follows: ; in, For the first The topological penalty factor weights of each feature point and The penalty coefficient is... The radius of the circumcircle of the space is calculated in real time. This is a reference value for the theoretical flange circumscribed circle radius. The spatial chord length is calculated in real time. This is a reference value for the chord length of theoretically adjacent feature points. The physical deformation tolerance threshold is defined as follows.

4. The method for estimating the pose and calculating the attitude angle of a power transmission tower based on image processing according to claim 1, characterized in that, The process of extracting the principal direction distribution and establishing a parameterized distortion model in step three is configured to be executed independently by default using only pure visual image features. Its specific steps include: The radial geometric residual is obtained by calculating the difference between the measured distance and the theoretical radius from each inlier in the high-confidence inlier set to the center of the fitted circle. The radial geometric residual is analyzed along the circumference distribution characteristics. If the extreme value direction of the residual shows a bimodal or uniaxial main direction trend, the deformation triggering condition is met, and it is determined that a systematic physical deformation has occurred. The preset rigid circle model is replaced with the parametric distortion model, which is then made into an ellipse model. The major axis of the ellipse model coincides with the direction of the radial geometric residual extremum. The parameters of the major and minor axes of the ellipse are estimated based on the magnitude of the radial residual extremum.

5. The method for estimating the pose and calculating the attitude angle of a power transmission tower based on image processing according to claim 4, characterized in that, The formula for the joint objective function in step four is: ; ; in, For total energy error, This is the reprojection error term. For planar structural residuals, For parametric ellipse constraint terms, The parameters of the major and minor semi-axes of the ellipse are given. These are the plane weighting coefficients. It is an adaptive elastic relaxation factor. For two-dimensional image pixel coordinates, This is the intrinsic parameter matrix of the depth camera. The camera pose matrix parameters to be optimized are... The three-dimensional spatial reference point is dynamically generated.

6. The method for estimating the pose and calculating the attitude angle of a power transmission tower based on image processing according to claim 5, characterized in that, The distortion constraint boundary in step four is represented as a deformation upper limit constraint barrier, and the nonlinear iterative optimization includes: The initial pose is established based on local curvature or weak perspective approximation to break the ambiguity of projective geometry coupling between perspective tilt and physical deformation. When solving the joint objective function using the Levenberg-Marquardt algorithm, the parameters of the major and minor axes of the ellipse are forcibly constrained to meet the following requirements: and ;in, This represents the semi-major axis parameter of the ellipse model in the parametric distortion model. This represents the minor semi-axis parameter of the ellipse model. This represents the theoretical reference value for the circumscribed circle radius of the flange. This represents the dimensionless physical deformation relative tolerance threshold. This represents the upper limit of the allowable offset of the ellipse's semi-axis relative to the reference value of the radius of the circumscribed circle of the theoretical flange; when the calculated parameter iteration gradient causes the ellipse parameters to exceed the deformation upper limit constraint barrier, the iteration gradient is truncated and forcibly pulled back to the limit boundary, thereby decoupling the perspective pose error from the physical micro-deformation.

7. The method for estimating the pose and calculating the attitude angle of a power transmission tower based on image processing according to claim 1, characterized in that, Step five, which calculates the relative rotation command for the full degrees of freedom, includes: The spatial features of the reference target surface and the target surface to be docked are unified into the same global coordinate system by using the relative pose matrix; Perform dot product and cross product operations on the normal vectors of the two target surface space fitting planes after unifying the coordinate system to calculate the tilt angle that makes the two planes parallel, wherein the tilt angle includes pitch angle and roll angle; Select a preset reference feature as a phase reference point, construct the reference radial vector pointing from the geometric center to the phase reference point, and rotate and compensate the reference radial vector of the target surface to be docked to the reference plane of the reference target surface according to the tilt angle. Calculate the absolute value of the angle between two coplanar radial vectors, and extract the positive and negative signs using the triple scalar product of the two coplanar radial vectors and the normal vector of the reference target surface to obtain the phase deflection angle with a clear rotation direction.

8. A system for estimating the pose and calculating the attitude angle of a power transmission tower based on image processing, characterized in that, include: Image data processing module (1) is used to acquire depth and color images of an optical calibration target surface containing periodic array features, extract two-dimensional image pixel coordinates of the centroid of sub-pixel level edge features on the target surface, and reconstruct the corresponding initial three-dimensional spatial coordinates. The topology feature filtering module (2) is used to combine the preset topology geometric prior constraints and physical deformation tolerance threshold to perform spatial structure consistency filtering on the initial three-dimensional spatial coordinates and obtain a set of high confidence interior points. The parametric distortion modeling module (3) is used to calculate the main direction distribution of the spatial radial residual based on the high confidence in-point set, and to establish a parametric distortion model to update the three-dimensional spatial reference point when the preset deformation triggering conditions are met. The joint decoupling optimization module (4) is used to construct a joint objective function that includes the three-dimensional space reference point, camera pose matrix parameters and distortion constraint boundary, and to perform nonlinear iterative optimization to decouple and output the relative pose matrix of the target camera. The full-degree-of-freedom solution module (5) is used to extract the normal vector of the spatial fitting plane and the reference radial vector of the reference feature point according to the optimized and converged relative pose matrix, and calculate the full-degree-of-freedom relative rotation command including the tilt angle and the phase deflection angle with the direction sign. The image processing-based transmission tower pose estimation and attitude angle calculation system is used to execute the image processing-based transmission tower pose estimation and attitude angle calculation method of any one of claims 1-7.

9. The image processing-based transmission tower pose estimation and attitude angle calculation system according to claim 8, characterized in that, The topological feature screening module (2) is further used to: perform statistical coarse screening by calculating the standardized scores of the reconstructed initial three-dimensional spatial coordinates, and then perform structural consistency screening by using the distance residual from the point to the fitting plane and the polar angle spacing residual; and calculate the spatial circumcircle radius and spatial chord length of the local features as the topological geometric prior constraints, and construct a topological penalty factor matrix with tolerance dead zone according to the physical deformation tolerance threshold, thereby outputting the high confidence inlier set.

10. The image processing-based transmission tower pose estimation and attitude angle calculation system according to claim 8, characterized in that, The full-degree-of-freedom solution module (5) is further configured with a sign determination subunit, which is used to obtain the reference side coplanar radial vector and the target side coplanar radial vector after the reference radial vector of the target surface to be docked is rotated and compensated to the reference plane of the reference target surface; calculate the absolute value of the angle between the reference side coplanar radial vector and the target side coplanar radial vector, and calculate the dot product between the cross product of the reference side coplanar radial vector and the target side coplanar radial vector and the reference target surface normal vector to obtain the sign determination scalar; and determine the clockwise or counterclockwise rotation direction of the phase deflection angle according to the sign of the sign determination scalar.

Images