Mobile phone visual tree barrier distance measurement method and device based on tower self-constraint

Abstract

This application discloses a mobile visual tree obstacle distance measurement method and device based on tower self-constraint, belonging to the field of computer vision technology. It includes: integrating visual SLAM, multi-source localization fusion, semantic segmentation guidance, and edge-cloud collaborative processing into a unified architecture to achieve intelligent, high-precision 3D reconstruction of power facilities. This solves the problems of high construction and maintenance costs, questionable measurement accuracy, lack of semantic constraints, strong network dependence, poor flexibility, and difficulty in direct application to long-distance, large-scale power inspection in existing technologies.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and in particular to a mobile phone vision tree obstacle distance measurement method and device based on tower self-constraint. Background Technology

[0002] In the operation and maintenance of power transmission lines, excessive tree growth can easily cause line discharge or even power outages. Therefore, monitoring the minimum distance of tree obstacles is a core task to ensure the safe operation of the power grid.

[0003] Currently, existing technologies can use tree obstacle recognition modules to identify and locate tall trees in power line corridors. These modules can identify tree targets from monitoring images and mark their positions within the images. Visual ranging technology is used to measure the clearance distance between the identified trees and the power lines. This technology is based on camera imaging principles and calculates spatial distance using pixel information in the image. The measured distance between the trees and the power lines is compared with the minimum safe distance required by transmission line safety regulations. When the measured distance is less than the safety threshold, the system determines that a tree obstacle hazard exists. The system provides remote access to external network users through a mobile client. When a tree obstacle distance is detected and does not meet safety requirements, an alarm message is sent to maintenance personnel, enabling real-time monitoring and early warning. An external hazard monitoring database for transmission lines is established to record historical monitoring data, alarm records, and handling status, providing data support for line maintenance decisions.

[0004] However, existing technologies are costly to build and maintain, have questionable measurement accuracy, lack semantic constraints, are highly dependent on networks, and have poor flexibility, making them difficult to apply directly to long-distance, large-scale power inspections, which urgently need to be addressed. Summary of the Invention

[0005] This application provides a mobile phone visual tree obstacle distance measurement method and device based on tower self-constraint, to solve the problems of high construction and maintenance costs, questionable measurement accuracy, lack of semantic constraints, strong network dependence, poor flexibility, and difficulty in direct application to long-distance large-scale power inspection in existing technologies.

[0006] The first aspect of this application provides a method for measuring tree barrier distance based on mobile phone vision using a self-constrained tower, comprising the following steps: acquiring multiple mobile phone scene images of a target tower, and identifying power semantic feature points and spatial combination relationships corresponding to the multiple mobile phone scene images, so as to extract at least one key tower component corresponding to the target tower based on the power semantic feature points and the spatial combination relationships of the feature points; reconstructing the at least one key tower component based on the spatial combination relationships of the feature points to obtain the corresponding relative scale reconstruction result, and converting the relative scale reconstruction result into an absolute scale reconstruction result using a pre-built multi-constraint joint optimization model, and performing a dense reconstruction operation on the absolute scale reconstruction result to generate a dense reconstruction result; segmenting the dense reconstruction result to obtain a tree point cloud, and determining the corresponding conductor spatial position based on the power semantic feature points, and generating a tree barrier distance measurement report corresponding to the target tower based on the minimum distance of the tree barrier corresponding to the minimum distance of the tree barrier based on the minimum distance of the tree barrier.

[0007] Optionally, in one embodiment of this application, the step of acquiring multiple mobile scene images of the target tower and identifying the power semantic feature points and spatial combination relationships corresponding to the multiple mobile scene images, so as to extract at least one key tower component corresponding to the target tower based on the power semantic feature points and the spatial combination relationships, includes: acquiring multiple mobile scene images of the target tower from different perspectives, and inputting each mobile scene image into a preset deep learning model to identify and locate the corresponding power semantic feature points, and establishing a spatial combination relationship of the same power semantic feature points among mobile scene images from different perspectives; extracting at least one key tower component of the target tower based on the power semantic feature points, the spatial combination relationships of the feature points, and a preset specification determination mechanism, wherein the at least one key tower component includes, but is not limited to, crossarms, insulator strings, and flanges.

[0008] Optionally, in one embodiment of this application, the step of reconstructing the at least one key tower component based on the spatial combination relationship of the feature points to obtain the corresponding relative scale reconstruction result, and converting the relative scale reconstruction result into an absolute scale reconstruction result using a pre-built multi-constraint joint optimization model, and performing a dense reconstruction operation on the absolute scale reconstruction result to generate a dense reconstruction result, includes: determining the corresponding target confidence match based on the spatial combination relationship of the feature points, and constructing a matching set based on the target confidence match and preset general image features, so as to reconstruct the at least one key tower component using a preset motion recovery structure strategy, epipolar geometric constraints, and the matching set to generate the relative scale reconstruction result; calculating the corresponding scale factor through the relative scale reconstruction result, scaling the measured length of each key tower component using the scale factor, and calculating the scaled measured length. The dimensional deviation from the known dimensions is used to construct corresponding component dimensional constraint terms based on the dimensional deviation; a corresponding reprojection error term is constructed based on the camera intrinsic parameters and camera pose of the target mobile phone; and a multi-constraint joint optimization model is constructed based on the reprojection error term, the component dimensional constraint term, and the preset tower geometric symmetry constraint. The camera intrinsic parameters of the target mobile phone are calculated through the multi-constraint joint optimization model to perform a self-calibration operation on the target mobile phone to obtain camera calibration parameters, thereby converting the relative scale reconstruction result into an absolute scale reconstruction result; the absolute scale reconstruction result and the camera calibration parameters are used as the initialization seed and joint optimization result of the three-dimensional Gaussian primitives, respectively; and based on the initialization seed and the joint optimization result, combined with a preset differentiable rendering strategy, the position, shape, opacity, and color parameters of each Gaussian primitive are iteratively optimized to generate the dense reconstruction result.

[0009] Optionally, in one embodiment of this application, the step of segmenting the dense reconstruction result to obtain a tree point cloud, determining the corresponding conductor spatial position based on the power semantic feature points, and generating a tree obstacle distance measurement report corresponding to the target tower based on the minimum distance of the tree obstacle corresponding to the tree obstacle cloud and the conductor spatial position, includes: determining the corresponding tower structure region based on the power semantic feature points and at least one key tower component, and performing two-dimensional semantic segmentation on the multiple mobile scene images based on the tower structure region to obtain multiple segmentation labels; back-projecting the multiple segmentation labels along a preset camera projection relationship to the dense reconstruction result, and assigning a semantic category to each three-dimensional Gaussian primitive or point cloud to extract the initial tree point cloud, and performing boundary region correction on the initial tree point cloud to obtain the tree point cloud; determining the corresponding conductor hanging point three-dimensional coordinates using the insulator connection points of the power semantic feature points, and based on the conductor hanging point... The system uses three-dimensional coordinates and the corresponding conductor type, span, and sag parameters from the line ledger for the target tower. It then fits the conductor's curve equation in three-dimensional space using the catenary equation to calculate the conductor's spatial position. The system traverses the tree point cloud, calculating and comparing the Euclidean distance from each tree point cloud to the conductor's spatial position to obtain the minimum Euclidean distance. Based on this minimum Euclidean distance, it determines the corresponding minimum distance to a tree barrier and records the nearest point location of the tree corresponding to this minimum distance. It then determines whether the nearest point location of the tree meets preset safety requirements. If the safety requirements are not met, a corresponding alarm flag is generated, and a corresponding defect work order is created. Based on the minimum distance to the tree barrier and the nearest point location of the tree, a tree barrier distance measurement report is generated and uploaded to the target power grid production management system. The tree barrier distance measurement report includes the minimum distance to the tree barrier, measurement uncertainty, a three-dimensional visualization, GPS coordinates of the measurement location, a measurement timestamp, and a safety assessment conclusion.

[0010] A second aspect of this application provides a mobile phone visual tree barrier distance measurement device based on tower self-constraint, comprising: a feature point detection module, used to acquire multiple mobile phone scene images of a target tower, and identify power semantic feature points and feature point spatial combination relationships corresponding to the multiple mobile phone scene images, so as to extract at least one key tower component corresponding to the target tower based on the power semantic feature points and the feature point spatial combination relationships; a point cloud reconstruction module, used to reconstruct the at least one key tower component based on the feature point spatial combination relationships, obtain the corresponding relative scale reconstruction result, and convert the relative scale reconstruction result into an absolute scale reconstruction result using a pre-built multi-constraint joint optimization model, and perform a dense reconstruction operation through the absolute scale reconstruction result to generate a dense reconstruction result; and a distance measurement module, used to segment the dense reconstruction result to obtain a tree point cloud, determine the corresponding conductor spatial position according to the power semantic feature points, and generate a tree barrier distance measurement report corresponding to the target tower based on the minimum distance of the tree barrier corresponding to the minimum distance of the tree barrier and the conductor spatial position and the tree point cloud.

[0011] Optionally, in one embodiment of this application, the feature point detection module includes: a collection unit, used to collect multiple mobile scene images from different perspectives corresponding to the target tower, and input each mobile scene image into a preset deep learning model to identify and locate the corresponding power semantic feature points, and establish a feature point spatial combination relationship between the same-name power semantic feature points in the mobile scene images from different perspectives; and an extraction unit, used to extract at least one key tower component of the target tower based on the power semantic feature points, the feature point spatial combination relationship, and a preset specification determination mechanism, wherein the at least one key tower component includes, but is not limited to, crossarms, insulator strings, and flanges.

[0012] Optionally, in one embodiment of this application, the point cloud reconstruction module includes: a reconstruction unit, configured to determine the corresponding target confidence match based on the spatial combination relationship of the feature points, and construct a matching set based on the target confidence match and preset general image features, so as to reconstruct the at least one key tower component using a preset motion recovery structure strategy, epipolar geometry constraints, and the matching set, thereby generating the relative scale reconstruction result; a first calculation unit, configured to calculate the corresponding scale factor through the relative scale reconstruction result, and scale the measured length of each key tower component using the scale factor, and calculate the size deviation between the scaled measured length and the known size, so as to construct the corresponding component size constraint term based on the size deviation; and a self-calibration unit, configured to determine the target mobile phone's camera internal dimensions based on the target mobile phone's internal dimensions. The system takes the camera pose as a reference and constructs a corresponding reprojection error term. Based on the reprojection error term, the component size constraint term, and the preset tower geometric symmetry constraint, it constructs a multi-constraint joint optimization model. The system calculates the camera intrinsic parameters of the target mobile phone through the multi-constraint joint optimization model to perform a self-calibration operation on the target mobile phone and obtain camera calibration parameters to convert the relative scale reconstruction result into an absolute scale reconstruction result. An iterative optimization unit is used to take the absolute scale reconstruction result and the camera calibration parameters as the initialization seed and joint optimization result of the three-dimensional Gaussian primitives, respectively. Based on the initialization seed and the joint optimization result, combined with a preset differentiable rendering strategy, iteratively optimizes the position, shape, opacity, and color parameters of each Gaussian primitive to generate the dense reconstruction result.

[0013] Optionally, in one embodiment of this application, the distance measurement module includes: a segmentation unit, used to determine the corresponding tower structure region based on the power semantic feature points and at least one key tower component, and to perform two-dimensional semantic segmentation on the multiple mobile scene images according to the tower structure region to obtain multiple segmentation labels; a correction unit, used to backproject the multiple segmentation labels along a preset camera projection relationship to the dense reconstruction result, and assign a semantic category to each three-dimensional Gaussian unit or point cloud to extract the initial tree point cloud, and to perform boundary region correction on the initial tree point cloud to obtain the tree point cloud; a second calculation unit, used to determine the corresponding conductor suspension point three-dimensional coordinates using the insulator connection points of the power semantic feature points, and to fit the conductor in three-dimensional space based on the conductor suspension point three-dimensional coordinates and the conductor type, span, and sag parameters of the line ledger corresponding to the target tower, and in combination with the catenary equation. The system uses the curve equation in the graph to calculate the spatial position of the conductor; a traversal unit is used to traverse the tree point cloud, calculate and compare the Euclidean distance from each tree point cloud to the spatial position of the conductor to obtain the minimum Euclidean distance, and determine the corresponding minimum distance of the tree barrier based on the minimum Euclidean distance, and record the position of the nearest tree point corresponding to the minimum distance of the tree barrier; a judgment unit is used to determine whether the position of the nearest tree point meets the preset safety requirements, and in response to the failure to meet the safety requirements, generate the corresponding alarm flag and generate the corresponding defect work order; a generation unit is used to generate the tree barrier distance measurement report based on the minimum distance of the tree barrier and the position of the nearest tree point, and upload the tree barrier distance measurement report to the target power grid production management system, wherein the tree barrier distance measurement report includes the minimum distance of the tree barrier, measurement uncertainty, three-dimensional visualization map, GPS coordinates of the measurement position, measurement timestamp, and safety assessment conclusion.

[0014] A third aspect of this application provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor. The processor executes the program to implement the mobile phone visual tree obstacle distance measurement method based on tower self-constraint as described in the above embodiments.

[0015] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described mobile phone visual tree obstacle distance measurement method based on tower self-constraint.

[0016] A fifth aspect of this application provides a computer program product, including a computer program that is executed to implement the above-described mobile phone visual tree obstacle distance measurement method based on tower self-constraint.

[0017] Therefore, the embodiments of this application have the following beneficial effects: The embodiments of this application can acquire multiple mobile scene images of a target iron tower and identify the power semantic feature points and spatial combination relationships corresponding to the multiple mobile scene images. Based on the power semantic feature points and spatial combination relationships, at least one key component of the iron tower corresponding to the target iron tower can be extracted. Based on the spatial combination relationships of the feature points, at least one key component of the iron tower can be reconstructed to obtain the corresponding relative scale reconstruction result. The relative scale reconstruction result is converted into an absolute scale reconstruction result using a pre-built multi-constraint joint optimization model. Dense reconstruction is then performed on the absolute scale reconstruction result to generate a dense reconstruction result. The dense reconstruction result is segmented to obtain tree point clouds. The corresponding conductor spatial position is determined based on the power semantic feature points. Based on the conductor spatial position and the tree point cloud, the minimum distance of the tree obstacle corresponding to the tree obstacle is calculated to generate a tree obstacle distance measurement report for the target iron tower. This application can automatically recover the true physical scale using only images taken by ordinary smartphones, combined with deep learning and 3D reconstruction algorithms, without relying on professional hardware equipment and manual targets, thereby achieving centimeter-level accurate measurement of safe distances. This solves the problems of high construction and maintenance costs, questionable measurement accuracy, lack of semantic constraints, strong network dependence, poor flexibility, and difficulty in direct application to long-distance, large-scale power inspection in existing technologies.

[0018] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0019] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein: Figure 1 This is a flowchart of a mobile phone visual tree obstacle distance measurement method based on tower self-constraint provided in an embodiment of this application; Figure 2 A schematic diagram of the execution logic of a mobile phone visual tree obstacle distance measurement method based on tower self-constraint provided in an embodiment of this application; Figure 3 This is an example diagram of a mobile phone visual tree obstacle distance measurement device based on tower self-constraint according to an embodiment of this application; Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0020] Among them, 10-Mobile visual tree obstacle distance measurement device based on tower self-constraint; 100-Feature point detection module, 200-Point cloud reconstruction module, 300-Distance measurement module; 401-Memory, 402-Processor, 403-Communication interface. Detailed Implementation

[0021] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0022] The following describes a method and apparatus for measuring tree barrier distance based on mobile phone vision using a self-constrained tower, according to embodiments of this application, with reference to the accompanying drawings. Addressing the problems mentioned in the background section, this application provides a method for measuring tree barrier distance using mobile phone vision based on a self-constrained tower. In this method, multiple mobile phone scene images of a target tower are acquired, and the corresponding power semantic feature points and spatial combination relationships of these feature points are identified. Based on these power semantic feature points and spatial combination relationships, at least one key tower component corresponding to the target tower is extracted. Based on the spatial combination relationships of the feature points, at least one key tower component is reconstructed to obtain the corresponding relative scale reconstruction result. The relative scale reconstruction result is then converted into an absolute scale reconstruction result using a pre-built multi-constraint joint optimization model. A dense reconstruction operation is performed on the absolute scale reconstruction result to generate a dense reconstruction result. The dense reconstruction result is segmented to obtain a tree point cloud. The corresponding conductor spatial position is determined based on the power semantic feature points. Based on the conductor spatial position and the tree point cloud, the minimum tree barrier distance corresponding to the tree barrier is calculated, and a tree barrier distance measurement report for the target tower is generated based on the minimum tree barrier distance. This application enables the automatic reconstruction of true physical scales using only images captured by ordinary smartphones, combined with deep learning and 3D reconstruction algorithms, without relying on specialized hardware or manual targets. This allows for centimeter-level precision measurement of safe distances. It solves the problems of existing technologies, such as high construction and maintenance costs, questionable measurement accuracy, lack of semantic constraints, strong network dependence, poor flexibility, and difficulty in direct application to long-distance, large-scale power line inspections.

[0023] Specifically, Figure 1 This is a flowchart illustrating a mobile phone visual tree obstacle distance measurement method based on tower self-constraint, provided as an embodiment of this application.

[0024] like Figure 1 As shown, the mobile phone visual tree obstacle distance measurement method based on tower self-constraint includes the following steps: In step S101, multiple mobile scene images of the target tower are acquired, and the power semantic feature points and spatial combination relationships of the multiple mobile scene images are identified, so as to extract at least one key tower component corresponding to the target tower based on the power semantic feature points and spatial combination relationships of the feature points.

[0025] Those skilled in the art should understand that in the operation and maintenance of power transmission lines, excessive tree growth can easily cause line discharge or even power outage accidents. Currently, existing tree obstacle measurement can continuously collect images of the line passage through fixed monitoring equipment, and automatically identify trees and measure distances using computer vision technology, thereby realizing intelligent and automated monitoring of tree obstacles on power transmission lines.

[0026] However, existing technologies have several drawbacks, such as the high cost and complex operation of high-precision measurement equipment based on RTK (Real-Time Kinematic) / GPS (Global Positioning System), the susceptibility of drone inspections to airspace control and complex terrain, the low accuracy and safety hazards of traditional manual visual inspection methods, and the fact that general computer vision measurement methods usually rely on the manual pre-deployment of physical targets or control points, making them difficult to apply directly to long-distance, large-scale power line inspections.

[0027] To address the aforementioned issues, the embodiments of this application can automatically recover the true physical scale using only images captured by ordinary smartphones, without relying on specialized hardware equipment or manual targets, and thereby achieve centimeter-level accurate measurement of safe distances.

[0028] In actual implementation, the embodiments of this application first take pictures of the iron tower scene with a mobile phone to obtain multiple mobile scene images, and then use the YOLO-Point deep learning model to perform feature detection on the multiple mobile scene images to generate corresponding power semantic feature points and their spatial combination relationships; then, the embodiments of this application can automatically extract the key components of the iron tower based on the detected semantic feature points and their spatial combination relationships.

[0029] Optionally, in one embodiment of this application, multiple mobile scene images of the target tower are acquired, and the power semantic feature points and spatial combination relationships of the multiple mobile scene images are identified. Based on the power semantic feature points and spatial combination relationships of the feature points, at least one key tower component corresponding to the target tower is extracted. This includes: acquiring multiple mobile scene images of the target tower from different perspectives, and inputting each mobile scene image into a preset deep learning model to identify and locate the corresponding power semantic feature points, and establishing a spatial combination relationship of the same power semantic feature points among the mobile scene images from different perspectives; and extracting at least one key tower component of the target tower based on the power semantic feature points, the spatial combination relationships of the feature points, and a preset specification determination mechanism. The at least one key tower component includes, but is not limited to, crossarms, insulator strings, and flanges.

[0030] Specifically, in the embodiments of this application, maintenance personnel can use ordinary smartphones to take multiple images (i.e., multiple mobile scene images, it is recommended to take 10-30 images) from different angles around the tower. When taking the images, ensure that the main structure of the tower is clearly visible and cover the area where the distance to the tree barrier needs to be monitored. During the shooting process, ensure that there is sufficient overlap between adjacent images (it is recommended to be more than 60%), and try to cover key components such as the tower crossarm and insulator strings from different directions and heights. This will provide a sufficient observation basis for subsequent feature matching and 3D reconstruction of components. At the same time, record the GPS coordinates and timestamps of the mobile phone.

[0031] Secondly, embodiments of this application can employ a pre-trained YOLO-Point deep learning model to automatically identify and locate electrical semantic feature points in the tower. This model can accurately detect structural features with clear physical meaning, such as crossarm endpoints, insulator connection points, and flange boundary points. The detection results include the feature point's category label, pixel coordinates, and confidence level.

[0032] After completing the detection of a single image, this embodiment of the application can further utilize semantic tags and structural topological relationships to establish the correspondence between feature points with the same name between images from different perspectives (i.e., the spatial combination relationship of feature points). Unlike traditional pure descriptor matching, the semantic-guided matching method of this embodiment of the application can use the category attributes of feature points and their relative positions in the tower structure (such as the vertical topological relationship between the "left end point of the upper crossarm" and the "left end point of the lower crossarm", and the relative orientation of feature points and the central axis of the tower) to disambiguate, thereby effectively solving the mismatch problem between left and right crossarms and multiple crossarms caused by the symmetrical structure of the tower. Moreover, the established cross-view semantic feature point correspondence (i.e., the spatial combination relationship of feature points) will be injected into the subsequent SfM process as a high-confidence match to participate in camera pose estimation and triangulation.

[0033] Subsequently, this application embodiment can automatically extract key components of the tower based on the detected semantic feature points and their spatial combination relationships, including crossarms (defined by the two endpoints of the same crossarm), insulator strings (defined by the sequence of insulator connection points), flanges (fitted by the flange boundary points), etc. These components are all standardized parts of the power industry, and their physical dimensions are known parameters: the length of the crossarm is a standard specification (such as 3.5 meters, 4.5 meters, etc.), the length of the insulator string can be calculated from the height of a single insulator and the number of insulators, and the diameter of the flange is a standard specification.

[0034] It should be noted that, in response to the problem of multiple standard specifications for similar components, this application embodiment can adopt a multi-path specification determination mechanism. For example, this application embodiment can prioritize obtaining voltage level and tower type information from the line ledger based on the tower number, and query a pre-set component size library to determine the specifications of each component. When the ledger information is unavailable, this application embodiment can use the invariant that the proportional relationship between components in relative reconstruction is not affected by scale to automatically determine the specifications: for example, the number of insulator discs can be directly counted from the image, and the height of a single disc is known. Therefore, the actual length of the crossarm can be deduced from the length of the insulator string as a reference, and compared with the standard specification library to select the specification with the smallest deviation as the determination result.

[0035] In addition, the embodiments of this application can also present the estimated component specifications to the operation and maintenance personnel for confirmation, as a backup guarantee for automatic judgment.

[0036] It is understood that the deep learning feature point detection model specifically designed for tower structure is capable of accurately identifying electrical semantic feature points with clear physical meaning, such as crossarm endpoints, insulator connection points, and flange boundary points, thereby providing accurate input for subsequent geometric constraints.

[0037] In step S102, based on the spatial combination relationship of feature points, at least one key component of the iron tower is reconstructed to obtain the corresponding relative scale reconstruction result. The relative scale reconstruction result is then converted into an absolute scale reconstruction result using a pre-built multi-constraint joint optimization model. Dense reconstruction is then performed using the absolute scale reconstruction result to generate a dense reconstruction result.

[0038] Furthermore, embodiments of this application can utilize a motion recovery structure strategy to recover camera pose and scene 3D sparse point cloud (i.e., relative scale reconstruction result) from multi-view images based on epipolar geometric constraints. A multi-constraint joint optimization model is established using the known physical dimensions of key tower components as constraints to convert the relative scale reconstruction result into a model with absolute metric (i.e., absolute scale reconstruction result). Subsequently, embodiments of this application can generate a dense 3D representation of the scene (i.e., dense reconstruction result) based on calibrated camera parameters (i.e., camera calibration parameters) and absolute scale information (i.e., absolute scale reconstruction result), combined with 3D Gaussian Splatting (3DGS) technology.

[0039] Optionally, in one embodiment of this application, at least one key component of the iron tower is reconstructed based on the spatial combination relationship of feature points to obtain the corresponding relative scale reconstruction result. The relative scale reconstruction result is then converted into an absolute scale reconstruction result using a pre-built multi-constraint joint optimization model. A dense reconstruction operation is performed using the absolute scale reconstruction result to generate a dense reconstruction result. This includes: determining the corresponding target confidence match based on the spatial combination relationship of feature points; constructing a matching set based on the target confidence match and preset general image features; reconstructing at least one key component of the iron tower using a preset motion recovery structure strategy, epipolar geometric constraints, and the matching set to generate a relative scale reconstruction result; calculating the corresponding scale factor using the relative scale reconstruction result; scaling the measured length of each key component of the iron tower using the scale factor; and calculating the scaled measured length. The dimensional deviation between the target phone and the known dimensions is used to construct corresponding component dimensional constraints. Based on the target phone's camera intrinsics and pose, a reprojection error term is constructed. A multi-constraint joint optimization model is built based on the reprojection error term, component dimensional constraints, and the preset tower geometric symmetry constraints. The target phone's camera intrinsics are calculated using this model to perform self-calibration, obtaining camera calibration parameters to convert the relative-scale reconstruction results into absolute-scale reconstruction results. The absolute-scale reconstruction results and camera calibration parameters are used as initialization seeds and joint optimization results for 3D Gaussian primitives, respectively. Based on these initialization seeds and joint optimization results, and combined with a preset differentiable rendering strategy, the position, shape, opacity, and color parameters of each Gaussian primitive are iteratively optimized to generate dense reconstruction results.

[0040] In actual implementation, the embodiments of this application can utilize SfM (Structure from Motion) technology to recover camera pose and scene 3D sparse point cloud from multi-view images based on epipolar geometric constraints.

[0041] It should be noted that SfM technology, as a computer vision technology that recovers camera pose and scene 3D structure through multi-view image sequences, is based on the principle of using epipolar geometry constraints to deduce 3D spatial information from the correspondence of 2D images. However, traditional SfM can only recover relative scale (the proportion is correct but the absolute size is unknown). One innovation of this application is to use the known component dimensions of the iron tower to achieve absolute scale recovery.

[0042] Epipolar geometry describes the geometric constraints on the imaging relationship of the same scene point under two viewpoints; the fundamental matrix F and the essential matrix encode the relative pose relationship between cameras. Embodiments of this application can utilize epipolar constraints to recover the relative pose of cameras from multiple images of a tower captured by a mobile phone, thereby providing a geometric basis for subsequent scale recovery.

[0043] During the reconstruction process, the cross-view semantic feature point correspondence obtained above is used as a high-confidence match to participate in the solution. Together with general image features (such as SIFT (Scale-Invariant Feature Transform)), they form a matching set to improve the reconstruction integrity and pose estimation stability of the weak texture metal component area of ​​the tower. The reconstruction results at this stage (i.e., the relative scale reconstruction results) have the correct geometric proportions, and the semantic feature points also obtain three-dimensional coordinates, but the absolute scale of the overall model is unknown.

[0044] Secondly, the embodiments of this application can use the known physical dimensions of the key components of the iron tower as constraints to establish a multi-constraint joint optimization model, and convert the reconstruction results of relative scale into a model with absolute measurement (i.e., absolute scale reconstruction results).

[0045] In the embodiments of this application, the objective function of the above-mentioned multi-constraint joint optimization model consists of the following three parts: 1. The reprojection error term in traditional bundle adjustment ensures visual geometric consistency; 2. Component size constraint, i.e., the measured length of each component in the reconstructed model after scaling factor. s The deviation between the scaled dimensions and the known standard dimensions covers the constraints on crossarm length, insulator string length, and flange diameter. 3. Geometric symmetry constraint of the tower, that is, the lengths of symmetrically arranged components (such as left and right crossarms) should be equal after reconstruction.

[0046] The mathematical expression for the objective function is:

[0047] in, Scale factor; For camera internal parameters; For camera pose; Three-dimensional points in a sparse point cloud of a scene; For the reconstruction model, the first The measured length (i.e., relative scale) of each key component; The known standard dimensions (i.e., absolute dimensions) of this critical component; To constrain weights; , This is the balance coefficient; This represents the actual observed pixel coordinates of the j-th 3D point in the i-th image; This represents the geometric symmetry constraint term of the tower, ensuring that symmetrically arranged components (such as left and right crossarms) have equal lengths.

[0048] Understandably, to improve the robustness of optimization, the weights of the constraints on each key component in the embodiments of this application are... The constraints of components with low detection confidence or insufficient observation perspectives are automatically downweighted based on their semantic detection confidence and the number of effective observation perspectives. At the same time, a robust kernel function (such as the Huber kernel) is introduced for the size constraint term. When a component causes abnormal constraint residuals due to detection errors or specification misjudgments, its impact on the overall optimization is effectively suppressed.

[0049] In addition, the scale estimates implied by multiple independent constraints such as crossarms, insulator strings, and flanges are cross-checked. Abnormal constraints that deviate from the overall scale estimate by more than a threshold (such as 3%) will be eliminated and re-optimized, thus ensuring that the system can still obtain a reliable scale solution when a single constraint fails. This is also a substantial advantage of multi-constraint design over a single-size reference.

[0050] It is understood that the embodiments of this application establish a multi-constraint joint optimization model by utilizing the known physical dimensions of standard components such as tower crossarms, insulator strings, and flanges, and solve for the scale factor from the SfM reconstruction results of the relative scale, thereby achieving automatic recovery of the absolute measurement scale. Secondly, the embodiments of this application can simultaneously use multiple tower component constraints (crossarms, insulators, flanges) for scale recovery, and detect measurement anomalies and improve reliability through cross-validation. When the scale factors calculated by different constraints are highly consistent, the measurement is confirmed to be valid; when significant differences occur, anomalies are automatically marked and a prompt for re-acquisition is made.

[0051] It is important to note that, during the optimization process, the embodiments of this application can incorporate camera focal length, principal point, and distortion parameters into the optimization variables. By utilizing the prior linearity of key tower components (such as the crossarm edge being a straight line in space) and known dimensional constraints, the intrinsic parameters of the camera can be jointly solved, achieving self-calibration of the camera through structural self-constraint, thereby eliminating the need for pre-calibration of ordinary smartphone cameras. After optimization, the embodiments of this application calculate the uncertainty of scale recovery based on the optimization residuals and the degree of consistency between various constraints, serving as the basis for the final measurement accuracy assessment.

[0052] Therefore, in the scale recovery process, the embodiments of this application optimize the camera intrinsic parameters online by utilizing the geometric constraints of the iron tower (such as symmetry, verticality, and parallelism), thereby eliminating the need for pre-calibration of the camera and effectively improving the convenience and robustness of tree barrier distance measurement.

[0053] Subsequently, embodiments of this application can generate a dense 3D representation of the scene using 3DGS technology based on calibrated camera parameters and absolute scale information. This technology models the scene as a set of 3D Gaussian distributions in space. Compared with traditional voxel or mesh representations, 3DGS technology achieves higher reconstruction quality under sparse viewpoint input and supports real-time rendering.

[0054] For the numerous slender rod structures and complex tree leaf textures present in the tower scene, traditional multi-view stereo matching methods are prone to depth estimation breaks and point cloud missing on slender rods. In contrast, 3D Gaussian representation has a stronger ability to express fine structures and complex textures, and is more suitable for the application scenario of this application.

[0055] In the dense reconstruction process, this embodiment uses the scale-corrected sparse point cloud (i.e., the absolute scale reconstruction result) as the initialization seed for 3D Gaussian primitives. The camera pose and intrinsic parameters are fixed as the jointly optimized result. The position, shape, opacity, and color parameters of each Gaussian primitive are iteratively optimized through differentiable rendering. Since the initialization and pose are inherited from the absolute scale model, the dense reconstruction result directly has the real metric scale, without the need for additional scale processing.

[0056] Therefore, the embodiments of this application ensure that the generated dense 3D scene maintains centimeter-level absolute measurement accuracy by transferring the calibrated absolute measurement information to the 3DGS dense reconstruction process, thus providing a precise 3D geometric basis for subsequent tree obstacle distance calculation.

[0057] In step S103, the dense reconstruction results are segmented to obtain tree point clouds, and the corresponding conductor spatial positions are determined based on power semantic feature points. Based on the conductor spatial positions and the minimum distance of tree obstacles corresponding to the tree point cloud, a tree obstacle distance measurement report corresponding to the target tower is generated.

[0058] Subsequently, embodiments of this application can automatically segment tree point clouds in the densely reconstructed 3D scene (i.e., dense reconstruction results), remove point clouds of other objects such as iron towers and wires, determine the spatial position of the wires, calculate the corresponding minimum distance to the tree barrier, and generate a corresponding tree barrier distance measurement report.

[0059] Therefore, the embodiments of this application integrate visual SLAM (Simultaneous Localization and Mapping), multi-source localization fusion, semantic segmentation guidance, and edge-cloud collaborative processing into a unified architecture, so that on-site measurement can be completed with only a mobile phone, without any fixed equipment or network dependence, and the absolute measurement accuracy at the centimeter level is achieved through the self-constraint of the tower structure, realizing intelligent and high-precision three-dimensional reconstruction of power facilities.

[0060] Optionally, in one embodiment of this application, the dense reconstruction result is segmented to obtain a tree point cloud, and the corresponding conductor spatial position is determined based on the power semantic feature points. Based on the conductor spatial position and the minimum distance to the tree obstacle corresponding to the tree point cloud, a tree obstacle distance measurement report corresponding to the target tower is generated. This includes: determining the corresponding tower structure region based on the power semantic feature points and at least one key tower component; performing two-dimensional semantic segmentation on multiple mobile scene images based on the tower structure region to obtain multiple segmentation labels; back-projecting the multiple segmentation labels along a preset camera projection relationship to the dense reconstruction result, and assigning a semantic category to each three-dimensional Gaussian primitive or point cloud to extract the initial tree point cloud; and performing boundary region correction on the initial tree point cloud to obtain the tree point cloud; determining the corresponding conductor hanging point three-dimensional coordinates using the insulator connection points of the power semantic feature points, and based on the conductor hanging point three-dimensional coordinates... The system records the conductor type, span, and sag parameters corresponding to the target and tower lines. It also fits the conductor's curve equation in three-dimensional space using the catenary equation to calculate the conductor's spatial position. The system traverses tree point clouds, calculating and comparing the Euclidean distance from each tree point cloud to the conductor's spatial position to obtain the minimum Euclidean distance. Based on this minimum Euclidean distance, it determines the corresponding minimum distance to a tree barrier and records the location of the nearest tree to the minimum barrier. It then determines whether the location of the nearest tree meets preset safety requirements. If the requirements are not met, it generates a corresponding alarm flag and a corresponding defect work order. Based on the minimum distance to the tree barrier and the location of the nearest tree, it generates a tree barrier distance measurement report and uploads it to the target power grid production management system. The tree barrier distance measurement report includes the minimum distance to the tree barrier, measurement uncertainty, a three-dimensional visualization, GPS coordinates of the measurement location, a measurement timestamp, and a safety assessment conclusion.

[0061] As one possible approach, the process of automatically segmenting the densely reconstructed 3D scene (i.e., the dense reconstruction result) in this application embodiment reuses the aforementioned power semantic feature points. That is, the tower structure area has been determined by the semantic feature points and key component extraction results, and its corresponding 3D point cloud can be directly marked and removed.

[0062] Based on this, the embodiments of this application can perform two-dimensional semantic segmentation on multi-view images (i.e., multiple mobile scene images), classifying the images into categories such as trees, iron towers, wires, sky, and ground, and back-projecting the segmentation labels along the camera projection relationship to three-dimensional space. Through multi-view voting, each three-dimensional Gaussian unit or point cloud is assigned a semantic category, and finally a pure tree point cloud is extracted. For boundary regions with blurred color and geometric features, spatial connectivity analysis and color consistency are combined for correction to ensure the integrity and accuracy of the tree point cloud.

[0063] Secondly, the embodiments of this application can determine the spatial location of the conductor. Those skilled in the art should understand that, since power conductors are only 1-3 cm in diameter, their images are weak in distant photographs, making it difficult to directly obtain a complete conductor model through dense reconstruction. The embodiments of this application can utilize the detected insulator connection points to determine the three-dimensional coordinates of the conductor's hanging point. Combined with the conductor type, span, and sag parameters (or typical values) in the line log, the curve equation of the conductor in three-dimensional space is fitted based on the catenary equation, thereby calculating the complete spatial location of the conductor. When the conductor imaging quality in the image is good, the embodiments of this application can also extract the pixel position of the conductor through straight line detection and back-project it into three-dimensional space to verify and correct the catenary calculation results. If necessary, it supports maintenance personnel in marking the conductor position on the image as an auxiliary method.

[0064] In a three-dimensional space with absolute measurement, the embodiments of this application can traverse each point in the tree point cloud, calculate the Euclidean distance from the point to the traverse spatial curve corresponding to the traverse spatial position, take the global minimum value (i.e. the minimum Euclidean distance) as the tree barrier safe distance (i.e. the minimum tree barrier distance), and record the corresponding tree nearest point position.

[0065] In addition, the embodiments of this application can compare the minimum distance to the fault with the safety distance standard threshold corresponding to the line voltage level, automatically determine whether the safety requirements are met, and generate alarm flags for cases that do not meet or are close to the critical value.

[0066] Finally, this embodiment of the application can generate a complete tree barrier distance measurement report. The report includes: the minimum distance value of the tree barrier (centimeter-level accuracy) and its measurement uncertainty (assessed based on the scale recovery uncertainty and reconstruction quality in step 5, such as "2.73 meters ± 0.05 meters"), a 3D visualization map marking the nearest point of the tree and the conductor position, GPS coordinates of the measurement location, measurement timestamp, and safety assessment conclusion. The report supports exporting and uploading to the power grid production management system. For tree barriers determined to not meet the safety distance requirements, a defect work order can be automatically generated. It also supports comparative analysis of measurement data from multiple periods at the same location, used to track tree growth trends and predict potential hazards.

[0067] Therefore, the embodiments of this application achieve precise restoration of absolute dimensions through joint constraints of multiple tower components and a self-calibration mechanism, with measurement accuracy reaching the centimeter level, meeting the requirements of power safety regulations. Secondly, the embodiments of this application automate the entire process from image acquisition to distance calculation, reducing human error and improving measurement consistency and repeatability. Thirdly, this application utilizes the tower structure itself as a natural target, eliminating the need for pre-deployed control points and adapting to various complex terrains and environmental conditions. Furthermore, the embodiments of this application can automatically generate measurement reports containing 3D visualization, GPS positioning, and timestamps, thus providing reliable data support for subsequent hazard investigation and accountability.

[0068] The execution logic of the mobile phone visual tree obstacle distance measurement method based on tower self-constraint of this application is explained below with reference to the accompanying drawings.

[0069] Figure 2 This is a schematic diagram illustrating the execution logic of the mobile phone visual tree obstacle distance measurement method based on tower self-constraint proposed in this application. Figure 2 As shown, the execution process of the mobile phone visual tree obstacle distance measurement method based on tower self-constraint in this application is as follows: S201: Taking photos of the iron tower scene with a mobile phone; S202: Detecting electrical semantic feature points using a deep learning model; S203: Extract key components of the tower, such as crossarms, insulators, flanges, etc. S204: Using structure-reconstruction-motion technique, camera pose and 3D sparse point cloud of scene are recovered from multi-view images based on epipolar geometry constraints; S205: Establish a multi-constraint joint optimization model using the known physical dimensions of the tower components as constraints; S206: Convert the reconstruction results of relative scales into a model with absolute metrics; S207: Based on calibrated camera parameters and absolute scale information, a dense 3D representation of the scene is generated using 3D Gaussian scattering technology; S208: Automatic segmentation of tree point clouds in densely reconstructed 3D scenes; S209: Calculate the minimum distance to the tree barrier; S2010: Generate a tree barrier distance measurement report.

[0070] In summary, the embodiments of this application only require a regular smartphone to complete the measurement task, without the need to carry professional measurement equipment or set up physical targets, which greatly reduces the equipment investment cost and the workload of maintenance personnel. Moreover, maintenance personnel can obtain the measurement results immediately after taking pictures at the inspection site, without relying on network transmission or back-end server processing, making it suitable for remote mountainous areas and areas without network coverage. In addition, maintenance personnel can complete the measurement from a safe distance, avoiding the safety risks of approaching high-voltage lines and climbing towers.

[0071] The method for measuring tree barrier distance based on mobile phone vision using self-constraint of iron towers, as proposed in this application, acquires multiple mobile phone scene images of the target iron tower and identifies the power semantic feature points and spatial combination relationships corresponding to the multiple mobile phone scene images. Based on the power semantic feature points and spatial combination relationships, at least one key component of the iron tower corresponding to the target iron tower is extracted. Based on the spatial combination relationships of the feature points, at least one key component of the iron tower is reconstructed to obtain the corresponding relative scale reconstruction result. The relative scale reconstruction result is converted into an absolute scale reconstruction result using a pre-built multi-constraint joint optimization model, and a dense reconstruction operation is performed on the absolute scale reconstruction result to generate a dense reconstruction result. The dense reconstruction result is segmented to obtain tree point clouds, and the corresponding conductor spatial position is determined according to the power semantic feature points. Based on the conductor spatial position and the tree point cloud, the minimum distance of the tree barrier corresponding to the tree barrier is calculated, and a tree barrier distance measurement report corresponding to the target iron tower is generated based on the minimum distance of the tree barrier. This application can automatically recover the real physical scale using only images taken by ordinary smartphones, combined with deep learning and 3D reconstruction algorithms, without relying on professional hardware equipment and manual targets, thereby achieving centimeter-level accurate measurement of safe distances.

[0072] Secondly, with reference to the accompanying drawings, a mobile phone visual tree obstacle distance measurement device based on tower self-constraint proposed according to an embodiment of this application is described.

[0073] Figure 3 This is a block diagram of a mobile phone visual tree obstacle distance measurement device based on tower self-constraint, according to an embodiment of this application.

[0074] like Figure 3 As shown, the mobile phone visual tree obstacle distance measurement device 10 based on tower self-constraint includes: a feature point detection module 100, a point cloud reconstruction module 200, and a distance measurement module 300.

[0075] The feature point detection module 100 is used to acquire multiple mobile scene images of the target tower and identify the power semantic feature points and feature point spatial combination relationships corresponding to the multiple mobile scene images, so as to extract at least one key tower component corresponding to the target tower based on the power semantic feature points and feature point spatial combination relationships.

[0076] The point cloud reconstruction module 200 is used to reconstruct at least one key component of the iron tower based on the spatial combination relationship of feature points, obtain the corresponding relative scale reconstruction result, and use a pre-built multi-constraint joint optimization model to convert the relative scale reconstruction result into an absolute scale reconstruction result. Furthermore, a dense reconstruction operation is performed on the absolute scale reconstruction result to generate a dense reconstruction result.

[0077] The distance measurement module 300 is used to segment the dense reconstruction results to obtain tree point clouds, determine the corresponding conductor spatial position based on power semantic feature points, and generate a tree barrier distance measurement report corresponding to the target tower based on the minimum distance of tree barriers corresponding to the conductor spatial position and tree point cloud.

[0078] Optionally, in one embodiment of this application, the feature point detection module 100 includes: an acquisition unit and an extraction unit.

[0079] The acquisition unit is used to acquire multiple mobile scene images from different perspectives corresponding to the target tower, and input each mobile scene image into a preset deep learning model to identify and locate the corresponding power semantic feature points, and establish the feature point spatial combination relationship of the same power semantic feature points between mobile scene images from different perspectives.

[0080] The extraction unit is used to extract at least one key tower component of the target tower based on power semantic feature points, spatial combination relationships of feature points and preset specification determination mechanisms. The at least one key tower component includes, but is not limited to, crossarms, insulator strings and flanges.

[0081] Optionally, in one embodiment of this application, the point cloud reconstruction module 200 includes: a reconstruction unit, a first calculation unit, a self-calibration unit, and an iterative optimization unit.

[0082] The reconstruction unit is used to determine the corresponding target confidence match based on the spatial combination relationship of feature points, and to construct a matching set based on the target confidence match and preset general image features, so as to reconstruct at least one key component of the tower using a preset motion recovery structure strategy, epipolar geometric constraints and matching set, and generate a relative scale reconstruction result.

[0083] The first calculation unit is used to calculate the corresponding scale factor through the relative scale reconstruction results, and to scale the measured length of each key component of the tower using the scale factor. It also calculates the size deviation between the scaled measured length and the known size, so as to construct the corresponding component size constraint terms based on the size deviation.

[0084] The self-calibration unit is used to construct the corresponding reprojection error term based on the camera intrinsic parameters and camera pose of the target mobile phone. Based on the reprojection error term, component size constraint term and preset tower geometric symmetry constraint, a multi-constraint joint optimization model is constructed. The camera intrinsic parameters of the target mobile phone are calculated through the multi-constraint joint optimization model to perform self-calibration operation on the target mobile phone and obtain camera calibration parameters to convert the relative scale reconstruction result into the absolute scale reconstruction result.

[0085] The iterative optimization unit is used to take the absolute scale reconstruction results and camera calibration parameters as the initialization seed and joint optimization results of the 3D Gaussian primitives, respectively. Based on the initialization seed and joint optimization results, and combined with the preset differentiable rendering strategy, it iteratively optimizes the position, shape, opacity and color parameters of each Gaussian primitive to generate a dense reconstruction result.

[0086] Optionally, in one embodiment of this application, the distance measurement module 300 includes: a segmentation unit, a correction unit, a second calculation unit, a traversal unit, a judgment unit, and a generation unit.

[0087] The segmentation unit is used to determine the corresponding tower structure region based on power semantic feature points and at least one key tower component, and to perform two-dimensional semantic segmentation on multiple mobile scene images based on the tower structure region to obtain multiple segmentation labels.

[0088] The correction unit is used to backproject multiple segmentation labels along a preset camera projection relationship into the dense reconstruction result, and assign a semantic category to each 3D Gaussian primitive or point cloud to extract the initial tree point cloud, and to perform boundary region correction on the initial tree point cloud to obtain the tree point cloud.

[0089] The second calculation unit is used to determine the three-dimensional coordinates of the corresponding conductor suspension point by using the insulator connection point of the power semantic feature point, and to calculate the spatial position of the conductor based on the three-dimensional coordinates of the conductor suspension point and the conductor type, span and sag parameters of the line ledger corresponding to the target tower, and to fit the curve equation of the conductor in three-dimensional space by combining the catenary equation.

[0090] The traversal unit is used to traverse the tree point cloud, calculate and compare the Euclidean distance from each tree point cloud to the spatial position of the traverse line, obtain the minimum Euclidean distance, determine the corresponding minimum distance to the tree barrier based on the minimum Euclidean distance, and record the position of the nearest tree point corresponding to the minimum distance to the tree barrier.

[0091] The judgment unit is used to determine whether the location of the nearest point of the tree meets the preset safety requirements. In response to the failure to meet the safety requirements, it generates the corresponding alarm flag and the corresponding defect work order.

[0092] The generation unit is used to generate a tree barrier distance measurement report based on the minimum distance to the tree barrier and the nearest point of the tree, and upload the tree barrier distance measurement report to the target power grid production management system. The tree barrier distance measurement report includes the minimum distance to the tree barrier, measurement uncertainty, three-dimensional visualization map, GPS coordinates of the measurement location, measurement timestamp, and safety assessment conclusion.

[0093] It should be noted that the foregoing explanation of the embodiment of the mobile phone visual tree obstacle distance measurement method based on tower self-constraint also applies to the mobile phone visual tree obstacle distance measurement device based on tower self-constraint in this embodiment, and will not be repeated here.

[0094] The mobile phone visual tree barrier distance measurement device based on tower self-constraint proposed in this application includes a feature point detection module 100, used to acquire multiple mobile phone scene images of the target tower and identify the power semantic feature points and feature point spatial combination relationships corresponding to the multiple mobile phone scene images, so as to extract at least one key tower component corresponding to the target tower based on the power semantic feature points and feature point spatial combination relationships; a point cloud reconstruction module 200, used to reconstruct at least one key tower component based on the feature point spatial combination relationships, obtain the corresponding relative scale reconstruction results, and use a pre-built multi-constraint joint optimization model to convert the relative scale reconstruction results into absolute scale reconstruction results, and perform dense reconstruction operation through the absolute scale reconstruction results to generate dense reconstruction results; and a distance measurement module 300, used to segment the dense reconstruction results to obtain tree point clouds, determine the corresponding conductor spatial position based on the power semantic feature points, and generate a tree barrier distance measurement report corresponding to the target tower based on the minimum distance of the tree barrier corresponding to the conductor spatial position and the tree point cloud. This application can automatically recover the true physical scale using only images taken by ordinary smartphones, without relying on professional hardware equipment and manual targets, and thus achieve centimeter-level accurate measurement of safe distances.

[0095] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device may include: The memory 401, the processor 402, and the computer program stored on the memory 401 and capable of running on the processor 402.

[0096] When the processor 402 executes the program, it implements the mobile phone visual tree obstacle distance measurement method based on tower self-constraint provided in the above embodiments.

[0097] Furthermore, electronic devices also include: Communication interface 403 is used for communication between memory 401 and processor 402.

[0098] The memory 401 is used to store computer programs that can run on the processor 402.

[0099] Memory 401 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0100] If the memory 401, processor 402, and communication interface 403 are implemented independently, then the communication interface 403, memory 401, and processor 402 can be interconnected via a bus to complete communication between them. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, Figure 4 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0101] Optionally, in a specific implementation, if the memory 401, processor 402, and communication interface 403 are integrated on a single chip, then the memory 401, processor 402, and communication interface 403 can communicate with each other through an internal interface.

[0102] Processor 402 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application.

[0103] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the above-described mobile phone visual tree obstacle distance measurement method based on tower self-constraint.

[0104] This application also provides a computer program product, including a computer program, which, when executed, is used to implement the above-described mobile phone visual tree obstacle distance measurement method based on tower self-constraint.

[0105] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0106] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0107] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or N executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0108] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0109] It should be understood that the various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. If implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0110] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0111] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0112] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of this application.

Claims

1. A method for measuring tree obstacle distance using mobile phone vision based on tower self-constraint, characterized in that, Includes the following steps: Multiple mobile scene images of the target tower are acquired, and the power semantic feature points and spatial combination relationships of the multiple mobile scene images are identified. Based on the power semantic feature points and the spatial combination relationships of the feature points, at least one key tower component corresponding to the target tower is extracted. Based on the spatial combination relationship of the feature points, the at least one key component of the iron tower is reconstructed to obtain the corresponding relative scale reconstruction result. The relative scale reconstruction result is then converted into an absolute scale reconstruction result using a pre-built multi-constraint joint optimization model. Dense reconstruction is then performed using the absolute scale reconstruction result to generate a dense reconstruction result. The dense reconstruction results are segmented to obtain tree point clouds, and the corresponding conductor spatial positions are determined based on the power semantic feature points. Based on the conductor spatial positions and the minimum distance of the tree obstacles corresponding to the tree point cloud, a tree obstacle distance measurement report corresponding to the target tower is generated according to the minimum distance of the tree obstacles.

2. The mobile phone visual tree obstacle distance measurement method based on tower self-constraint according to claim 1, characterized in that, The process of acquiring multiple mobile scene images of the target tower and identifying the power semantic feature points and spatial combination relationships corresponding to the multiple mobile scene images, in order to extract at least one key tower component corresponding to the target tower based on the power semantic feature points and the spatial combination relationships of the feature points, includes: Multiple mobile scene images from different perspectives corresponding to the target iron tower are collected, and each mobile scene image is input into a preset deep learning model to identify and locate the corresponding power semantic feature points, and establish the feature point spatial combination relationship of the same power semantic feature points between mobile scene images from different perspectives. Based on the power semantic feature points, the spatial combination relationship of the feature points, and the preset specification determination mechanism, at least one key tower component of the target tower is extracted, wherein the at least one key tower component includes, but is not limited to, crossarms, insulator strings, and flanges.

3. The mobile phone visual tree obstacle distance measurement method based on tower self-constraint according to claim 2, characterized in that, Based on the spatial combination relationship of the feature points, the at least one key component of the iron tower is reconstructed to obtain the corresponding relative scale reconstruction result. Then, a pre-built multi-constraint joint optimization model is used to convert the relative scale reconstruction result into an absolute scale reconstruction result. Finally, a dense reconstruction operation is performed on the absolute scale reconstruction result to generate a dense reconstruction result, including: The corresponding target confidence match is determined based on the spatial combination relationship of the feature points, and a matching set is constructed based on the target confidence match and the preset general image features. The at least one key component of the iron tower is reconstructed using the preset motion recovery structure strategy, epipolar geometric constraints and the matching set, and the relative scale reconstruction result is generated. The corresponding scale factor is calculated based on the relative scale reconstruction results, and the measured length of each key component of the tower is scaled using the scale factor. The size deviation between the scaled measured length and the known size is calculated, so as to construct the corresponding component size constraint item based on the size deviation. Based on the camera intrinsic parameters and camera pose of the target mobile phone, a corresponding reprojection error term is constructed. Based on the reprojection error term, the component size constraint term, and the preset tower geometric symmetry constraint, the multi-constraint joint optimization model is constructed. The camera intrinsic parameters of the target mobile phone are calculated through the multi-constraint joint optimization model to perform a self-calibration operation on the target mobile phone and obtain camera calibration parameters to convert the relative scale reconstruction result into an absolute scale reconstruction result. The absolute scale reconstruction result and the camera calibration parameters are used as the initialization seed and joint optimization result of the three-dimensional Gaussian primitives, respectively. Based on the initialization seed and the joint optimization result, and combined with a preset differentiable rendering strategy, the position, shape, opacity and color parameters of each Gaussian primitive are iteratively optimized to generate the dense reconstruction result.

4. The mobile phone visual tree obstacle distance measurement method based on tower self-constraint according to claim 3, characterized in that, The process involves segmenting the dense reconstruction results to obtain a tree point cloud, determining the corresponding conductor spatial location based on the power semantic feature points, and generating a tree barrier distance measurement report corresponding to the target tower based on the minimum distance between the conductor spatial location and the tree point cloud. Based on the power semantic feature points and at least one key component of the iron tower, the corresponding iron tower structure region is determined, and the multiple mobile terminal scene images are subjected to two-dimensional semantic segmentation according to the iron tower structure region to obtain multiple segmentation labels. The multiple segmentation labels are back-projected along a preset camera projection relationship into the dense reconstruction result, and each 3D Gaussian primitive or point cloud is assigned a semantic category to extract the initial tree point cloud. The initial tree point cloud is then corrected for boundary regions to obtain the tree point cloud. The insulator connection points of the power semantic feature points are used to determine the three-dimensional coordinates of the corresponding conductor suspension points. Based on the three-dimensional coordinates of the conductor suspension points and the conductor type, span and sag parameters of the line ledger corresponding to the target tower, and combined with the catenary equation to fit the curve equation of the conductor in three-dimensional space, the spatial position of the conductor is calculated. Traverse the tree point cloud, calculate and compare the Euclidean distance from each tree point cloud to the spatial position of the traverse line to obtain the minimum Euclidean distance, determine the corresponding minimum distance of the tree barrier based on the minimum Euclidean distance, and record the position of the nearest tree point corresponding to the minimum distance of the tree barrier. Determine whether the location of the nearest point of the tree meets the preset safety requirements. If the safety requirements are not met, generate a corresponding alarm flag and generate a corresponding defect work order. Based on the minimum distance to the tree barrier and the nearest point of the tree, a tree barrier distance measurement report is generated and uploaded to the target power grid production management system. The tree barrier distance measurement report includes the minimum distance to the tree barrier, measurement uncertainty, three-dimensional visualization map, GPS coordinates of the measurement location, measurement timestamp, and safety assessment conclusion.

5. A mobile phone visual tree obstacle distance measurement device based on tower self-constraint, characterized in that, include: The feature point detection module is used to acquire multiple mobile scene images of the target tower and identify the power semantic feature points and feature point spatial combination relationships corresponding to the multiple mobile scene images, so as to extract at least one key tower component corresponding to the target tower based on the power semantic feature points and the feature point spatial combination relationships. The point cloud reconstruction module is used to reconstruct at least one key component of the iron tower based on the spatial combination relationship of the feature points, obtain the corresponding relative scale reconstruction result, and use a pre-built multi-constraint joint optimization model to convert the relative scale reconstruction result into an absolute scale reconstruction result, and perform a dense reconstruction operation on the absolute scale reconstruction result to generate a dense reconstruction result. The distance measurement module is used to segment the dense reconstruction results to obtain tree point clouds, determine the corresponding conductor spatial position based on the power semantic feature points, and generate a tree obstacle distance measurement report corresponding to the target tower based on the minimum distance of the tree obstacle corresponding to the conductor spatial position and the tree point cloud.

6. The mobile phone visual tree obstacle distance measurement device based on tower self-constraint according to claim 5, characterized in that, The feature point detection module includes: The acquisition unit is used to acquire multiple mobile scene images from different perspectives corresponding to the target iron tower, and input each mobile scene image into a preset deep learning model to identify and locate the corresponding power semantic feature points, and establish the feature point spatial combination relationship of the same power semantic feature points between mobile scene images from different perspectives. The extraction unit is used to extract at least one key tower component of the target tower based on the power semantic feature points, the spatial combination relationship of the feature points and the preset specification determination mechanism, wherein the at least one key tower component includes, but is not limited to, crossarms, insulator strings and flanges.

7. The mobile phone visual tree obstacle distance measurement device based on tower self-constraint according to claim 6, characterized in that, The point cloud reconstruction module includes: The reconstruction unit is used to determine the corresponding target confidence match based on the spatial combination relationship of the feature points, and to construct a matching set based on the target confidence match and the preset general image features, so as to reconstruct the at least one key component of the iron tower using the preset motion recovery structure strategy, epipolar geometry constraints and the matching set, and generate the relative scale reconstruction result. The first calculation unit is used to calculate the corresponding scale factor through the relative scale reconstruction result, and to scale the measured length of each key component of the tower using the scale factor, and to calculate the size deviation between the scaled measured length and the known size, so as to construct the corresponding component size constraint item based on the size deviation. The self-calibration unit is used to construct a corresponding reprojection error term based on the camera intrinsic parameters and camera pose of the target mobile phone, and to construct the multi-constraint joint optimization model based on the reprojection error term, the component size constraint term and the preset tower geometric symmetry constraint. The multi-constraint joint optimization model is used to calculate the camera intrinsic parameters of the target mobile phone to perform self-calibration operation on the target mobile phone and obtain camera calibration parameters to convert the relative scale reconstruction result into the absolute scale reconstruction result. The iterative optimization unit is used to take the absolute scale reconstruction result and the camera calibration parameters as the initialization seed and joint optimization result of the three-dimensional Gaussian primitives, respectively, and based on the initialization seed and the joint optimization result, combined with a preset differentiable rendering strategy, iteratively optimize the position, shape, opacity and color parameters of each Gaussian primitive to generate the dense reconstruction result.

8. An electronic device, characterized in that, include: The method includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the mobile phone visual tree obstacle distance measurement method based on tower self-constraint as described in any one of claims 1-4.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by the processor to implement the mobile phone visual tree obstacle distance measurement method based on tower self-constraint as described in any one of claims 1-4.

10. A computer program product, comprising a computer program, characterized in that, The computer program is executed to implement the mobile phone visual tree obstacle distance measurement method based on tower self-constraint as described in any one of claims 1-4.

Images