A cable dome node detection and three-dimensional coordinate reconstruction system, method and readable storage medium

Abstract

The application discloses a multi-module optimized UAV-YOLOv11-based cable dome node detection and three-dimensional coordinate reconstruction system and method based on multi-module optimized UAV-YOLOv11, and belongs to the technical field of cross engineering structure intelligent monitoring and engineering informatics. The system comprises: an image acquisition module, which is used for acquiring high-resolution images of a cable dome and performing pretreatment; a multi-module optimized detection module, which is based on a UAV-YOLOv11 model integrated with a C2PSA-BiFormer attention mechanism, a C3k2-GhostModule lightweight module and an NWD-CIoU loss function, and outputs two-dimensional pixel coordinates of nodes; a structure knowledge constraint module, which extracts structure geometry, topology and symmetry characteristics based on a shape finding algorithm and converts the characteristics into mathematical constraint equations; a coordinate reconstruction optimization module, which combines the two-dimensional pixel coordinates with the mathematical constraint equations to perform three-dimensional inversion optimization solving; and a cross-platform collaborative control module, which is used for scheduling automatic operation of the whole process. The application realizes high-precision and automatic three-dimensional monitoring of cable dome nodes.

Description

Technical Field

[0001] This invention relates to the interdisciplinary fields of intelligent monitoring of engineering structures and engineering informatics, specifically to spatial structure node target detection and 3D coordinate reconstruction based on UAV images, and particularly to a 3D coordinate inversion and reconstruction technology for large-span spatial structure nodes such as cable domes that integrates deep learning recognition with structural prior knowledge constraints. Specifically, it is a cable dome node detection and 3D coordinate reconstruction system, method, and readable storage medium based on multi-module optimized UAV-YOLOv11. Background Technology

[0002] Cable-stayed dome structures, as a highly efficient form of large-span spatial structure, are widely used in large public buildings such as stadiums and exhibition centers. As key force-transmitting components of cable-stayed dome structures, the accurate monitoring of their spatial positions is crucial for ensuring the construction precision and operational safety of the structure. Existing cable-stayed dome node monitoring technologies mainly suffer from the following problems: 1. Limited limitations of traditional measurement methods: While traditional total station or GPS measurement methods offer high accuracy, they require the deployment of control networks, resulting in low efficiency, high cost, and difficulty in achieving full-site, high-frequency real-time monitoring. 2. Limited accuracy of visual inspection: Existing visual inspection methods based on UAV imagery often exhibit small-scale, densely distributed node targets from high-altitude aerial perspectives, and are highly susceptible to interference from complex backgrounds such as cables and poles, leading to insufficient accuracy and robustness in node detection. 3. Lack of physical constraints in reconstruction: Existing 3D reconstruction methods are mostly based on pure visual geometry principles (such as SFM and MVS), lacking guidance from prior structural knowledge during the process of inverting from 2D pixels to 3D coordinates. Due to image noise and matching errors, purely visual reconstruction results often contain systematic errors, and the reconstructed node coordinates may violate the physical geometry of the structure (such as changes in rod length or incorrect topological connections), resulting in a lack of engineering interpretability. 4. Low level of automation: The various stages of existing systems (acquisition, identification, and reconstruction) are often independent of each other, lacking a collaborative integration mechanism and relying on manual intervention, making it difficult to meet the needs of modern engineering for automated and intelligent monitoring.

[0003] Therefore, developing an integrated system that can achieve intelligent recognition, automated three-dimensional coordinate reconstruction, and high-precision reconstruction driven by UAV imagery is a technical problem that urgently needs to be solved in the field of monitoring large-span spatial structures. Summary of the Invention

[0004] The purpose of this invention is to address the defects and shortcomings in the existing process of monitoring the spatial coordinates of cable dome structure nodes, especially the problems of low efficiency, high cost and difficulty in achieving real-time monitoring of the entire field by traditional measurement methods, and the unstable recognition and reconstruction accuracy of existing visual methods based on UAV imagery under complex working conditions. This invention provides a cable dome node detection and three-dimensional coordinate reconstruction system, method and readable storage medium based on multi-module optimized UAV-YOLOv11.

[0005] To achieve the objectives of this invention, the technical solution adopted is as follows: A system for detecting cable dome nodes and reconstructing 3D coordinates based on multi-module optimization is disclosed, comprising: The image acquisition module is configured to be mounted on an unmanned aerial vehicle platform to acquire high-resolution image data of the cable dome structure and perform camera calibration and preprocessing. A multi-module optimized detection module is connected to the image acquisition module to receive the image data, identify the cable dome nodes based on the optimized UAV-YOLOv11 deep learning model, and output the two-dimensional pixel coordinate information of the nodes. The structural knowledge constraint module is used for the shape-finding algorithm based on cable dome structures to extract the geometric characteristics, topological connections and symmetry characteristics of the structure and transform them into mathematical constraint equations. The coordinate reconstruction and optimization module is connected to the multi-module optimization and detection module and the structural knowledge constraint module. It is used to perform three-dimensional coordinate inversion based on the two-dimensional pixel coordinate information and camera parameters, and to optimize and solve the mathematical constraint equations to output the three-dimensional coordinates of the nodes. The cross-platform collaborative control module is used to control the data interaction and collaborative work of the above modules, realizing the fully automated operation from image acquisition to 3D coordinate output.

[0006] The UAV-YOLOv11 deep learning model in the multi-module optimized detection module includes a backbone network, a neck network, and a detection head. The backbone network or neck network is embedded with a C2PSA-BiFormer attention enhancement submodule, which adopts a two-layer routing attention mechanism to enhance the model's global perception of node features and suppress background interference. The backbone network or neck network is embedded with a lightweight C3k2-GhostModule submodule, which is used to reduce the number of model parameters and computational complexity while maintaining feature extraction capabilities. The detection head uses the NWD-CIoU composite loss function submodule for bounding box regression optimization. The NWD-CIoU composite loss function submodule combines normalized Wasserstein distance and cross-union ratio loss to improve the positioning accuracy of small-scale nodes.

[0007] The structural knowledge constraint module is specifically configured for: The theoretical geometric shape of the cable dome was obtained by performing shape-finding calculations using the MATLAB environment. Based on the theoretical geometry, a constraint set is constructed that includes member length constraints, node connection topological constraints, and overall structural symmetry constraints. The constraint set is standardized and used as a regularization term or constraint condition in the coordinate reconstruction optimization module.

[0008] The working process of the coordinate reconstruction and optimization module includes: Establish a camera imaging geometric model and construct the mapping relationship between the pixel coordinate system and the world coordinate system; Back projection calculation is performed based on the two-dimensional pixel coordinate information to obtain the initial solution of the three-dimensional coordinates of the node; Construct an objective function that includes a reprojection error minimization term and a structural prior constraint error minimization term, and use an optimization algorithm to iteratively correct the initial solution to output the final node 3D coordinates.

[0009] The system adopts a hierarchical deployment architecture: the image acquisition module is deployed on the UAV, the multi-module optimization and detection module is deployed on the edge computing terminal or the cloud, and the coordinate reconstruction optimization module and the structural knowledge constraint module are deployed on the cloud or ground workstation.

[0010] Furthermore, this invention also discloses a method for cable dome node detection and three-dimensional coordinate reconstruction based on the above-described system, comprising the following steps: S1: Use drones to plan flight paths to collect images of the cable dome, and perform data filtering, labeling and enhancement processing; S2: Construct and train a multi-module optimized UAV-YOLOv11 node detection model, introduce a two-layer routing attention mechanism and a lightweight convolution module, perform inference on the acquired images, and output the two-dimensional pixel coordinates of the nodes; S3: Based on the design parameters of the cable dome structure, perform shape-finding calculations to generate a set of mathematical constraints that include geometry, topology, and symmetry. S4: Establish a camera imaging model, back-project the two-dimensional pixel coordinates of the nodes to obtain the three-dimensional initial values, introduce the set of mathematical constraints for iterative optimization, and output the final three-dimensional coordinates.

[0011] In step S2, the training process of the multi-module optimized UAV-YOLOv11 node detection model includes: The C2PSA-BiFormer module is introduced to suppress interference from complex backgrounds on cable-stayed structures; The C3k2-GhostModule is introduced to reduce model inference overhead; The NWD-CIoU composite loss function is used instead of the traditional IoU loss function to enhance the robustness of localization for small target nodes.

[0012] The method also includes a periodic monitoring mode: Repeat steps S1 to S4 according to a set cycle to obtain three-dimensional coordinate data at different time points; By comparing historical coordinate data, the trend of nodal displacement or deviation can be calculated for structural construction verification or operational health monitoring.

[0013] Furthermore, the present invention also discloses a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described above.

[0014] The beneficial effects of this invention are as follows: The detection module of the system described in this invention features multi-module collaborative optimization, achieving an mAP@0.5:0.95 of 88.7% and a recall rate of 99.6%. It balances high-precision detection with lightweight design, making it directly adaptable to UAV edge deployment scenarios. It can perform 3D reconstruction by incorporating prior structural knowledge constraints, with an average node measurement error of only 2.72cm, superior to traditional systems and meeting industry-standard engineering-grade accuracy requirements. Through cross-platform collaborative control, it achieves fully automated operation from image acquisition to coordinate output, eliminating the need for manual intervention and significantly improving monitoring efficiency while reducing labor costs and operational errors.

[0015] This invention uses the physical and geometric laws of structures as constraints to solve coordinates, solving the problem that the reconstruction results of traditional systems have no physical meaning. The engineering reliability and interpretability of monitoring data are significantly improved. It adopts a modular design, and the structural knowledge constraints can be replaced to adapt to the monitoring of different large-span spatial structures. The hardware units can be flexibly combined and upgraded to adapt to the needs of different monitoring scenarios. It adopts a hierarchical deployment mode of UAV + edge terminal + cloud. Both the software and hardware are engineered and adapted to complex on-site environments. It is easy to operate and can be directly applied to structural construction and operation monitoring. Attached Figure Description

[0016] Figure 1 Diagram of UAV-YOLOv11 network architecture; Figure 2 A schematic diagram of the BRA (Two-Layer Routing Attention) optimization process; Figure 3Optimize the schematic diagram for the GhostModule module; Figure 4 A flowchart for knowledge-guided 3D coordinate reconstruction. Detailed Implementation

[0017] The following is in conjunction with the appendix Figure 1 To be continued Figure 4 The present invention is further illustrated by the embodiments: Example 1: A cable dome node detection and 3D coordinate reconstruction system based on multi-module optimization, comprising: The image acquisition module, configured to be mounted on a drone platform, acquires high-resolution image data of the cable dome structure and performs camera calibration and preprocessing. Specifically, the image acquisition module utilizes a drone equipped with a high-resolution camera to plan a flight path based on the scale of the cable dome. The acquired raw images undergo distortion correction and noise reduction preprocessing. The system supports self-calibration of shooting height to ensure that the image resolution meets the requirements for small target detection.

[0018] A multi-module optimized detection module, connected to the image acquisition module, is used to receive the image data, identify cable dome nodes based on an optimized UAV-YOLOv11 deep learning model, and output the two-dimensional pixel coordinate information of the nodes; the UAV-YOLOv11 deep learning model in the multi-module optimized detection module includes a backbone network, a neck network, and a detection head.

[0019] The backbone network or neck network embeds a C2PSA-BiFormer attention enhancement submodule (such as...). Figure 2 As shown, the C2PSA-BiFormer attention enhancement submodule employs a two-layer routing attention mechanism to enhance the model's global perception of node features and suppress background interference.

[0020] The backbone network or neck network embeds a lightweight C3k2-GhostModule submodule (such as...). Figure 3 (As shown), by using the idea of ​​GhostNet, more feature maps are generated through linear transformation, which significantly reduces the computational cost (FLOPs) and number of parameters while maintaining accuracy, making it possible to deploy on UAV onboard computers or edge computing boxes. In other words, it is used to reduce the number of model parameters and computational complexity while maintaining feature extraction capabilities.

[0021] To address the issue of small target size in aerial photography, the detection head employs a composite loss function submodule combining Normalized Wasserstein Distance (NWD) and Cross-Union-U (CIoU) for bounding box regression optimization. NWD is insensitive to small positional deviations, effectively solving the problem of IoU being overly sensitive to positional deviations in small target detection and improving the stability of bounding box regression. The NWD-CIoU composite loss function submodule combines normalized Wasserstein distance and cross-union ratio loss to improve the positioning accuracy of small-scale nodes.

[0022] The system also includes a structural knowledge constraint module, used for shape-finding algorithms based on cable dome structures to extract the geometric characteristics, topological connections, and symmetry properties of the structure, and transform them into mathematical constraint equations. Specifically, the structural knowledge constraint module is configured to perform shape-finding calculations for the cable dome using the MATLAB environment to obtain the theoretical geometric shape of the structure. Based on this theoretical geometric shape, it constructs a constraint set including member length constraints, node connection topological constraints, and overall structural symmetry constraints. This constraint set is then standardized and used as regularization terms or constraint conditions in the coordinate reconstruction optimization module. Simultaneously, the above constraints are transformed into a system of mathematical equations. ,in is the three-dimensional coordinate vector of the node.

[0023] The system also includes a coordinate reconstruction and optimization module, which is connected to the multi-module optimization and detection module and the structural knowledge constraint module. It is used to perform three-dimensional coordinate inversion based on the two-dimensional pixel coordinate information and camera parameters, and to optimize and solve the mathematical constraint equations to output the three-dimensional coordinates of the nodes. Specifically, it includes: Establish camera imaging geometry model Construct a mapping relationship between the pixel coordinate system and the world coordinate system; based on the two-dimensional pixel coordinate information... Perform back projection calculations to obtain the 3D coordinates of the nodes. The initial solution; construct the objective function containing the reprojection error minimization term and the structural prior constraint error minimization term: ; The first term represents the reprojection error, and the second term represents the structural prior constraint error. The initial solution is iteratively corrected using the Levenberg-Marquardt algorithm to output the final three-dimensional coordinates of the nodes.

[0024] The system also includes a cross-platform collaborative control module, which controls the data interaction and collaborative work of the above modules to achieve fully automated operation from image acquisition to 3D coordinate output.

[0025] The cable dome node detection and 3D coordinate reconstruction system based on multi-module optimization in this embodiment adopts a hierarchical deployment architecture: the image acquisition module is deployed on the UAV, the multi-module optimization detection module is deployed on the edge computing terminal or the cloud, and the coordinate reconstruction optimization module and the structural knowledge constraint module are deployed on the cloud or ground workstation. The collaborative operation of each level is achieved through the above deployment.

[0026] Example 2: This invention also discloses a method for cable dome node detection and three-dimensional coordinate reconstruction based on the system described in Example 1, comprising the following steps: S1: Use drones to plan flight paths to collect images of the cable dome, and perform data filtering, labeling and enhancement processing; S2: Construct and train a multi-module optimized UAV-YOLOv11 node detection model, introducing a two-layer routing attention mechanism and a lightweight convolution module to perform inference on the acquired images and output the two-dimensional pixel coordinates of the nodes; the training process of the multi-module optimized UAV-YOLOv11 node detection model includes: The C2PSA-BiFormer module is introduced to suppress interference from complex backgrounds on cable-stayed structures; The C3k2-GhostModule is introduced to reduce model inference overhead; The NWD-CIoU composite loss function is used instead of the traditional IoU loss function to enhance the robustness of localization for small target nodes.

[0027] S3: Based on the design parameters of the cable dome structure, perform shape-finding calculations to generate a set of mathematical constraints that include geometry, topology, and symmetry. S4: Establish a camera imaging model, back-project the two-dimensional pixel coordinates of the nodes to obtain the three-dimensional initial values, introduce the set of mathematical constraints for iterative optimization, and output the final three-dimensional coordinates.

[0028] S5: Establish a periodic monitoring mode, including repeating steps S1 to S4 according to a set period to obtain three-dimensional coordinate data at different time nodes; By comparing historical coordinate data, the trend of nodal displacement or deviation can be calculated for structural construction verification or operational health monitoring.

[0029] Example 3: A computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the steps of the method as described in Example 2.

[0030] The embodiments disclosed herein are preferred embodiments, but are not limited thereto. Those skilled in the art can readily grasp the spirit of the present invention based on the above embodiments and make different extensions and variations, but as long as they do not depart from the spirit of the present invention, they are all within the protection scope of the present invention.

Claims

1. A system for detecting cable dome nodes and reconstructing three-dimensional coordinates based on multi-module optimization, characterized in that, include: The image acquisition module is configured to be mounted on an unmanned aerial vehicle platform to acquire high-resolution image data of the cable dome structure and perform camera calibration and preprocessing. A multi-module optimized detection module is connected to the image acquisition module to receive the image data, identify the cable dome nodes based on the optimized UAV-YOLOv11 deep learning model, and output the two-dimensional pixel coordinate information of the nodes. The structural knowledge constraint module is used for the shape-finding algorithm based on cable dome structures to extract the geometric characteristics, topological connections and symmetry characteristics of the structure and transform them into mathematical constraint equations. The coordinate reconstruction and optimization module is connected to the multi-module optimization and detection module and the structural knowledge constraint module. It is used to perform three-dimensional coordinate inversion based on the two-dimensional pixel coordinate information and camera parameters, and to optimize and solve the mathematical constraint equations to output the three-dimensional coordinates of the nodes. The cross-platform collaborative control module is used to control the data interaction and collaborative work of the above modules, realizing the fully automated operation from image acquisition to 3D coordinate output.

2. The system according to claim 1, characterized in that, The UAV-YOLOv11 deep learning model in the multi-module optimized detection module includes a backbone network, a neck network, and a detection head. The backbone network or neck network is embedded with a C2PSA-BiFormer attention enhancement submodule, which adopts a two-layer routing attention mechanism to enhance the model's global perception of node features and suppress background interference. The backbone network or neck network is embedded with a lightweight C3k2-GhostModule submodule, which is used to reduce the number of model parameters and computational complexity while maintaining feature extraction capabilities. The detection head uses the NWD-CIoU composite loss function submodule for bounding box regression optimization. The NWD-CIoU composite loss function submodule combines normalized Wasserstein distance and cross-union ratio loss to improve the positioning accuracy of small-scale nodes.

3. The cable dome node detection and three-dimensional coordinate reconstruction system based on multi-module optimization according to claim 1, characterized in that, The structural knowledge constraint module is specifically configured for: The theoretical geometric shape of the cable dome was obtained by performing shape-finding calculations using the MATLAB environment. Based on the theoretical geometry, a constraint set is constructed that includes member length constraints, node connection topological constraints, and overall structural symmetry constraints. The constraint set is standardized and used as a regularization term or constraint condition in the coordinate reconstruction optimization module.

4. The cable dome node detection and three-dimensional coordinate reconstruction system based on multi-module optimization according to claim 1, characterized in that, The working process of the coordinate reconstruction and optimization module includes: Establish a camera imaging geometric model and construct the mapping relationship between the pixel coordinate system and the world coordinate system; Back projection calculation is performed based on the two-dimensional pixel coordinate information to obtain the initial solution of the three-dimensional coordinates of the node; Construct an objective function that includes a reprojection error minimization term and a structural prior constraint error minimization term, and use an optimization algorithm to iteratively correct the initial solution to output the final node 3D coordinates.

5. The cable dome node detection and three-dimensional coordinate reconstruction system based on multi-module optimization according to claim 1, characterized in that, The system adopts a hierarchical deployment architecture: the image acquisition module is deployed on the UAV, the multi-module optimization and detection module is deployed on the edge computing terminal or the cloud, and the coordinate reconstruction optimization module and the structural knowledge constraint module are deployed on the cloud or ground workstation.

6. A method for detecting cable dome nodes and reconstructing three-dimensional coordinates based on the system described in any one of claims 1-5, characterized in that, Includes the following steps: S1: Use drones to plan flight paths to collect images of the cable dome, and perform data filtering, labeling and enhancement processing; S2: Construct and train a multi-module optimized UAV-YOLOv11 node detection model, introduce a two-layer routing attention mechanism and a lightweight convolution module, perform inference on the acquired images, and output the two-dimensional pixel coordinates of the nodes; S3: Based on the design parameters of the cable dome structure, perform shape-finding calculations to generate a set of mathematical constraints that include geometry, topology, and symmetry. S4: Establish a camera imaging model, back-project the two-dimensional pixel coordinates of the nodes to obtain the three-dimensional initial values, introduce the set of mathematical constraints for iterative optimization, and output the final three-dimensional coordinates.

7. The method for detecting cable dome nodes and reconstructing three-dimensional coordinates according to claim 6, characterized in that, In step S2, the training process of the multi-module optimized UAV-YOLOv11 node detection model includes: The C2PSA-BiFormer module is introduced to suppress interference from complex backgrounds on cable-stayed structures; The C3k2-GhostModule is introduced to reduce model inference overhead; The NWD-CIoU composite loss function is used instead of the traditional IoU loss function to enhance the robustness of localization for small target nodes.

8. The method for detecting cable dome nodes and reconstructing three-dimensional coordinates according to claim 6, characterized in that, The method also includes a periodic monitoring mode: Steps S1 to S4 are repeated periodically to obtain three-dimensional coordinate data at different time points; By comparing historical coordinate data, the trend of nodal displacement or deviation can be calculated for structural construction verification or operational health monitoring.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the steps of the method according to any one of claims 6 to 8.

Images