Construction site potential safety hazard detection method and system based on unmanned aerial vehicle inspection vision

Abstract

The invention discloses a construction site potential safety hazard detection method and system based on unmanned aerial vehicle inspection vision, and particularly relates to the technical field of construction site safety monitoring, and the system comprises a data acquisition and topology construction module which is used for collecting an unmanned aerial vehicle multi-source sensing flow and aligning a space prior map, outputting a dynamic scene topological graph containing the cross-modal spatial-temporal characteristics; the double-layer optimization evaluation calculation engine is embedded with combined mathematical derivation logic comprising an upper-layer strategy solver and a lower-layer feature evaluator and is used for solving a space-time risk potential energy index and a self-adaptive risk judgment threshold value in a multi-dimensional constraint space; and the closed-loop response and servo control module is used for comparing the index with a threshold value to generate a hidden danger alarm work order of a dynamic space intrusion level, and sending a reverse viewpoint blind compensation instruction to an unmanned aerial vehicle flight control system through a gradient conversion model. According to the invention, visual leak detection and shielding blind areas in a severe environment are effectively overcome, and efficient closed-loop management and control of substation cross construction dynamic space invasion hidden dangers are realized.

Description

A Method and System for Detecting Safety Hazards at Construction Sites Based on Unmanned Aerial Vehicle (UAV) Inspection Vision Technical Field

[0001] This invention relates to the field of construction site safety monitoring technology, and more specifically, to a method and system for detecting safety hazards at construction sites based on UAV inspection vision. Background Technology

[0002] With the rapid advancement of power infrastructure construction, the number of new, expanded, and upgraded substations is increasing, leading to a highly complex and dynamic working environment at construction sites. These high-risk operations often involve large-scale hoisting machinery, operation of high-voltage live conductors, and cross-operation by multiple trades, resulting in rapidly changing spatial safety conditions at the construction site. To ensure safe and compliant construction operations, frequent dynamic monitoring and measurement of the spatial topological relationships between various mobile machinery, construction personnel, and static live facilities are necessary. In recent years, with the synergistic development of the low-altitude economy and artificial intelligence technology, drones equipped with high-definition visual sensors have been widely adopted in daily inspection tasks at construction sites and power projects due to their maneuverability and wide coverage. By combining computer vision and deep learning algorithms such as graph neural networks, intelligent inspection systems can extract the spatiotemporal geometric features of operating equipment and scene components from continuous aerial image sequences, thereby constructing a digital virtual scene reflecting the actual operating status of the site. This provides a rich data foundation for the management of engineering safety hazards and the quantitative assessment of operational status.

[0003] Currently, existing visual inspection technologies for construction safety still have significant limitations when dealing with complex, overlapping work scenarios in substations. On the one hand, traditional safety early warning models often rely on fixed physical distance thresholds for unidirectional judgment. Static judgment mechanisms are prone to missing serious hidden dangers when faced with common adverse working conditions on construction sites, such as backlighting, dust, or partial equipment obstruction, due to the uncertainty in visual feature extraction. On the other hand, conventional drone inspections usually follow preset fixed three-dimensional routes and lack adaptive tracking capabilities for sudden high-risk operations. This not only makes it difficult for the inspection camera to actively avoid spatial obstructions to obtain high-confidence data on hidden nodes, but also makes it impossible to form real-time closed-loop servo control when potential spatial intrusion hazards occur, ultimately resulting in unavoidable blind spots and lags in hazard identification. Summary of the Invention

[0004] The purpose of this invention is to provide a method and system for detecting safety hazards at construction sites based on UAV inspection vision. By adaptively and dynamically reducing the risk threshold and using UAV active viewpoint blind spot filling based on potential energy gradient, it not only effectively overcomes visual omissions and blind spots in harsh environments, but also achieves efficient closed-loop management of dynamic spatial intrusion hazards during cross-construction in substations.

[0005] This invention is achieved through the following technical solution:

[0006] A method for detecting safety hazards at construction sites based on UAV inspection vision includes the following steps: controlling a UAV to inspect a substation construction site to simultaneously acquire continuous visual image sequences of lifting machinery and electrical equipment, as well as a UAV spatial pose sequence; based on the continuous visual image sequences and spatial pose sequences, combined with a pre-set prior spatial map of energized bodies, extracting the spatiotemporal geometric features of working equipment nodes, personnel nodes, and energized body nodes through a graph neural network to construct a dynamic scene topology map representing the transient correlation of cross-construction; targeting potential spatial intrusion hazards in the dynamic scene topology map, constructing a two-layer optimization model to couple and optimize the UAV observation parameters and risk identification network, combined with pre-set electromagnetic exclusion zone spatial constraints and visual observability constraints, to output a spatiotemporal risk potential energy index representing the intrusion probability of equipment nodes and energized body nodes; based on the spatiotemporal risk potential energy index and the adaptive risk judgment threshold output by the two-layer optimization model, quantifying the dynamic spatial intrusion level of lifting machinery and energized bodies or personnel, generating detection results, and driving the UAV to perform active viewpoint blind spot filling along the risk potential energy gradient to complete the closed-loop detection of safety hazards at the construction site.

[0007] Optionally, the spatiotemporal risk potential energy index is a continuous scalar function, and its calculation formula is as follows:

[0008] in, Indicators representing spatiotemporal risk potential energy; These represent the weighting coefficients of the electrical potential energy term, the motion collision term, and the visual information entropy term, respectively. The set represents the set of high-voltage charged body nodes contained in the prior spatial map of charged bodies; i represents the set. The index of the i-th high-voltage charged body node in the data; This represents the equivalent charge hazard factor corresponding to the voltage level of the i-th high-voltage charged body node; This represents the extracted three-dimensional spatial coordinate vector of the end of the crane boom; Represents the three-dimensional spatial coordinate vector of the i-th high-voltage charged body node; Indicates the current inspection time; This indicates the length of the predicted future time domain; t is the time integration variable; The collision probability density function represents the kinematic interference between the lifting machinery and the personnel. and Let represent the velocity vectors of the lifting machinery node and the personnel node at time t, respectively; This represents the function for calculating information entropy. This represents the visual feature observation matrix that includes occlusion rate and scale distortion rate when extracting hazard features from the current UAV observation viewpoint.

[0009] Optionally, the upper-level optimization of the two-level optimization model takes maximizing risk separability and minimizing observation cost as joint objectives, solving for the optimal observation viewpoint set and adaptive risk decision threshold for the UAV. Its upper-level objective function is defined as:

[0010] in, This represents the objective function value of the upper-level optimization. This represents the set of UAV observation viewpoint parameters to be optimized, including yaw angle, pitch angle, and relative distance. This represents the adaptive risk decision threshold to be optimized. The set of positive samples representing historical high-risk space intrusion threats; u is the index of the positive samples in this set; This represents the set of negative samples for safe construction standards; v is the index of the negative sample in this set. and These represent the observation viewpoints respectively. Below, the spatiotemporal risk potential energy indicators output by positive sample u and negative sample v; Indicates the adaptive risk decision threshold The risk of constraints can be separated from marginal parameters; This represents the penalty adjustment constant coefficient; Indicates the drone's current pose Shift to target viewpoint The algebraic model function of time and power consumption.

[0011] Optionally, the lower-level optimization of the two-level optimization model is given and Under the constraints, the network weight set of the risk identification network is updated, and its lower-level objective function is defined as:

[0012] in, This represents the objective function value of the lower-level optimization. This represents the set of all learnable tensor weights for the risk identification network; The cross-entropy loss function represents the risk state classification. Indicates the current network weight set The spatiotemporal risk potential energy index of the predicted output; This represents the true label vector of the weakly supervised risk input; Represents the weight coefficients of the smoothing regularization term; The spatiotemporal risk potential energy index representing the predicted output For the observation viewpoint The gradient vector of the partial derivatives.

[0013] Optional, the preset electromagnetic exclusion zone space constraint is calculated using the following formula:

[0014] in, This indicates the time window for predicting the motion trajectories of drones and cranes. and Let represent the three-dimensional spatial coordinates of the UAV's center of mass and the end of the crane boom at time t, respectively; Represents the set of spatial coordinate points of a three-dimensional envelope cylinder set; and These represent the minimum safe clearance settings for drones and cranes, respectively, in accordance with electrical safety regulations.

[0015] Optionally, the adaptive risk decision threshold is calculated based on the visual uncertainty caused by the harsh environment of the construction site, and its mapping formula is as follows:

[0016] in, This represents the adaptive risk decision threshold that is optimized and solved by the upper layer and output to the decision module. This represents the preset baseline static risk threshold under ideal unobstructed lighting conditions; k represents the threshold sensitivity gain constant. This represents the covariance matrix output by the risk identification network when extracting hazard feature vectors under backlight or occlusion conditions at the current viewpoint; The operator represents the trace of the covariance matrix, used to quantify the total variance of the eigenvectors.

[0017] Optionally, the UAV can be driven to perform active viewpoint blind spot detection along the risk potential energy gradient, specifically by calculating the spatiotemporal risk potential energy index. Regarding the current three-dimensional spatial coordinates of the drone The partial derivatives are used to obtain the risk potential gradient vector in three-dimensional space. The formula is Determine whether the visual information entropy in the current state is higher than the tolerance threshold. If so, then change the gradient vector. The vector is converted into a velocity command vector for the UAV flight control system through coefficient mapping. The formula is ;in, Represents the risk potential gradient vector; Represents the coordinate variable Operators for finding gradients in three-dimensional space; Indicators representing spatiotemporal risk potential energy; This represents the three-axis velocity command vector sent to the drone's underlying flight control system. This represents the velocity mapping gain coefficient, which is greater than 0.

[0018] Optionally, a dynamic scene topology map representing the transient correlation of cross-construction is constructed. Specifically, this involves: using a bipartite graph matching algorithm, performing affine transformation and mapping between two-dimensional crane pixel nodes extracted from continuous visual image sequences and three-dimensional BIM component nodes in the prior spatial map of charged bodies; identifying and quantifying hidden crane node components obscured by transformers or walls in the visual viewpoint; constructing a state evolution model based on the material physical properties and rigid body kinematic equations of the crane's slewing and luffing mechanisms; and combining the state evolution model with a Kalman filter to perform temporal extrapolation and completion of the spatial positions of hidden node components, thereby completing the digital reconstruction of hidden nodes in the dynamic scene topology map.

[0019] This construction site safety hazard detection system based on UAV inspection vision is used to implement a method for detecting safety hazards at construction sites based on UAV inspection vision. It includes: a data acquisition and topology construction module, used to collect multi-source perception streams from the UAV and align them with spatial prior maps to output a dynamic scene topology map containing cross-modal spatiotemporal features; a two-layer optimization evaluation calculation engine, embedding joint mathematical derivation logic including an upper-layer policy solver and a lower-layer feature evaluator, used to calculate spatiotemporal risk potential energy indicators and adaptive risk judgment thresholds under multi-dimensional constrained space; and a closed-loop response and servo control module, used to compare indicators and thresholds to generate dynamic spatial intrusion level hazard alarm work orders, and send reverse viewpoint blind spot filling commands to the UAV flight control system via a gradient transformation model.

[0020] Optionally, the computing power allocation of the dual-layer optimization evaluation computing engine adopts a cloud-edge decoupled asynchronous update mechanism. Specifically, the lower-layer feature evaluator is compiled into a TensorRT model and deployed on the UAV's airborne edge computing node. It calculates the spatiotemporal risk potential energy index under the current viewpoint by forward inference at a set frequency to ensure the real-time performance of space intrusion hazard detection. The upper-layer policy solver is deployed on the construction site edge server or in the cloud. It asynchronously solves the new optimal observation viewpoint set and adaptive risk decision threshold with a multi-frame sliding window period and sends them down to the airborne edge computing node. Through edge-cloud collaboration, it eliminates the identification blind spots under harsh working conditions.

[0021] The technical solution of this invention has at least the following advantages and beneficial effects: On the one hand, this invention adaptively and dynamically downgrades the risk judgment threshold based on the visual uncertainty caused by the construction site environment, enabling the system to dynamically improve the sensitivity of hidden danger alarms when facing adverse working conditions such as dust or visual obstruction, thus completely overcoming the problem of missed detection caused by a single fixed threshold; on the other hand, this invention drives the UAV to perform active viewpoint blind spot filling based on the partial derivative gradient of the risk potential energy, which can not only guide the UAV to perform adaptive maneuvers in real time to the unobstructed three-dimensional space where the risk uncertainty decreases the fastest, but also ensures millisecond-level response at the edge side by combining the asynchronous computing architecture of edge-cloud collaboration, truly building a closed-loop safety defense system from hidden danger perception to active intervention in flight trajectory. Attached Figure Description

[0022] Figure 1 is a flowchart illustrating the construction site safety hazard detection method based on UAV inspection vision provided by the present invention; Figure 2 is a schematic diagram illustrating the principle of the construction site safety hazard detection system based on UAV inspection vision provided by the present invention. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0024] Referring to Figures 1 and 2, this embodiment provides a method and system for detecting safety hazards at construction sites based on UAV inspection vision. This addresses the technical challenge of accurately, in real-time, and in a closed-loop manner detecting and controlling spatial intrusion hazards caused by dynamic operations of large machinery, cross-operations of multiple trades, and complex electromagnetic and visual environmental interference in high-risk power construction environments such as substations.

[0025] The security hazard detection system provided in this embodiment mainly consists of three parts at the hardware level: front-end perception and control unit, edge computing server, and cloud strategy planning center.

[0026] The front-end perception and control unit is integrated onto an industrial-grade drone with high-precision real-time dynamic differential positioning capabilities. As a specific example, this embodiment uses a drone equipped with a high-performance gimbal camera as the flight platform. The drone's gimbal integrates a 20x optical zoom camera, a wide-angle camera, a thermal imaging camera, and a laser rangefinder module, enabling it to acquire high-definition visual image sequences and precise distance information of targets at the construction site in all weather conditions and from multiple dimensions. The drone's built-in flight control system integrates an inertial measurement unit (IMU), including a three-axis accelerometer and a three-axis gyroscope, and is deeply integrated with an RTK-GNSS module, continuously outputting centimeter-level precision three-dimensional spatial pose sequences for the drone, including longitude, latitude, altitude, and roll, pitch, and yaw attitude angles. The drone also carries an embedded edge computing device as an onboard lower-level feature estimator for performing high-frequency real-time model inference tasks.

[0027] Edge computing servers are deployed in mobile command vehicles or temporary server rooms at construction sites. As a concrete example, the server could be an industrial computer equipped with a graphics processing unit (GPU). This server maintains real-time data interaction with drones via low-latency wireless communication links such as 5G or microwave, acting as the upper-layer policy solver in a two-layer optimization evaluation computing engine, responsible for performing computationally intensive optimization solutions and policy generation tasks.

[0028] The cloud-based policy planning center can be a high-performance computing cluster deployed on a public or private cloud. It is responsible for storing massive amounts of historical inspection data, 3D spatial prior maps, and algorithm model libraries, and undertakes offline training and iterative updates of the models. During system operation, the cloud center also serves as supplementary computing power to the upper-layer policy solver, providing support for handling ultra-large-scale scenarios or performing global backtracking analysis.

[0029] At the software level, this system includes a data acquisition and topology construction module, a two-layer optimization evaluation calculation engine, and a closed-loop response and servo control module. The data acquisition and topology construction module runs on the airborne Jetson Xavier NX device of the UAV, responsible for collecting and preprocessing all sensor data streams. The two-layer optimization evaluation calculation engine adopts a cloud-edge decoupled asynchronous update mechanism. Its lower-layer feature estimator is compiled into a TensorRT accelerated model and then deployed on the airborne Jetson Xavier NX, while the upper-layer policy solver is deployed on an edge computing server. The closed-loop response and servo control module also runs on the airborne device, responsible for parsing upper-layer commands and generating control signals for the UAV's underlying flight control. The modules exchange data efficiently and reliably through communication protocols based on gRPC or ROS2 (Robot Operating System 2).

[0030] The safety hazard detection method described in this embodiment can be broken down into the following four core steps: Step 1: Data acquisition and dynamic scene topology graph construction.

[0031] Before carrying out the inspection task, the complete 3D Building Information Model (BIM) of the substation to be inspected is first imported into the system. This BIM model is typically stored in IFC or Revit format, accurately recording the geometric information, material properties, and semantic information of all permanent facilities within the substation. The system parses the model and converts it into a priori spatial map of energized bodies. This map specifically marks the location, shape, size, and corresponding voltage level of all high-voltage energized bodies. For example, each 220kV busbar, each 110kV porcelain insulator string, and each high-voltage bushing of the main transformer are represented in the map as nodes with 3D spatial coordinates and equivalent charge hazard coefficient attributes.

[0032] At the start of the mission, operators deploy the drone in a safe area of ​​the substation construction site and initiate its autonomous inspection program. The drone takes off according to a preset initial path or under manual remote control and begins continuous aerial hovering or circling observation of designated overlapping construction areas, such as the main hoisting area.

[0033] During flight, the data acquisition and topology construction modules synchronously collect and align multi-source heterogeneous data streams at high frequency. Specifically, the modules call the UAV's onboard SDK's application programming interface (API) to acquire 4K resolution continuous visual image sequences output by the gimbal camera at a frequency of 30 Hz, acceleration and angular velocity data output by the IMU at a frequency of 100 Hz, and the UAV's spatial pose sequence composed of latitude, longitude, altitude, and attitude angles output by the RTK-GNSS module at a frequency of 10 Hz. All data streams are timestamped with high precision to achieve accurate timing alignment subsequently.

[0034] Based on the acquired continuous visual image sequence and spatial pose sequence, and combined with the pre-loaded prior spatial map of charged bodies, the module begins to dynamically construct a dynamic scene topology graph representing the transient correlations of cross-construction. A topology graph is a data structure composed of nodes and edges, where nodes represent entities in the scene and edges represent relationships between entities.

[0035] Operational Equipment Nodes: The system employs a YOLOv8x target detection model fine-tuned with industry data to detect cranes in real-time within each frame of visual imagery. Once a crane is detected, a cascaded HR-Net high-resolution attitude estimation network is immediately invoked to accurately identify and locate multiple key structural points of the crane, such as the center of the tower base, the center of the slewing platform, the boom hinge point, the center of the boom end pulley, and the center of the hook. These two-dimensional pixel coordinates are back-projected into a three-dimensional world coordinate system by combining the UAV's current spatial pose and the camera intrinsic parameter matrix, forming the three-dimensional skeleton nodes of the crane. Each node is assigned kinematic characteristics such as velocity and acceleration.

[0036] Personnel nodes: The system also uses the YOLOv8x model to detect construction workers in the image and back-projects the geometric center point of their bounding box into three-dimensional spatial coordinates to form personnel nodes.

[0037] Charged body nodes: By aligning the UAV's 3D spatial pose with the prior spatial map of charged bodies, the system can query the 3D coordinates of all charged body nodes within the current field of view cone and import these nodes into the current dynamic scene topology map.

[0038] In actual construction sites, certain parts of a crane, such as the tower body or counterweight obscured by large transformers or buildings, may not be visible from the drone's current view, forming hidden nodes. To eliminate omissions caused by such partial obstruction, this embodiment incorporates steps including spatial alignment and state extrapolation compensation.

[0039] An algorithm based on bipartite graph matching is used to optimally match the set of visible pixel nodes of a 2D crane extracted from a visual image with the node set of a preloaded standard 3D crane model. This process achieves the optimal mapping from 2D observation to the 3D model by solving an affine transformation matrix.

[0040] After alignment, for latent nodes that exist in the 3D model but have no corresponding match in the 2D image, the system initiates a state extrapolation completion mechanism based on a Kalman filter. This mechanism presupposes the rigid body kinematics model of the crane as the state evolution model. For example, it is known that when the crane boom is performing pitch and luffing motion, the trajectory of the boom tip relative to the boom hinge point is a sphere with the hinge point as the center and the boom length as the radius. The Kalman filter uses this prior kinematic model as the prediction step and the observed positions of all visible nodes as the update step, thereby continuously and reliably extrapolating the spatial positions of the occluded latent nodes in time, ultimately completing the digital reconstruction of all nodes in the dynamic scene topology map.

[0041] The edges of the topological graph represent the spatiotemporal geometric relationships between nodes. The system calculates the Euclidean distance, relative velocity, relative acceleration, etc., between any two nodes and uses these quantified relationships as attributes of the edges. For example, the distance between the end node of a crane boom and the nearest high-voltage busbar node is a crucial edge.

[0042] At this point, a dynamic scene topology map that can comprehensively and dynamically represent the complex spatiotemporal relationships among equipment, personnel, and live conductors in the current construction scenario has been constructed and sent to the subsequent two-layer optimization evaluation calculation engine in the form of streaming data.

[0043] Step 2: Two-layer optimization assessment and spatiotemporal risk potential energy calculation.

[0044] To accurately quantify and assess the potential spatial intrusion risks inherent in dynamic scene topology maps, this embodiment constructs a novel two-layer optimization model. The core idea of ​​this model is to couple the optimization of the UAV's observation viewpoint with the optimization of the system's risk identification, and solve them jointly through a game theory-like framework, thereby outputting a continuous scalar function that comprehensively reflects various risk factors—the spatiotemporal risk potential energy index.

[0045] Specifically, this embodiment uses the spatiotemporal risk potential energy index Defined as a continuous scalar function consisting of a weighted sum of three key physical terms. This function is designed to comprehensively quantify the overall risk of critical moving parts, such as the end effector of a crane boom, intruding into restricted areas. Its specific mathematical model is as follows:

[0046] The first term: Electrical potential energy. This term simulates the physical concept of electric potential in an electrostatic field, used to characterize the risk of potential electric shock or discharge when lifting machinery approaches a charged body. Among these... It is an adjustable weighting factor used to balance the proportion of this item in the total risk. This is a set of all high-voltage charged body nodes located within the current region of interest, selected from the prior spatial map of charged bodies. Index i iterates through each charged body node in this set. This is an equivalent charge hazard factor obtained by looking up the voltage level of the i-th charged node. For example, a lookup table can be preset in the system, corresponding to 110kV. 220kV corresponds The higher the voltage level, the greater the risk factor. It is a three-dimensional spatial coordinate vector of the end of the boom or hook of the crane extracted from the dynamic scene topology map. It is the three-dimensional spatial coordinate vector of the i-th high-voltage charged body node. The square of the Euclidean distance between the two points was calculated. It can be seen that this value increases sharply as the distance between the lifting machinery and the charged body decreases, which is consistent with physical intuition. The second term: Motion Collision Term. This term is used to assess the probability of a physical collision between the lifting machinery and on-site personnel within a short future time window. It is its weighting coefficient. The lower limit of integration. This is the current inspection time. It is a preset length of the predicted future time domain, for example, set to 5 seconds, and t is the time integral variable that slides within this time window. This is a collision probability density function used to calculate the probability of kinematic interference between the lifting machinery and personnel. In this embodiment, the function is calculated by considering the velocity vector of the crane nodes. velocity vectors of personnel nodes Linear extrapolation is performed to predict the positions of the two objects at a future time t, and the probability of overlapping volume is calculated based on their respective size models. The third term is the visual information entropy term. This is a key innovation of this invention, incorporating the uncertainty of observation quality into the risk assessment. It is its weighting coefficient. It is a function for calculating information entropy. Its input... It is a visual feature observation matrix that quantifies the visual quality when extracting hazard-related features from the current UAV viewpoint. Specifically, This includes indicators such as image occlusion rate, scale distortion rate, and image contrast reduction caused by backlighting or dust for key targets like lifting machinery and personnel. When the drone's viewpoint is poor, causing targets to be occluded, too small, or blurry, the features output by the recognition network become unstable. The correlation values ​​of the matrix will increase, leading to its information entropy. This also increases accordingly. The introduction of this term means that when the situation is unclear, even if the physical risks calculated by the first two terms are not high, the overall spatiotemporal risk potential energy index will be significantly increased, thereby prompting the system to take action to eliminate this uncertainty.

[0047] In this embodiment, the two-layer optimization evaluation calculation engine solves the above model through asynchronous iteration of an upper-layer policy solver and a lower-layer feature estimator.

[0048] Specifically, the upper-layer policy solver is deployed on an edge computing server, solving the problem on a longer timescale, such as every 2 seconds or after processing a sliding window containing 60 frames of images. Its core objective is to better differentiate risks at a macro level, while simultaneously making drone flights more economical. Its upper-layer objective function... Defined as a joint objective that maximizes risk separability and minimizes observation cost:

[0049] In this function, The operator indicates that a set of observation viewpoint parameters for a UAV needs to be found. and an adaptive risk decision threshold This is to maximize the function value within the curly braces. It is a vector to be optimized, which can be specified as the yaw angle, pitch angle and distance of the UAV relative to the target. and These are, respectively, a positive sample set of high-risk space intrusion hazards mined from historical data and a negative sample set of safe construction standards. and These represent the assumptions that the drone is at the viewpoint. The spatiotemporal risk potential energy index is calculated for a known positive sample u and a negative sample v. It's the Hinge loss function, whose goal is to ensure that the risk value of a positive sample is at least one step higher than the risk value of a negative sample, determined by a threshold. The margin of decision . It is a penalty adjustment constant, and It is used to calculate the drone's current pose. Fly to the target viewpoint The cost function of energy and time required.

[0050] Since this is a complex non-convex optimization problem, this embodiment employs a derivative-free evolutionary algorithm, such as the Covariance Matrix Adaptive Evolutionary Strategy (CMA-ES), to solve the objective function. The solution yields a set of optimal observation viewpoints for a given period of time. A dynamic risk judgment threshold that adapts to the current environment. .

[0051] Specifically, the lower-level feature evaluator runs on an onboard Jetson Xavier NX device, receiving data from the upper layer. and As a constraint, its task is to fine-tune the weights of its internal risk identification network under a given observation strategy, making its predictions more accurate and smoother. Its lower-level objective function... Defined as the joint objective of minimizing classification loss and prediction smoothness:

[0052] In this function, The operator indicates that we need to update the weight set of the risk identification network. To minimize the function value within the curly braces. It is a standard cross-entropy loss function used to penalize the predicted spatiotemporal risk potential index. Weakly supervised real risk labels of input The difference between them. The second term is a key smoothing regularization term, where, These are weighting coefficients. The risk prediction value was calculated in relation to the observation viewpoint parameters. The partial derivative gradient vector is calculated. By minimizing the L2 norm squared of the gradient vector, the risk identification network can be effectively constrained, ensuring that its output risk value remains stable even with minor jitters or changes in the UAV's viewpoint, thus avoiding drastic jumps in the results. This lower-level optimization process is fine-tuned on the airborne device via online learning using standard stochastic gradient descent (SGD) or ADAM optimizers.

[0053] Throughout the two-level optimization process, the system must also comply with a series of preset constraints.

[0054] Electromagnetic exclusion zone spatial constraints: To ensure that the UAV and lifting machinery will never accidentally enter the high-voltage area, the system constructs a set of envelope cylinders with a three-dimensional repulsive force field for all high-voltage busbars and transformer equipment based on the prior spatial map of charged bodies. The radius of this envelope is set according to the safety regulations for the corresponding equipment voltage level. Then, at each step of the optimization solution, the system enforces the following set of dynamic collision avoidance and electric shock avoidance constraint inequalities:

[0055] in, It is a time window for predicting the future movement trajectories of drones and cranes. and These are the predicted three-dimensional spatial coordinates of the UAV's center of mass and the crane boom's end effector at a future time t. and This refers to the minimum safe clearance that drones and cranes must maintain, strictly defined according to electrical safety regulations. Any solution that violates this constraint will be discarded.

[0056] Specifically, the adaptive risk decision threshold output by the upper-level optimization. It is not a fixed value, but rather a dynamic degradation calculation based on the visual uncertainties at the site. This mechanism is key to coping with the harsh environment of construction sites. Its mapping formula is:

[0057] in, is a preset baseline static risk threshold under ideal lighting and unobstructed conditions, for example, 0.8. k is a threshold sensitivity gain constant greater than 0. This is the covariance matrix output by the risk identification network when extracting the hazard feature vector from the current viewpoint. This covariance matrix can be approximated using techniques such as Monte Carlo Dropout during network inference, and it directly reflects the model's uncertainty regarding the current input visual information. The trace of the covariance matrix quantifies the total variance of the eigenvectors across all dimensions. When there is dust, backlighting, or partial occlusion of the target, the quality of the visual input deteriorates, leading to an increase in the variance of the model's predictions, thus affecting the trace of the covariance matrix. This also increases. At this point, the subtraction term in the formula becomes larger, causing the final decision threshold to... The threshold is dynamically lowered. For example, it may be reduced from 0.8 to 0.6. This means that the system becomes more sensitive when visibility is poor, and even a relatively low spatiotemporal risk potential energy index may trigger an alarm, thereby greatly improving the system's sensitivity to hazard alarms under harsh operating conditions and avoiding fatal missed detections due to visual uncertainty.

[0058] Step 3: Closed-loop response and servo control.

[0059] When the two-layer optimization evaluation calculation engine calculates the real-time spatiotemporal risk potential energy index With adaptive risk decision threshold After that, the system enters the closed-loop response and servo control stage.

[0060] The closed-loop response and servo control module first... and Compare them.

[0061] like If the current state is deemed safe, the system will continue to perform the monitoring task.

[0062] like If a spatial intrusion risk is detected, it is determined that a spatial intrusion risk has occurred or is about to occur. The system will immediately generate a dynamic spatial intrusion level risk alarm work order. This work order will be based on... Exceeding The severity of the risk is categorized into different risk levels, such as alert, warning, and danger. Work orders include the time, location, the numbers of the equipment and live parts involved, a snapshot image of the risk, and a quantified risk value. This information is pushed in real-time via wireless network to the mobile terminals of on-site managers or the monitoring screen in the command center, accompanied by audible and visual alarms, prompting managers to take immediate intervention measures.

[0063] In some cases, even if the risk value has not yet exceeded the threshold, if the visual information entropy term in the risk composition... The abnormally high value indicates a serious problem with the current observation viewpoint, leading to low reliability of the risk assessment results. To address this issue, this embodiment designs an active viewpoint blind spot compensation control law for UAVs based on the risk potential energy gradient.

[0064] The specific parameterized control logic of this control law is as follows: The system calculates the spatiotemporal risk potential energy index. Regarding the current three-dimensional spatial coordinates of the drone The partial derivatives of this are used to obtain a risk potential gradient vector in three-dimensional space. Its mathematical expression is: Gradient vector The direction points to the risk potential energy index in three-dimensional space. The direction of fastest ascent. Since the increase in risk potential energy mainly comes from the increase in visual uncertainty, this direction also corresponds to the direction in which the degree of occlusion or blurriness intensifies the fastest.

[0065] The system determines whether the visual information entropy in the current state exceeds a preset tolerance threshold. If it does, it indicates that the problem of unclear vision has become severe enough to require immediate attention. At this point, the system will assign the gradient vector... Reverse the process and multiply by a velocity mapping gain coefficient. This generates a three-axis velocity command vector that is ultimately sent to the UAV's underlying flight control system. Its mathematical expression is: .

[0066] Among them, the speed command vector The physical meaning is: to drive the drone to perform adaptive maneuvers in the direction of the fastest decrease in the risk potential energy gradient. Understandably, the drone will automatically and intelligently fly to a new position that can most quickly reduce visual uncertainty. This position usually means better avoidance of obstructions or obtaining a better shooting angle and distance.

[0067] The drone will continue to execute this speed command until it moves to a new viewpoint where the visual information entropy term converges and falls below the tolerance threshold. This means the drone has found a position with a clear view. At this point, the active viewpoint filling process ends, and the system uses this high-quality viewpoint to recalculate the spatiotemporal risk potential energy index. It outputs a high-confidence spatial intrusion risk assessment result.

[0068] Through the steps described above, this embodiment constructs a complete closed loop from data acquisition, risk quantification, dynamic decision-making to flight control. It is no longer a passive observer, but an intelligent agent capable of interacting with the environment and proactively eliminating its own perceived uncertainties, thereby fundamentally improving the accuracy, real-time performance, and reliability of safety hazard detection at complex construction sites such as substations.

[0069] In summary, this embodiment achieves in-depth quantification of construction safety risks by introducing a comprehensive spatiotemporal risk potential energy index that combines electrical potential energy, motion collision, and visual information entropy, and constructing a novel dual-layer optimization model of UAV viewpoint and risk identification network. In particular, the dynamic degradation mechanism of adaptive risk judgment threshold and the UAV active viewpoint blind spot compensation control law based on risk potential energy gradient effectively overcome the limitations of traditional technologies in the face of adverse working conditions and visual occlusion. Its cloud-edge decoupled asynchronous computing architecture ensures millisecond-level real-time response while utilizing the powerful computing power of the edge cloud for complex strategy optimization, ultimately achieving efficient, accurate, and closed-loop intelligent management and control of dynamic spatial intrusion hazards during cross-construction in substations. This demonstrates extremely high practical application value and technological advancement.

[0070] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for detecting safety hazards at construction sites based on UAV inspection vision, characterized in that, The method includes the following steps: controlling a drone to conduct inspections at a substation construction site to simultaneously acquire continuous visual image sequences containing lifting machinery and electrical equipment, as well as a drone spatial pose sequence; based on the continuous visual image sequences and spatial pose sequences, combined with a pre-set prior spatial map of charged bodies, extracting the spatiotemporal geometric features of working equipment nodes, personnel nodes, and charged body nodes through a graph neural network to construct a dynamic scene topology map representing the transient correlation of cross-construction; targeting potential spatial intrusion hazards in the dynamic scene topology map, combining pre-set electromagnetic exclusion zone spatial constraints and visual observability constraints, constructing a two-layer optimization model to couple and optimize the drone observation parameters and risk identification network to output a spatiotemporal risk potential energy index representing the intrusion probability of equipment nodes and charged body nodes; based on the spatiotemporal risk potential energy index and the adaptive risk judgment threshold output by the two-layer optimization model, quantifying the dynamic spatial intrusion level of lifting machinery and charged bodies or personnel, generating detection results, and driving the drone to perform active viewpoint blind spot filling along the risk potential energy gradient to complete the closed-loop detection of safety hazards at the construction site.

2. The method for detecting safety hazards at construction sites based on UAV inspection vision as described in claim 1, characterized in that, The spatiotemporal risk potential energy index is a continuous scalar function, and its calculation formula is as follows: in, Indicators representing spatiotemporal risk potential energy; These represent the weighting coefficients of the electrical potential energy term, the motion collision term, and the visual information entropy term, respectively. The set represents the set of high-voltage charged body nodes contained in the prior spatial map of charged bodies; i represents the set. The index of the i-th high-voltage charged body node in the data; This represents the equivalent charge hazard factor corresponding to the voltage level of the i-th high-voltage charged body node; This represents the extracted three-dimensional spatial coordinate vector of the end of the crane boom; Represents the three-dimensional spatial coordinate vector of the i-th high-voltage charged body node; Indicates the current inspection time; This indicates the length of the predicted future time domain; t is the time integration variable; The collision probability density function represents the kinematic interference between the lifting machinery and the personnel. and Let represent the velocity vectors of the lifting machinery node and the personnel node at time t, respectively; This represents the function for calculating information entropy. This represents the visual feature observation matrix that includes occlusion rate and scale distortion rate when extracting hazard features from the current UAV observation viewpoint.

3. The method for detecting safety hazards at construction sites based on UAV inspection vision as described in claim 2, characterized in that, The upper-level optimization of the two-level optimization model takes maximizing risk separability and minimizing observation cost as joint objectives, and solves for the optimal observation viewpoint set and adaptive risk decision threshold for the UAV. Its upper-level objective function is defined as follows: in, This represents the objective function value of the upper-level optimization. This represents the set of UAV observation viewpoint parameters to be optimized, including yaw angle, pitch angle, and relative distance. This represents the adaptive risk decision threshold to be optimized. The set of positive samples representing historical high-risk space intrusion threats; u is the index of the positive samples in this set; This represents the set of negative samples for safe construction standards; v is the index of the negative sample in this set. and These represent the observation viewpoints respectively. Below, the spatiotemporal risk potential energy indicators output by positive sample u and negative sample v; Indicates the adaptive risk decision threshold The risk of constraints can be separated from marginal parameters; This represents the penalty adjustment constant coefficient; Indicates the drone's current pose Shift to target viewpoint The algebraic model function of time and energy consumption.

4. The method for detecting safety hazards at construction sites based on UAV inspection vision as described in claim 3, characterized in that, The lower-level optimization of the two-level optimization model is given and Under the constraints, the network weight set of the risk identification network is updated, and its lower-level objective function is defined as: in, This represents the objective function value of the lower-level optimization. This represents the set of all learnable tensor weights for the risk identification network; The cross-entropy loss function represents the risk state classification. Indicates the current network weight set The spatiotemporal risk potential energy index of the predicted output; This represents the true label vector of the weakly supervised risk input; Represents the weight coefficients of the smoothing regularization term; The spatiotemporal risk potential energy index representing the predicted output For the observation viewpoint The gradient vector of the partial derivatives.

5. The method for detecting safety hazards at construction sites based on UAV inspection vision as described in claim 4, characterized in that, The pre-defined electromagnetic exclusion zone spatial constraint is calculated using the following formula: in, This indicates the time window for predicting the motion trajectories of drones and cranes. and Let represent the three-dimensional spatial coordinates of the UAV's center of mass and the end of the crane boom at time t, respectively; Represents the set of spatial coordinate points of a three-dimensional envelope cylinder set; and These represent the minimum safe clearance settings for drones and cranes, respectively, in accordance with electrical safety regulations.

6. The method for detecting safety hazards at construction sites based on UAV inspection vision as described in claim 5, characterized in that, The adaptive risk judgment threshold is calculated based on the visual uncertainty caused by the harsh environment of the construction site, and its mapping formula is as follows: in, This represents the adaptive risk decision threshold that is optimized and solved by the upper layer and output to the decision module. This represents the preset baseline static risk threshold under ideal unobstructed lighting conditions; k represents the threshold sensitivity gain constant. This represents the covariance matrix output by the risk identification network when extracting hazard feature vectors under backlight or occlusion conditions at the current viewpoint; The operator represents the trace of the covariance matrix, used to quantify the total variance of the eigenvectors.

7. The method for detecting safety hazards at construction sites based on UAV inspection vision as described in claim 6, characterized in that, Driving the UAV to perform active viewpoint blind spot filling along the risk potential energy gradient specifically involves: calculating the spatiotemporal risk potential energy index. Regarding the current three-dimensional spatial coordinates of the drone The partial derivatives are used to obtain the risk potential gradient vector in three-dimensional space. The formula is Determine whether the visual information entropy in the current state is higher than the tolerance threshold. If so, then change the gradient vector. The vector is converted into a velocity command vector for the UAV flight control system through coefficient mapping. The formula is ;in, Represents the risk potential gradient vector; Represents the coordinate variable Operators for finding gradients in three-dimensional space; Indicators representing spatiotemporal risk potential energy; This represents the three-axis velocity command vector sent to the drone's underlying flight control system. This represents the velocity mapping gain coefficient, which is greater than 0.

8. The method for detecting safety hazards at construction sites based on UAV inspection vision as described in claim 7, characterized in that, A dynamic scene topology map representing the transient correlation of cross-construction is constructed. Specifically, a bipartite graph matching algorithm is used to perform affine transformation and mapping on the two-dimensional crane pixel nodes extracted from continuous visual image sequences and the preset standard three-dimensional model node set of cranes. Hidden crane node components that are obscured by transformers or walls in the visual view are identified and quantified. A state evolution model is constructed based on the material physical properties and rigid body kinematic equations of the crane's slewing and luffing mechanisms. The spatial position of the hidden node components is temporally extrapolated and completed by combining the state evolution model and Kalman filter, thus completing the digital reconstruction of hidden nodes in the dynamic scene topology map.

9. A construction site safety hazard detection system based on UAV inspection vision, used to implement the construction site safety hazard detection method based on UAV inspection vision as described in any one of claims 1 to 8, characterized in that, include: The data acquisition and topology construction module is used to collect multi-source perception streams from UAVs and align them with spatial prior maps to output a dynamic scene topology map containing cross-modal spatiotemporal features. The dual-layer optimization evaluation calculation engine, which embeds the joint mathematical derivation logic of the upper-layer policy solver and the lower-layer feature evaluator, is used to solve the spatiotemporal risk potential energy index and adaptive risk judgment threshold under multi-dimensional constraint space; the closed-loop response and servo control module is used to compare the index and threshold to generate dynamic space intrusion level hidden danger alarm work orders, and send reverse viewpoint blind spot filling instructions to the UAV flight control system through the gradient transformation model.

10. The construction site safety hazard detection system based on UAV inspection vision according to claim 9, characterized in that, The computing power allocation of the dual-layer optimization evaluation computing engine adopts a cloud-edge decoupled asynchronous update mechanism, which is as follows: The lower-level feature evaluator is compiled into a TensorRT model and deployed on the UAV's airborne edge computing node. It calculates the spatiotemporal risk potential energy index under the current viewpoint by forward inference at a set frequency, ensuring the real-time performance of space intrusion hazard detection. The upper-level policy solver is deployed on the construction site edge server or in the cloud. It asynchronously solves the new optimal observation viewpoint set and adaptive risk decision threshold with a multi-frame sliding window period and sends them down to the airborne edge computing node. Through edge-cloud collaboration, it eliminates the identification blind spots under harsh working conditions.

Images