Edge-computing-based nest-type unmanned aerial vehicle construction intelligent inspection and early warning method and system

Abstract

The application discloses a kind of based on edge computing's machine nest type unmanned aerial vehicle construction intelligent inspection and early warning method, system.Method includes: the three-end collaborative network of unmanned aerial vehicle end, edge computing base station and user end are built;With construction inspection point as node, build weighted directed graph, edge weight is the composite function of flight distance and data transmission efficiency, according to unmanned aerial vehicle state and real-time environment dynamically adjust edge weight, determine optimal traversal order by solving weighted traveling salesman problem;The unmanned aerial vehicle trajectory between adjacent inspection points, communication link selection and edge computing resource allocation are jointly modeled as sequence decision process, and the joint strategy is obtained by using deep reinforcement learning algorithm optimization;Unmanned aerial vehicle executes inspection according to optimal traversal order and joint strategy, and runs improved target detection model to identify anomaly.The joint optimization of path planning, resource scheduling and anomaly detection of unmanned aerial vehicle construction inspection is realized, and the inspection efficiency and real-time performance are improved.

Description

Technical Field

[0001] This invention belongs to the field of smart construction and drone inspection technology, specifically relating to a method and system for intelligent inspection and early warning of construction using nested drones based on edge computing. Background Technology

[0002] With the continuous expansion of infrastructure construction, construction inspection tasks are becoming increasingly heavy. Traditional manual inspection methods suffer from problems such as low efficiency, small coverage, and significant safety hazards, making it difficult to meet the real-time and refined requirements of modern construction management.

[0003] In recent years, drone technology has been gradually applied to construction inspection due to its advantages such as maneuverability and wide field of view. Drones can be equipped with high-definition cameras to automatically collect image data of the construction site and identify construction anomalies through target detection algorithms. However, existing drone inspection solutions still have the following technical shortcomings:

[0004] First, path planning and data processing are disconnected. Traditional solutions typically pre-define fixed flight paths, with the drone collecting data along these paths before transmitting it back to the cloud or ground server for processing. This approach does not consider the impact of data transmission efficiency on path selection, which can easily lead to transmission congestion and increased latency when the amount of data collected is large or the communication link quality is poor.

[0005] Second, resource scheduling lacks coordinated optimization. In existing solutions, the selection of UAV flight trajectories and communication links, as well as the allocation of edge computing resources, are usually decided separately, making it difficult to achieve global optimization. For example, a UAV may choose the path with the shortest flight distance, but if the communication conditions along that path are poor, data transmission will be slow, which will increase the overall mission completion latency.

[0006] Third, edge computing capabilities are underutilized. Although some solutions introduce edge computing nodes for local data processing, the scheduling of computing resources for these nodes is relatively static and fails to dynamically adjust based on the drone's location, channel status, and task queue, resulting in low utilization of computing resources.

[0007] Fourth, the target detection model is not well adapted to construction scenarios. The construction site environment is complex, and there are challenges such as small abnormal targets (such as cracks and settlement) having high similarity to the background. It is difficult for general target detection models to balance detection accuracy and real-time performance.

[0008] Therefore, how to achieve joint optimization of path planning, communication scheduling, computing resource allocation and anomaly detection in UAV construction inspection is a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0009] This invention aims to solve the technical problems of the disconnect and lack of joint optimization between path planning, communication scheduling, computing resource allocation and anomaly detection in existing UAV construction inspection, and provides a method and system for intelligent inspection and early warning of UAV construction based on edge computing.

[0010] In a first aspect, embodiments of this application provide a method for intelligent inspection and early warning of construction using nested unmanned aerial vehicles (UAVs) based on edge computing, the method comprising:

[0011] Construct a three-terminal collaborative network consisting of drone terminals, edge computing base stations, and user terminals;

[0012] A weighted directed graph is constructed with construction inspection points as nodes. The edge weights are composite functions of flight distance and data transmission efficiency. The edge weights are dynamically adjusted according to the UAV status and real-time environment. The optimal traversal order of each inspection point is determined by solving the weighted traveling salesman problem, thereby obtaining the sequential relationship between adjacent inspection points.

[0013] The drone trajectory, communication link selection, and edge computing resource allocation between adjacent inspection points are jointly modeled as a sequential decision-making process.

[0014] A decision model for the sequential decision-making process is trained using a deep reinforcement learning algorithm with the goal of minimizing task completion latency, and a joint strategy for UAV trajectory, base station scheduling, and computing resource allocation is obtained through optimization.

[0015] The drone flies along the optimal traversal order and collects construction data. A containerized microservice platform is deployed at the edge computing base station. The containerized microservice platform includes decoding services, inference services and encapsulation services, and processes the data collected by the drone concurrently in a multi-stage pipeline manner.

[0016] The object detection model is run in the inference service. The model improves its detection capability by introducing a spatial information enhancement module, an attention mechanism optimization module, and a multi-scale fusion structure.

[0017] The drone performs inspections according to the optimal traversal order and the joint strategy. The edge computing base station processes the collected data in real time through the multi-stage pipeline and runs the target detection model. When an anomaly is detected, it issues a warning to the user terminal through the communication protocol.

[0018] Secondly, embodiments of this application provide an edge computing-based intelligent inspection and early warning system for nested UAV construction, applied to the edge computing-based intelligent inspection and early warning method for nested UAV construction as described in the first aspect. The system includes:

[0019] A three-terminal collaborative network consisting of drone terminals, edge computing base stations, and user terminals;

[0020] The UAV terminal is used to construct a weighted directed graph with construction inspection points as nodes. The edge weight is a composite function of flight distance and data transmission efficiency. The edge weight is dynamically adjusted according to the UAV status and real-time environment. The optimal traversal order of each inspection point is determined by solving the weighted traveling salesman problem, thereby obtaining the sequential relationship between adjacent inspection points.

[0021] The drone terminal is also used to jointly model the drone trajectory, communication link selection and edge computing resource allocation between the adjacent inspection points as a sequential decision process;

[0022] The UAV terminal or the edge computing base station is used to train the decision model of the sequence decision process with the goal of minimizing task completion latency using a deep reinforcement learning algorithm, and optimizes the joint strategy of UAV trajectory, base station scheduling and computing resource allocation.

[0023] The drone is used to fly along the optimal traversal order and collect construction data. The edge computing base station is deployed with a containerized microservice platform, which includes decoding service, inference service and encapsulation service, and processes the data collected by the drone concurrently in a multi-stage pipeline manner.

[0024] The inference service runs an object detection model, which improves its detection capabilities by introducing a spatial information enhancement module, an attention mechanism optimization module, and a multi-scale fusion structure.

[0025] The drone terminal is used to perform inspections according to the optimal traversal order and the joint strategy. The edge computing base station is used to process the collected data in real time through the multi-stage pipeline and run the target detection model. When an anomaly is detected, it sends an early warning to the user terminal through the communication protocol.

[0026] Thirdly, embodiments of this application provide an electronic device, including:

[0027] processor;

[0028] Memory used to store processor-executable instructions;

[0029] The processor is configured to implement the method as described in the first aspect when executing the instructions.

[0030] Fourthly, embodiments of this application provide a computer-readable storage medium storing a program that instructs a device to perform the method described in the first aspect.

[0031] Compared with existing technologies, the advantages of this invention are as follows: By constructing a weighted directed graph with edge weights that are a composite function of flight distance and data transmission efficiency, and dynamically adjusting the weights according to the UAV battery status and real-time wind speed, joint optimization of flight efficiency and transmission efficiency is achieved in path planning. The UAV trajectory, communication link selection, and edge computing resource allocation are jointly modeled as a sequential decision-making process, and joint optimization is performed using deep reinforcement learning, avoiding the global suboptimal problem caused by separate decision-making. Multi-stage pipeline microservices are deployed at the edge computing base station to achieve concurrent processing of video decoding, model inference, and result encapsulation, reducing end-to-end processing latency. By introducing a spatial information enhancement module, an attention mechanism optimization module, and a multi-scale fusion structure to improve the target detection model, the detection accuracy of small-sized abnormal targets in complex construction scenarios is enhanced. Attached Figure Description

[0032] Figure 1 This is a schematic diagram of a method for intelligent inspection and early warning of construction using nested drones based on edge computing, provided as an embodiment of this application.

[0033] Figure 2 This is a schematic diagram of a method for dynamically adjusting edge weights provided in an embodiment of this application.

[0034] Figure 3 A schematic diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0035] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them.

[0036] It should be noted that in the embodiments of this application, "at least one" refers to one or more, and "more than one" refers to two or more. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in the specification of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application.

[0037] Based on the embodiments described in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0038] Example 1

[0039] Figure 1 This is a schematic diagram illustrating a method for intelligent inspection and early warning of nested unmanned aerial vehicle (UAV) construction based on edge computing, provided as an embodiment of this application. Figure 1As shown, a method for intelligent inspection and early warning of construction using nested unmanned aerial vehicles (UAVs) based on edge computing includes:

[0040] S1. Construct a three-terminal collaborative network consisting of drone terminals, edge computing base stations, and user terminals. Establish communication and collaboration relationships among the drones, edge computing base stations, and user terminals to enable bidirectional data flow, providing the infrastructure for subsequent data collection, processing, and control.

[0041] At the edge of a typical construction site, an edge computing base station is deployed, sharing the same physical location as or adjacent to a drone nest. This base station runs a lightweight containerized management platform (such as K3s). After the drone automatically takes off from its nest, it flies along a pre-set or dynamically planned inspection path. Its onboard high-definition camera continuously collects video streams of the construction site and transmits them back to the co-located edge computing base station in real time via 5G or Wi-Fi links. The containerized platform within the base station then schedules a decoding microservice to perform hardware decoding of the video stream. Subsequently, it calls an improved target detection model deployed in an inference container to analyze each frame of the image, identifying anomalies such as not wearing a safety helmet, unauthorized entry, or ground cracks. The detection results are processed into structured warning information by an encapsulation service. Control commands (such as hovering, zooming, or return-to-home) are transmitted back to the drone via the MQTT protocol, and simultaneously pushed to the user terminal screen in the remote monitoring room in real time via the WebSocket protocol. At the same time, flight path adjustment commands issued by the operator on the terminal are also sent to the drone through the same encrypted channel, thus forming a closed-loop control process from data acquisition, edge processing, anomaly warning, to command feedback. In this solution, due to the co-location of edge base stations and drone nests, the video transmission distance is extremely short, and the containerized microservices adopt pipelined concurrent processing. The overall latency from drone image acquisition to user receiving the early warning can be controlled within 120 milliseconds, achieving near real-time intelligent monitoring of construction safety.

[0042] S2. A weighted directed graph is constructed with construction inspection points as nodes. The edge weights are composite functions of flight distance and data transmission efficiency, and these edge weights are dynamically adjusted based on the UAV's status and real-time environment. The optimal traversal order of each inspection point is determined by solving the weighted traveling salesman problem, thus obtaining the sequential relationship between adjacent inspection points. Each inspection point on the construction site is abstracted as a node in the graph, and a comprehensive weight is calculated for the transfer path between every two nodes. This weight considers both flight distance and data transmission speed, and can be automatically adjusted based on real-time conditions such as the UAV's current battery level and wind speed. Finally, a mathematical optimization method is used to find the optimal order for visiting all inspection points and to clarify the sequential relationship between adjacent inspection points.

[0043] To simultaneously optimize flight efficiency and data transmission efficiency, edge weights are defined. as follows:

[0044] ,

[0045] in, To the inspection point arrive edge weights, The flight distance between the two points. The maximum distance between all inspection point pairs (pre-calculated based on the coordinates of all inspection points before the start of each task, and remains unchanged during the task, serving as a normalization factor). The estimated transmission rate of the current communication link. The maximum possible transmission rate is determined based on the communication protocol standard used; for example, 1Gbps for 5G links and 500Mbps for Wi-Fi 6. If the maximum transmission rate of the actual communication link cannot be obtained or is under extreme fluctuation, then... The default empirical value of 100Mbps is adopted, and the maximum value of this weight is limited to within 0.5 to avoid distortion of edge weights due to improper normalization. Let be the weighting coefficient, satisfying Dynamically switch according to task phase. During the data acquisition phase, set Prioritize flight efficiency; during the data transmission phase, set... Prioritizing transmission efficiency. When the drone's remaining battery power falls below a preset threshold, the system automatically increases its capacity. The value of makes the weighted traveling salesman problem more inclined to choose the path with the shorter flight distance. For headwind flight, the additional energy consumption is converted into an equivalent distance increment. Used for correction The corrected distance is denoted as Then substitute the values ​​into the formula above to calculate the edge weights.

[0046] Figure 2 This is a schematic diagram of a method for dynamically adjusting edge weights provided in an embodiment of this application. Figure 2 As shown, the dynamic adjustment of the edge weights specifically includes:

[0047] S2.1 During the UAV's inspection mission, the remaining battery power and current wind speed and direction data are acquired in real time. The UAV's onboard flight control system collects the remaining battery percentage data every second through the battery management module. At the same time, it obtains the wind speed and direction information of the current location through the onboard anemometer or by accessing the meteorological service interface. This data is reported to the edge computing base station in real time through the communication link as input parameters for dynamic adjustment of edge weights.

[0048] S2.2. Based on the remaining battery power, dynamically adjust the coefficient of the flight distance component in the edge weights so that when the battery power is low, the solution process of the weighted traveling salesman problem tends to choose the path with shorter flight distance. The edge computing base station presets a battery power threshold (30% in this embodiment, which can be configured by the user within the range of 20% to 40% depending on the drone model and construction environment). When the drone's remaining battery power is higher than this threshold, the weight coefficient α is set to the default value of 0.4, so that the path planning takes into account both flight efficiency and transmission efficiency; when the remaining battery power is lower than this threshold, the system gradually increases α to between 0.7 and 0.9 (the specific value is inversely proportional to the remaining battery power, the lower the battery power, the larger α), so that the weighted traveling salesman problem imposes a greater penalty on long-distance paths when solving, thereby guiding the drone to prioritize the path with shorter flight distance to ensure a safe return.

[0049] S2.3. Based on the current wind speed and direction, and considering the relative position between the UAV and the next target inspection point, calculate the additional energy consumption and time cost required for headwind flight, and convert this into an equivalent edge weight increment to correct the original edge weights. The system obtains the current wind direction and the UAV's flight direction to the next inspection point, calculates the angle between them, and determines whether it is a tailwind if the angle is less than 90 degrees, or a headwind if it is greater than 90 degrees. For headwind situations, calculate the additional flight time and energy consumption based on the wind speed and headwind angle, convert this additional cost into an equivalent distance increment according to a preset ratio, and add it to the original edge weights so that the path planning algorithm can automatically avoid unfavorable wind resistance segments.

[0050] To address low battery levels and headwind conditions, the edge weights are adjusted as follows:

[0051] ,

[0052] in, The corrected edge weights, This is the equivalent distance increment calculated under headwind conditions. This is the drag coefficient. This is a low battery penalty item. This is a low battery indicator function (0 or 1). It monitors battery level and wind speed and direction in real time, and indicates the angle between the wind direction and flight direction. >90° (i.e., headwind situation), calculate the equivalent distance increment. , The adjustment coefficient (in this embodiment, it is 0.5, and the unit is seconds·meters / second, which can be calibrated according to the drone model) Wind speed. A penalty is activated when battery level falls below a threshold. . Take the absolute value to ensure The value is non-negative, meaning the stronger the headwind and the smaller the angle between the wind direction and the flight direction (closer to a direct headwind), the larger the equivalent distance increment. Corrected edge weights Completely replace the original edge weight This is used to solve the weighted traveling salesman problem.

[0053] S2.4 The composite function is a linear weighted function, and its weighting coefficients switch between prioritizing flight efficiency and prioritizing data transmission efficiency according to the task stage. The system divides the inspection task into a data acquisition stage and a data transmission stage. In the data acquisition stage, a higher weighting coefficient for flight efficiency and a lower weighting coefficient for data transmission efficiency are set to ensure that the UAV quickly covers all inspection points. In the data transmission stage, the weighting coefficients are switched to increase the proportion of data transmission efficiency, so that path planning prioritizes flight segments with good channel quality and high transmission rates. The switching of weighting coefficients can be manually triggered by the user or automatically determined based on the amount of data remaining to be transmitted.

[0054] The process of determining the optimal traversal order of each inspection point by solving the weighted traveling salesman problem, thereby obtaining the sequential relationship between adjacent inspection points, includes:

[0055] S2.5 After determining the optimal traversal order of each inspection point by solving the weighted traveling salesman problem, the order is output as an ordered sequence of inspection points. Two adjacent points in this sequence constitute a flight segment that needs to be planned. The start point, end point, edge weight, and corresponding dynamic adjustment parameters of each flight segment are used as input constraints for the next step of the UAV trajectory and resource joint optimization model.

[0056] The edge computing base station runs a weighted traveling salesman problem (TSP) algorithm, aiming to minimize the sum of all edge weights to calculate the optimal order for visiting all construction inspection points. For example, the output sequence might be: Inspection Point A → Inspection Point C → Inspection Point B → Inspection Point D. The system extracts each pair of adjacent points in this sequence as an independent flight segment: Segment 1 is A to C, Segment 2 is C to B, and Segment 3 is B to D. For each segment, the system records its starting point coordinates, ending point coordinates, the combined edge weights calculated in steps S2.2 and S2.3, and parameters such as the remaining battery power threshold and wind speed compensation used during dynamic adjustment. This data is packaged into a structured data format and used as input constraints for subsequent deep reinforcement learning decision models. These constraints are used to further optimize the UAV's specific flight trajectory, communication link selection, and edge computing resource allocation scheme between adjacent inspection points, thus forming a two-level decision architecture from macro-path planning to micro-trajectory and resource joint optimization.

[0057] S3. Jointly model the drone trajectory, communication link selection, and edge computing resource allocation between adjacent inspection points as a sequential decision process. For the flight process between two adjacent inspection points, the three issues of drone flight route selection, which base station to communicate with, and how much computing resources the base station allocates to process the data are merged into a unified mathematical decision model for overall optimization. This step specifically includes:

[0058] The drone trajectory, communication link selection, and edge computing resource allocation between adjacent inspection points are jointly modeled as a Markov decision process. A multi-dimensional state space is defined, including drone state, channel state, base station load, and task queue, as well as a composite action space containing drone control commands and resource allocation commands.

[0059] Edge computing base stations model the flight and processing process between two adjacent inspection points as a Markov decision process, which consists of four parts: state space, action space, state transition probabilities, and reward function. These four elements are defined below.

[0060] The state space is defined as a set of multi-dimensional vectors, each containing information in four dimensions: UAV state (including current 3D position coordinates, flight speed vector, remaining battery power, and the amount of data yet to be transmitted), channel state (including signal strength, signal-to-noise ratio, and estimated transmission rate with each base station), base station load (including CPU utilization, memory usage, and task queue length of each base station), and task queue state (including the number of video frames yet to be decoded, inferred, and encapsulated, and their priorities). At each decision-making moment, the system collects the above data and concatenates them into a state vector, which serves as the input to the decision model.

[0061] In this scheme, the state transition probabilities are not explicitly modeled, but are implicitly learned through interaction sampling between a deep reinforcement learning algorithm and the environment. The design of the reward function will be explained in detail in subsequent step S4.2.

[0062] For the action space, the system defines two types of actions: UAV control commands, including the next moment's flight speed adjustment, heading angle change, and altitude adjustment; and resource allocation commands, including selecting which edge computing base station to use for data transmission, allocating the transmission power level to the task, and requesting the edge computing base station to allocate computing resource quotas for the task. These actions are jointly output by the decision model, forming a composite action vector.

[0063] The system models the state transition probabilities of the Markov decision process as unknown but deterministic environmental dynamics. Instead of explicitly calculating the transition probability matrix, it learns by collecting sample data through interaction with the real environment. The reward function is designed as a negative of the task completion delay; that is, the shorter the total time for completing the current flight segment and data processing, the higher the reward value. The deep reinforcement learning algorithm gradually learns the optimal mapping strategy from state to action by continuously trying different actions and observing reward feedback, ultimately achieving a joint control decision that minimizes the overall task completion delay in any state.

[0064] S4. A decision model for the sequential decision-making process is trained using a deep reinforcement learning algorithm with the goal of minimizing task completion latency, optimizing the joint strategy for UAV trajectory, base station scheduling, and computing resource allocation. With the objective of minimizing the total time from the start of data transmission to the final completion of processing, deep reinforcement learning is used to train the above decision model, resulting in a comprehensive control scheme that can simultaneously determine how the UAV flies, which base station the data is transmitted to, and how much computing power the base station uses. This step includes:

[0065] S4.1 The task completion latency is defined as the end-to-end time from the start of UAV transmission of inspection data to the user receiving the final warning or analysis result. The system uses the moment when the UAV's onboard camera begins to collect and send the first frame of image as the timing start point, and the moment when the detection result or warning pop-up corresponding to that frame of image is successfully rendered on the user's display as the timing end point. The time difference between the two is defined as the task completion latency. This latency includes video encoding latency, wireless transmission latency, edge base station decoding latency, model inference latency, result encapsulation latency, and network backhaul latency. The system records timestamps at each processing stage through data embedding for subsequent calculation and optimization.

[0066] S4.2 Design a reward function, the core of which is the negative value of the task completion delay, and introduce penalty terms to constrain the safety of the UAV trajectory, the stability of the communication link, and the fairness of computing resource allocation. The base term of the reward function is set as the negative of the task completion delay, that is, the shorter the delay, the higher the reward. Based on this, the system introduces three penalty terms: the first is a safety penalty, which gives a negative reward when the minimum distance between the UAV flight trajectory and ground obstacles or construction machinery is lower than a preset safety threshold; the second is a communication penalty, which gives a negative reward when the UAV frequently switches between different base stations or is in a signal dead zone for a long time; the third is a fairness penalty, which gives a negative reward when the task queue length of a certain edge computing base station exceeds a set multiple of the average queue length of other base stations. The final reward value is the base term minus the weighted sum of the penalty terms.

[0067] S4.3. A deep deterministic policy gradient algorithm based on an actor-critic framework is used to train the Markov decision process offline. The actor network generates composite actions based on the current multidimensional state, and the critic network evaluates the value of the state-action pairs. During the offline training phase, the system collects state transition samples of the UAV in a simulation environment or historical flight missions to construct an experience replay pool. The actor network takes the current state vector as input and outputs a continuous action vector, including the flight speed increment, heading angle change, target base station number, and computational resource request. The critic network takes the state vector and the action vector output by the actor network as input and outputs a scalar value score to evaluate the expected cumulative reward of performing the action in that state. During training, the system randomly samples mini-batch samples from the experience replay pool, updates the critic network parameters by minimizing the critic network's prediction error, and simultaneously updates the actor network parameters along the direction of increasing the critic score using the policy gradient method.

[0068] S4.4 The trained deep neural network model is deployed in the collaborative decision-making module of the UAV and edge computing base station to generate a joint optimal strategy for UAV flight trajectory, communication link selection, and edge computing resource allocation in real time online. After offline training, the system exports the model parameters of the actor network and deploys them in the UAV's onboard decision-making module and the collaborative scheduling module of the edge computing base station, respectively. In actual inspection tasks, the UAV or edge computing base station collects current state information in real time and inputs it into the deployed actor network. The network forward inferences and outputs the optimal composite action. The UAV adjusts its flight trajectory in real time according to the output speed increment and heading angle change, and establishes a communication link according to the output base station number and transmission power level. The edge computing base station allocates corresponding CPU or GPU computing power according to the output computing resource request amount. The entire process is repeated in each decision cycle to achieve online real-time joint optimal decision-making.

[0069] S5. The drone flies along the optimal traversal sequence and collects construction data. A containerized microservice platform is deployed at the edge computing base station. The containerized microservice platform includes decoding services, inference services, and packaging services, and processes the data collected by the drone concurrently in a multi-stage pipeline manner. The drone flies in the planned sequence and takes pictures of the construction site along the way. At the same time, a lightweight container runtime environment is built on the ground edge computing base station. This environment contains three independent service modules: data decoding, intelligent analysis, and result packaging, and processes the data transmitted back by the drone in parallel in a pipeline operation manner.

[0070] The total processing delay of the pipeline can be modeled (basic formula) as follows:

[0071] ,

[0072] in, To account for the processing delay of each single frame at each stage, For single-frame decoding latency, For single-frame inference latency, This refers to the single-frame encapsulation delay. Total number of frames This introduces additional overhead for pipeline startup and shutdown. By decoupling the three phases through message middleware (Redis Streams / Kafka), each service can scale up and down independently, with overall throughput limited by the slowest phase.

[0073] Considering the impact of service parallelism and message queue backlog, a refined formula can be used to calculate the total processing latency:

[0074] ,

[0075] in, These represent the processing rates of the decoding service, inference service, and encapsulation service, respectively. The minimum value among the three is the bottleneck processing rate of the entire pipeline. The relationship between these three rates and the single-frame processing latency is as follows: , , , Therefore, when the single-frame decoding latency is 8ms, the corresponding processing rate is 125 frames per second; the inference latency of 25ms corresponds to 40 frames per second; and the encapsulation latency of 2ms corresponds to 500 frames per second. At this point, the bottleneck is the inference service. Let 1, 2, and 3 represent the three message queues: decoding to inference, inference to encapsulation, and encapsulation to output, respectively. For the first The number of backlogged frames in each queue. The average waiting time for each frame in this queue. This covers fixed overhead for network transmission, data serialization, etc. It should be noted that the basic formula... It is suitable for scenarios where services are executed serially, while the refined formula, by introducing queue backlog and parallel processing rate, more accurately reflects the actual situation of pipeline concurrent processing.

[0076] Suppose a certain inspection task involves a drone collecting 300 frames of video. =300), average latency at each stage: decoding 8ms, inference 25ms, encapsulation 2ms; bottleneck is in inference (25ms / frame → 40 frames / second), queue backlog: decoding → inference queue 15 frames, waiting time approximately 120ms. Calculation: The system can expand the inference service to a replica of the edge computing-based nested drone construction intelligent inspection and early warning system, increasing the bottleneck rate to approximately 160 frames / second and reducing the total latency to approximately 2 seconds.

[0077] The containerized microservice platform deployed on the edge computing base station adopts a three-stage pipeline architecture, which is implemented as follows:

[0078] Decoding Service: This service runs in an independent container, integrating a hardware decoder driver. It subscribes to video stream topics transmitted from drones via a message middleware. When a drone pushes a compressed video stream in H.264 or H.265 format to the base station via a 5G or Wi-Fi link, the decoding service automatically pulls the data stream, calls the GPU or dedicated decoding chip for real-time hardware decoding, and restores the compressed video to original image frames in YUV or RGB format. After decoding, the service republishes the image frames and their timestamps to the inference topic in the message middleware for downstream services to consume.

[0079] Inference Service: This service runs in a standalone container, deploying an improved RETAC-YOLO11 object detection model that has been trained and quantized offline. The inference service subscribes to inference topics through a message broker, simultaneously fetching multiple image frames using a multi-replica concurrent approach, and leveraging the GPU for parallel inference analysis. After each image frame undergoes forward propagation by the model, it outputs the detected anomalous object category, confidence score, and bounding box coordinates. The inference service associates the detection results with the corresponding image frame identifier and publishes them to the unpackaged topic in the message broker. By adjusting the number of replicas of the inference service, dynamic scaling to match the decoding service's frame rate can be achieved.

[0080] Encapsulation Service: This service runs in an independent container, subscribing to the topics to be encapsulated through a message middleware and receiving detection results output by the inference service. The encapsulation service filters and aggregates the detection results according to preset business rules, such as merging detection results of the same target in multiple consecutive frames into a single warning event, or removing false detections based on a confidence threshold. After processing, the encapsulation service converts the results into structured messages in JSON or Protobuf format, and classifies them into warning events or control commands based on content type, publishing them to the warning and control topics of the message middleware respectively.

[0081] Asynchronous communication mechanism: The three services do not directly call each other, but instead decouple and communicate asynchronously through a message middleware. The message middleware uses a lightweight deployment of Redis Streams or Kafka, and each service independently maintains its consumption progress. When the decoding service processes quickly, a certain number of image frames will accumulate in the inference topic, and the inference service will pull and consume them according to its own processing capacity; the same applies when the inference service processes quickly. This pipelined concurrent processing method allows the three services to run simultaneously without blocking each other, and the overall throughput is limited by the slowest link rather than the sum of all links, thus significantly reducing end-to-end processing latency.

[0082] S6. Run the target detection model in the inference service. The model improves its detection capability by introducing a spatial information enhancement module, an attention mechanism optimization module, and a multi-scale fusion structure. Deploy an improved intelligent recognition model in the analysis service module of the edge base station. This model improves its ability to identify construction anomalies by adding components specifically for extracting spatial structure information, optimizing the attention allocation mechanism, and fusing features at different scales.

[0083] The target detection model running in the inference service of the edge computing base station is an improved RETAC-YOLO11 model, which has the following three improvements on the standard YOLO11:

[0084] First, a RetBlock module is introduced to enhance spatial structure perception. The RetBlock module is embedded in the feature extraction layer of the model's backbone network. This module adds a parallel Manhattan self-attention computation branch to the traditional convolution operation. When the input feature map passes through the RetBlock, local texture features are extracted through standard convolution, and the spatial distance weights between each location in the feature map and other locations are calculated through the Manhattan self-attention mechanism. Specifically, for any two locations on the feature map... and Its Manhattan distance is defined as ,in and These are the coordinates of two locations on the feature map. The Manhattan self-attention mechanism uses the negative value of this distance, normalized by softmax, as the attention weight; that is, locations closer to each other receive higher attention scores. Compared to standard self-attention based on dot products, Manhattan self-attention does not depend on the similarity of feature values, but is determined solely by spatial location. Therefore, it can capture the spatial structure information of the target more purely, and its computational complexity is also reduced. Down to ,in These represent the height, width, and number of channels of the feature map, respectively.

[0085] The output feature maps of the two branches are concatenated along the channel dimension and then fused through pointwise convolution. In this way, the model can more effectively perceive the spatial structure information of targets in construction images, such as edges, corners, and orientations. It has a significant improvement in identifying targets that are sensitive to spatial structure, such as the posture of people not wearing safety helmets and illegal splicing of scaffolding.

[0086] Second, the Agent Attention mechanism is integrated to optimize feature focusing. An Agent Attention module is inserted at the network neck, between the feature pyramid and the path aggregation network. This module introduces a set of learnable surrogate tokens, the number of which is much smaller than the total number of pixels in the input feature map. The module's computation process borrows from the Transformer's attention mechanism, specifically in two steps: First, the learnable surrogate tokens are used as query vectors to aggregate global context information from the key and value vectors of the original feature map, forming a compact surrogate feature representation. Since the number of surrogate tokens is much smaller than the number of pixels in the feature map, the computational complexity of this step is significantly reduced. Second, each position in the original feature map is used as a query vector, with the surrogate token as the key, to broadcast global context information from the surrogate features generated in the first step. Through this surrogate interaction mechanism, the module can automatically learn to allocate higher attention weights to areas in the image that may contain potentially dangerous targets. Examples include the edges of high-altitude operations, the edges of deep foundation pits, and areas with excessively high stacks of material. This allows the model to prioritize these high-risk areas with limited computational resources, improving the detection rate of dangerous targets.

[0087] Third, the ContextGuide framework is fused to improve the accuracy of small object detection. At the network head, before the detection head, the ContextGuide framework is fused. This framework receives multi-scale feature maps from the neck network, including high-resolution, low-semantic shallow feature maps and low-resolution, high-semantic deep feature maps. The ContextGuide framework first performs global average pooling on each layer's feature maps to extract global context descriptors for each channel dimension. These descriptors are then non-linearly transformed through point convolutional layers to generate channel attention weights for each layer's feature maps. Finally, the generated channel attention weights are multiplied back onto the original feature maps of the corresponding layers to obtain enhanced feature maps for each layer. Based on this, the ContextGuide framework introduces a set of learnable fusion weight parameters γ1, γ2, and γ3 (corresponding to shallow, mid-layer, and deep feature maps, respectively), which are used to weight and sum the enhanced multi-layer feature maps to obtain a fused feature map. This fused feature map is then fed into the detection head for object classification and bounding box regression. Through this context-guided weighted fusion method, the model can dynamically adjust the contribution ratio of features at different scales according to the overall content of the image, which can effectively improve the detection accuracy for small-sized abnormal targets at construction sites (such as small ground cracks, loose bolts, and detached protective nets).

[0088] S7. The UAV performs inspections according to the optimal traversal order and the joint strategy. The edge computing base station processes the collected data in real time through the multi-stage pipeline and runs the target detection model. When an anomaly is detected, an early warning is sent to the user terminal through the communication protocol. The UAV performs the actual inspection task according to the planned order and the optimized control scheme. The edge base station processes the data transmitted back by the UAV in real time through the pipeline and runs the recognition model. When an anomaly is detected, an alarm is sent to the user terminal through the network.

[0089] Edge computing base stations and user terminals interact with each other through a hybrid communication protocol, specifically a combination of WebSocket and MQTT protocols, with each protocol handling data transmission tasks with different real-time requirements.

[0090] A persistent WebSocket connection is established between the edge computing base station and the user terminal. When the user terminal opens the monitoring page, it actively initiates a WebSocket handshake request to the edge computing base station. After successful authentication, a full-duplex persistent connection channel is maintained. On this channel, the edge computing base station continuously pushes real-time status data of the drone at millisecond intervals, including current location coordinates, flight speed, remaining battery power, the currently executing flight segment number, and a low-latency preview of the real-time video stream. The user terminal can obtain a near real-time monitoring experience without repeatedly initiating HTTP requests. At the same time, this WebSocket channel also serves as a high-priority push channel for early warning events. When the inference service detects a construction anomaly, the encapsulation service immediately generates a structured early warning event, including the anomaly type, confidence level, target bounding box coordinates, and the timestamp of the corresponding video frame, and pushes it directly to the user terminal through the established WebSocket connection. Upon receiving the event, the user terminal alerts the operator through pop-ups, sound prompts, or highlighted markers.

[0091] The edge computing base station also runs an MQTT broker service, with client-side applications subscribing to relevant topics as MQTT clients. For data with lower real-time requirements, the MQTT protocol is used for transmission. Structured early warning event messages are pushed via WebSocket and simultaneously published to the MQTT early warning archive topic for persistent storage and post-event tracing. In addition, system telemetry data such as drone flight logs, edge computing base station CPU and GPU utilization, memory usage, network bandwidth consumption, and processing latency statistics for each microservice are periodically uploaded in batches via the MQTT protocol. Client-side applications can subscribe to these topics as needed to obtain the system's health status. The MQTT protocol uses QoS level 1 to ensure messages are delivered at least once and supports a will message mechanism, automatically notifying all subscribed clients when the edge computing base station experiences an abnormal disconnection.

[0092] When the inference service detects an anomaly, the encapsulation service simultaneously generates two identical alert event messages. One is pushed immediately via a WebSocket channel without waiting for acknowledgment, employing a send-and-discard mode to reduce latency; if the WebSocket connection is abnormally disconnected, the message will be discarded without retrying, as real-time alerts prioritize timeliness over reliability. The other is sent to a persistent topic via the MQTT protocol, configured with a retention policy of 7 days for automatic deletion after expiration. The MQTT message uses QoS level 1 to ensure at least one delivery, used for post-event tracing and offline analysis. This design ensures that the end-to-end latency of real-time alerts is unaffected by the message acknowledgment mechanism, while critical alert data is reliably stored.

[0093] Example 2

[0094] This application provides an edge computing-based nested UAV construction intelligent inspection and early warning system, applied to the method described in Embodiment 1, including:

[0095] A three-terminal collaborative network consisting of drone terminals, edge computing base stations, and user terminals;

[0096] The UAV terminal is used to construct a weighted directed graph with construction inspection points as nodes. The edge weight is a composite function of flight distance and data transmission efficiency. The edge weight is dynamically adjusted according to the UAV status and real-time environment. The optimal traversal order of each inspection point is determined by solving the weighted traveling salesman problem, thereby obtaining the sequential relationship between adjacent inspection points.

[0097] The drone terminal is also used to jointly model the drone trajectory, communication link selection and edge computing resource allocation between the adjacent inspection points as a sequential decision process;

[0098] The UAV terminal or the edge computing base station is used to train the decision model of the sequence decision process with the goal of minimizing task completion latency using a deep reinforcement learning algorithm, and optimizes the joint strategy of UAV trajectory, base station scheduling and computing resource allocation.

[0099] The drone is used to fly along the optimal traversal order and collect construction data. The edge computing base station is deployed with a containerized microservice platform, which includes decoding service, inference service and encapsulation service, and processes the data collected by the drone concurrently in a multi-stage pipeline manner.

[0100] The inference service runs an object detection model, which improves its detection capabilities by introducing a spatial information enhancement module, an attention mechanism optimization module, and a multi-scale fusion structure.

[0101] The drone terminal is used to perform inspections according to the optimal traversal order and the joint strategy. The edge computing base station is used to process the collected data in real time through the multi-stage pipeline and run the target detection model. When an anomaly is detected, it sends an early warning to the user terminal through the communication protocol.

[0102] Figure 3 This is an electronic device provided in one embodiment of this application. For example... Figure 3 As shown, the electronic device includes at least the following components: processor 301 and memory 300, communication interface 303, and bus 302.

[0103] In this embodiment of the application, memory 300 is used to store executable instructions of processor 301, which, when configured to execute instructions, implements the method as described in the first aspect.

[0104] In embodiments of this application, a computer-readable storage medium includes instructions that instruct a device to perform the method as described in the first aspect. For example, the instructions instruct the device to perform... Figure 1 The method is shown in the process steps.

[0105] In one embodiment of this application, the program operating in the electronic device may be a program that controls a central processing unit (CPU) or similar device to achieve the functions of the above-described embodiments of the present invention (a program that enables the computer to function). Information processed by these systems is then temporarily stored in random access memory (RAM) during processing, and subsequently stored in various ROMs such as read-only memory (FlashROM) and hard disk drives (HDDs), and read, corrected, and written by the CPU as needed.

[0106] It should be noted that a portion of the electronic device described above can also be implemented using a computer. In this case, the program for implementing the control function can be recorded on a computer-readable recording medium, and the program recorded on the recording medium can be read into the computer and executed.

[0107] It should be noted that the computer mentioned here refers to a computer built into an electronic device, employing hardware including an operating system and peripheral devices. Furthermore, computer-readable recording media refers to removable media such as floppy disks, magneto-optical disks, ROMs, and CD-ROMs, as well as storage systems such as hard drives built into the computer.

[0108] Furthermore, computer-readable recording media can include: media that dynamically stores programs for short periods of time, such as communication lines used when transmitting programs via networks like the Internet or communication lines like telephone lines; and media that store programs for fixed periods of time, such as volatile memory inside a computer that serves as a server or client in this case. In addition, the aforementioned program can be a program used to implement the above-mentioned functions, or it can be a program that can implement the above-mentioned functions by combining them with programs already recorded in the computer.

[0109] Furthermore, the electronic device in the above embodiments can also be implemented as an assembly (system group) composed of multiple systems. Each system constituting the system group can possess some or all of the functions or functional blocks of the electronic device in the above embodiments. As a system group, it is sufficient to have all the functions or functional blocks of the electronic device.

[0110] Those skilled in the art should recognize that the above embodiments are only used to illustrate this application and are not intended to limit this application. Any appropriate changes and variations made to the above embodiments within the essential spirit and scope of this application fall within the scope of protection claimed in this application.

Claims

1. A method for intelligent inspection and early warning of construction using nested unmanned aerial vehicles (UAVs) based on edge computing, characterized in that, Includes the following steps: Construct a three-terminal collaborative network consisting of drone terminals, edge computing base stations, and user terminals; A weighted directed graph is constructed with construction inspection points as nodes. The edge weights are composite functions of flight distance and data transmission efficiency. The edge weights are dynamically adjusted according to the UAV status and real-time environment. The optimal traversal order of each inspection point is determined by solving the weighted traveling salesman problem, thereby obtaining the sequential relationship between adjacent inspection points. The drone trajectory, communication link selection, and edge computing resource allocation between adjacent inspection points are jointly modeled as a sequential decision-making process. A decision model for the sequential decision-making process is trained using a deep reinforcement learning algorithm with the goal of minimizing task completion latency, and a joint strategy for UAV trajectory, base station scheduling, and computing resource allocation is obtained through optimization. The drone flies along the optimal traversal order and collects construction data. A containerized microservice platform is deployed at the edge computing base station. The containerized microservice platform includes decoding services, inference services and encapsulation services, and processes the data collected by the drone concurrently in a multi-stage pipeline manner. The object detection model is run in the inference service. The model improves its detection capability by introducing a spatial information enhancement module, an attention mechanism optimization module, and a multi-scale fusion structure. The drone performs inspections according to the optimal traversal order and the joint strategy. The edge computing base station processes the collected data in real time through the multi-stage pipeline and runs the target detection model. When an anomaly is detected, it issues a warning to the user terminal through the communication protocol.

2. The method according to claim 1, characterized in that, The edge computing base station is deployed at the construction site and co-located with the drone nest. The edge computing base station is equipped with a containerized microservice platform to carry and schedule microservices. The three-terminal collaborative network collects data from the drone terminal and transmits it to the edge computing base station. The edge computing base station processes the data and generates control commands to send back to the drone terminal. The user terminal receives the detection results and issues control commands through the communication protocol, forming a real-time feedback closed loop of collection, processing and control.

3. The method according to claim 1, characterized in that, The dynamic adjustment of the edge weights specifically includes: During the inspection mission, the drone's remaining battery power and current wind speed and direction data are acquired in real time. Based on the remaining battery power, the coefficient of the flight distance component in the edge weights is dynamically adjusted so that when the battery power is low, the solution process of the weighted traveling salesman problem tends to choose the path with the shorter flight distance. Based on the current wind speed and direction, and combined with the relative position between the drone and the next target inspection point, the additional energy consumption and time cost required for flying against the wind are calculated, and converted into an equivalent edge weight increment to correct the original edge weight. The composite function is a linear weighted function, and its weighting coefficients switch between prioritizing flight efficiency and prioritizing data transmission efficiency according to the mission phase.

4. The method according to claim 1, characterized in that, The process of determining the optimal traversal order of each inspection point by solving the weighted traveling salesman problem, thereby obtaining the sequential relationship between adjacent inspection points, includes: After determining the optimal traversal order of each inspection point by solving the weighted traveling salesman problem, the order is output as an ordered sequence of inspection points. Two adjacent points in this sequence constitute a flight segment that needs to be planned. The start point, end point, edge weight, and corresponding dynamic adjustment parameters of each flight segment are used as input constraints for the next step of the UAV trajectory and resource joint optimization model.

5. The method according to claim 1, characterized in that, The process of jointly modeling the drone trajectory, communication link selection, and edge computing resource allocation between adjacent inspection points as a sequential decision-making process specifically includes: The drone trajectory, communication link selection, and edge computing resource allocation between adjacent inspection points are jointly modeled as a Markov decision process. A multi-dimensional state space is defined, including drone state, channel state, base station load, and task queue, as well as a composite action space containing drone control commands and resource allocation commands.

6. The method according to claim 5, characterized in that, The decision model for the sequential decision-making process is trained using a deep reinforcement learning algorithm with the objective of minimizing task completion latency, and the joint strategy for UAV trajectory, base station scheduling, and computing resource allocation is optimized, including: The task completion delay is defined as the end-to-end time from when the drone starts transmitting inspection data to when the user receives the final warning or analysis result. The reward function is designed with the core being the negative value of the task completion delay, and a penalty term is introduced to constrain the safety of the drone trajectory, the stability of the communication link, and the fairness of the allocation of computing resources. A deep deterministic policy gradient algorithm based on an actor-critic framework is used to train the Markov decision process offline, wherein the actor network generates compound actions based on the current multidimensional state, and the critic network evaluates the value of the state-action pair; The trained deep neural network model is deployed in the collaborative decision-making module of the UAV and edge computing base station to generate a joint optimal strategy for UAV flight trajectory, communication link selection and edge computing resource allocation in real time online.

7. The method according to claim 1, characterized in that, The containerized microservice platform adopts a three-stage pipeline, including: Decoding service, used for real-time hardware decoding of video streams transmitted back by drones; The inference service deploys a target detection model based on the improved RETAC-YOLO11 to perform concurrent inference analysis on the decoded video frames; The encapsulation service receives the detection results output by the inference service and encapsulates them into structured early warning events or control commands according to preset rules. The decoding service, inference service, and encapsulation service are deployed as independent containerized microservices and communicate asynchronously through a message middleware, enabling the video stream to be processed concurrently in a pipeline manner between the three stages, thereby reducing end-to-end processing latency.

8. The method according to claim 7, characterized in that, The target detection model running in the inference service is an improved RETAC-YOLO11 model, the improvements including: A RetBlock module is introduced into the feature extraction layer of the backbone network to enhance the model's ability to perceive the spatial structure information of construction targets in images; An Agent Attention mechanism module is integrated into the network neck to optimize feature map computation and information flow, prioritizing focus on potentially dangerous target regions; By integrating the ContextGuide framework into the network header and aggregating multi-scale feature maps, the detection accuracy for small-sized construction targets is improved.

9. The method according to claim 1, characterized in that, The edge computing base station and the user terminal interact through a hybrid communication protocol; Among them, the WebSocket protocol is used to establish a real-time, two-way, long-term connection for user-side monitoring of the drone inspection status, as well as the real-time push of early warning events; Among them, the MQTT protocol is used to transmit structured early warning event messages and non-real-time system logs and device telemetry data; When the inference service detects an anomaly, the encapsulation service generates an alert event and pushes it to the user terminal via the WebSocket channel first.

10. A nested unmanned aerial vehicle (UAV) construction intelligent inspection and early warning system based on edge computing, applied to the method described in any one of claims 1 to 9, characterized in that, The system includes: A three-terminal collaborative network consisting of drone terminals, edge computing base stations, and user terminals; The UAV terminal is used to construct a weighted directed graph with construction inspection points as nodes. The edge weight is a composite function of flight distance and data transmission efficiency. The edge weight is dynamically adjusted according to the UAV status and real-time environment. The optimal traversal order of each inspection point is determined by solving the weighted traveling salesman problem, thereby obtaining the sequential relationship between adjacent inspection points. The drone terminal is also used to jointly model the drone trajectory, communication link selection and edge computing resource allocation between the adjacent inspection points as a sequential decision process; The UAV terminal or the edge computing base station is used to train the decision model of the sequence decision process with the goal of minimizing task completion latency using a deep reinforcement learning algorithm, and optimizes the joint strategy of UAV trajectory, base station scheduling and computing resource allocation. The drone is used to fly along the optimal traversal order and collect construction data. The edge computing base station is deployed with a containerized microservice platform, which includes decoding service, inference service and encapsulation service, and processes the data collected by the drone concurrently in a multi-stage pipeline manner. The inference service runs an object detection model, which improves its detection capabilities by introducing a spatial information enhancement module, an attention mechanism optimization module, and a multi-scale fusion structure. The drone terminal is used to perform inspections according to the optimal traversal order and the joint strategy. The edge computing base station is used to process the collected data in real time through the multi-stage pipeline and run the target detection model. When an anomaly is detected, it sends an early warning to the user terminal through the communication protocol.

Images