Highway construction unmanned aerial vehicle hidden danger intelligent identification method and system based on spatial logic judgment

Abstract

The invention discloses a road construction unmanned aerial vehicle potential safety hazard intelligent identification method and system based on multi-class identification and pixel-level space logic. A construction site image is collected through an unmanned aerial vehicle, and a plurality of elements such as a foundation pit, hoisting equipment, paving equipment, a protective fence and operating personnel are identified by using a target detection model; a space judgment area is dynamically constructed in a two-dimensional image coordinate system based on a pixel geometrical relationship, and camera calibration or physical size conversion is not needed; logical judgment is conducted through the inclusion relation between the target center point and the judgment area, and three kinds of potential safety hazards of foundation pit protection deficiency, hoisting warning deficiency and personnel invasion are automatically recognized. According to the method, detection and logical reasoning are separated, dependence on hidden danger samples is remarkably reduced, real-time processing over 20 fps is achieved on edge equipment, and the method has the advantages of being easy to deploy, high in generalization ability and high in accuracy and is suitable for the field of highway construction safety monitoring.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and computer vision technology, and in particular to a method and system for intelligent identification of potential hazards by highway construction drones based on spatial logic judgment. Background Technology

[0002] Currently, safety hazard identification at highway construction sites mainly relies on manual inspections or single-target detection methods based on video surveillance. Manual inspections are labor-intensive, inefficient, and limited by perspective and environment, easily leading to missed detections and misjudgments. While video surveillance or image recognition technologies can achieve partial automation, most methods only detect single-category targets (such as personnel, machinery, or protective facilities), lacking comprehensive analysis of spatial relationships between multiple categories of objects. Existing deep learning-based hazard identification methods typically rely on large-scale sample data with hazard annotations, obtaining the identification model through end-to-end training. However, this approach suffers from difficulties in obtaining hazard samples, poor model transferability, and insufficient adaptability to new scenarios.

[0003] Meanwhile, traditional methods often rely on precise size measurement or 3D reconstruction, requiring camera calibration and scale recovery. The system implementation is complex and costly, making it unsuitable for rapid identification tasks in drone aerial photography scenarios.

[0004] In summary, existing technologies generally have the following shortcomings:

[0005] 1. Hazard identification relies on manual or single-type inspections, making it impossible to make comprehensive judgments on multiple targets;

[0006] 2. The identification model relies on a large number of potential hazard samples, resulting in weak generalization ability;

[0007] 3. It requires precise physical dimensions or 3D modeling, making application deployment complex and unsuitable for rapid identification in drone aerial photography scenarios. Summary of the Invention

[0008] The purpose of this invention is to provide a method and system for intelligent identification of potential hazards in highway construction drones based on spatial logic judgment, thereby solving the aforementioned problems existing in the prior art.

[0009] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0010] A method for intelligent identification of potential hazards using unmanned aerial vehicles (UAVs) in highway construction based on spatial logic judgment includes the following steps:

[0011] S1. Collect inspection images of highway construction sites using drones equipped with visible light cameras;

[0012] S2. Use a deep learning target detection model to identify and locate multiple construction elements in the inspection images, and obtain the bounding box coordinate information of each type of target; construction elements include at least foundation pits, lifting and hoisting equipment, paving equipment, protective fences and workers;

[0013] S3. In the two-dimensional image pixel coordinate system, the corresponding spatial determination region is constructed by calculating the bounding box coordinate information through geometric formulas. The calculation process is entirely based on pixel-level geometric relationships and does not require camera parameter calibration or physical size conversion.

[0014] S4. Based on the pixel-level inclusion relationship between the spatial determination area and the target center point, perform standardized spatial logic judgment to automatically identify safety hazards; the judgment logic includes judgment of the integrity of the foundation pit protection, judgment of the lack of hoisting and lifting warning, and judgment of the intrusion of paving operation personnel.

[0015] S5. Output the hazard identification results, including hazard location markings, hazard category information, and structured data.

[0016] Furthermore, in step S3, the spatial determination region is calculated and constructed using the following geometric formula, and the spatial determination region construction process achieves a processing speed of over 20fps on the edge computing device:

[0017] For the foundation pit target, let its bounding box width be w and height be h. Then, expand it by 0.1 times the width or height pixel distance in four directions to form four rectangular boundary band regions. The upper boundary band has a vertical range of [y_min-0.1h, y_min+0.1h] and a horizontal range of [x_min, x_max]. The lower boundary band has a vertical range of [y_max-0.1h, y_max+0.1h] and a horizontal range of [x_min, x_max]. The left boundary band has a horizontal range of [x_min-0.1w, x_min+0.1w] and a vertical range of [y_min, y_max]. The right boundary band has a horizontal range of [x_max-0.1w, x_max+0.1w] and a vertical range of [y_min, y_max].

[0018] For a target lifting and hoisting equipment, let the coordinates of its bounding box center point be (x_c, y_c), where x_c = (x_min + x_max) / 2, y_c = (y_min + y_max) / 2, and the maximum side length r = max(w, h). Then, a circular warning area is constructed with the center point as the center and r as the radius. This area is defined by the equation (x - x_c)² + (y - y_c)² ≤ r².

[0019] For the paving equipment target, let its bounding box coordinates be (x_min, y_min, x_max, y_max). Then, expand outward by one size to form a rectangular danger buffer zone with coordinates (x_min-w, y_min-h, x_max+w, y_max+h).

[0020] Furthermore, in step S4, the spatial logic judgment adopts a unified judgment criterion: extract the pixel coordinates of the center point of the target bounding box, and determine whether the center point falls within the corresponding spatial judgment area. If it falls within the area, it is determined that the spatial association condition is met.

[0021] Furthermore, the deep learning target detection model is trained using a construction site image dataset, and the dataset is labeled with at least five categories: foundation pit, lifting and hoisting equipment, paving equipment, protective fencing, and workers. The target detection model is deployed separately from the backend logic judgment module, and the detection layer does not rely on hazard samples for training.

[0022] Furthermore, the hazard identification results are presented using a combination of visual annotation and JSON structured data output. The JSON data fields include at least the image number, hazard category, target bounding box coordinates, and hazard location coordinates.

[0023] In another embodiment, a highway construction drone hazard identification system based on spatial logic judgment includes:

[0024] The drone image acquisition module is used to acquire high-definition inspection images of the construction site;

[0025] The multi-category target recognition module, based on a deep learning target detection model, identifies and locates foundation pits, lifting equipment, paving equipment, protective fences, and workers in inspection images, and outputs bounding box coordinate information.

[0026] The pixel-level spatial range calculation module is connected to the target multi-category recognition module through a data interface. It is used to calculate and construct the spatial determination region in the two-dimensional image coordinate system based on the bounding box coordinate information and geometric formulas. The calculation process is entirely based on pixel-level geometric operations and does not require camera parameter calibration or physical size conversion.

[0027] The spatial logical relationship judgment module performs judgments on the integrity of foundation pit protection, the absence of hoisting and lifting warning, and the intrusion of paving workers, based on the pixel-level inclusion relationship between the target center point and the spatial judgment area.

[0028] The hazard labeling and output module is used to label images of identified hazards and output structured data.

[0029] Furthermore, the system is deployed on GPU servers or edge computing devices, supporting video stream processing or batch image analysis, and the spatial determination region calculation process achieves a processing speed of over 20fps on edge computing devices.

[0030] Furthermore, the pixel-level spatial range calculation module includes a foundation pit boundary zone calculation unit, a hoisting warning area calculation unit, and a paving buffer area calculation unit. Each calculation unit operates independently and outputs standardized geometric area description data through geometric formulas.

[0031] Furthermore, the system supports loading preset combinations of spatial judgment parameters for different construction scenarios. These parameter combinations are determined based on construction safety specifications and include the boundary zone expansion ratio, the warning zone radius coefficient, and the buffer zone expansion multiple.

[0032] Furthermore, the drone image acquisition module adopts a multi-rotor drone, equipped with a high-definition camera and gimbal stabilization structure, and supports flight path planning, fixed-point hovering and automatic shooting functions. The acquired image data is sent to the back-end processing platform wirelessly.

[0033] The beneficial effects of this invention are:

[0034] This invention achieves intelligent hazard identification without the need for precise dimensional measurement by uniformly identifying multiple key elements at a construction site (such as foundation pits, lifting equipment, paving equipment, protective fencing, and workers) and establishing spatial logical relationships within the image pixel space. Compared with existing technologies that rely on single-class detection or end-to-end hazard classification, this invention has the following significant advantages:

[0035] (1) The identification mechanism is more universal and scalable.

[0036] Traditional methods rely on a large number of training samples labeled with potential hazards. This invention only requires regular target detection data to complete the identification task. It uses logical rules to infer potential hazards, which significantly reduces the amount of sample labeling. The model can be quickly transferred to different construction scenarios without retraining the hazard model.

[0037] (2) The computing structure is simplified and the system deployment is more efficient.

[0038] This invention replaces complex 3D modeling or precise calibration with "target detection + spatial logic judgment". The recognition logic relies only on pixel-level geometric relationships, reducing the algorithm complexity compared to traditional 3D reconstruction methods. It can achieve real-time inference on edge computing devices, meeting the actual needs of UAVs to recognize while flying.

[0039] (3) Improved accuracy and consistency in hazard identification

[0040] By employing unified spatial logic rules (such as the principle that the target's center point falls within the judgment area), this invention achieves standardized judgment of potential hazards such as lack of protection, lack of vigilance, and personnel intrusion. The logical judgment results are stable and highly consistent.

[0041] (4) Reduce manual intervention and improve inspection efficiency

[0042] Traditionally, a safety officer can inspect about 3 to 5 high-risk work sites per day on average. This invention combines drone inspection with automatic identification, which can process more than 2,000 images in the same amount of time, improving inspection efficiency and realizing automatic hazard labeling and report generation, reducing the workload of manual review.

[0043] (5) The system is highly adaptable and has low application costs.

[0044] This invention does not rely on specific camera parameters or scene scale, and is applicable to different brands of drones and various construction types (bridges, roadbeds, road paving, etc.). It can be quickly integrated into existing safety inspection platforms or mobile systems, and has good engineering feasibility and promotional value. Attached Figure Description

[0045] Figure 1 This is a flowchart of the intelligent identification method for hidden dangers in highway construction drones based on spatial logic judgment according to the present invention;

[0046] Figure 2 This is a schematic diagram of the system structure of the present invention. Detailed Implementation

[0047] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0048] Reference Figure 1 The method for intelligent identification of potential hazards using UAVs in highway construction, based on spatial logic judgment, includes the following steps:

[0049] S1. Collect inspection images of highway construction sites using drones equipped with visible light cameras;

[0050] This step specifically involves using a drone equipped with a visible light camera to collect inspection images of the highway construction site. This process is completed by the drone's image acquisition module. Specifically, a highly stable multi-rotor drone is selected as the flight platform, equipped with a high-definition visible light camera. This camera features a three-axis gimbal stabilization structure, supporting image stabilization and autofocus functions to ensure clear, unobstructed image data is acquired during flight.

[0051] Regarding flight path design, targeted flight paths were developed based on the characteristics of different work areas at the construction site. For the foundation pit work area, a vertical overhead shooting method was used, with the camera's optical axis perpendicular to the ground, capturing the entire foundation pit from top to bottom. For the lifting and hoisting work area, an oblique or surround shooting method was used, with the camera's optical axis at a certain angle to the horizontal plane, capturing the spatial relationship between the hoisting equipment and the surrounding environment from a side perspective. For the paver work area, the same oblique or surround shooting method was used to ensure clear coverage of the machinery's operating range and personnel activity areas. Flight path planning must ensure comprehensive image acquisition coverage of all high-risk areas, avoiding blind spots and dead zones.

[0052] Data acquisition supports two working modes: fixed-frame photography mode or continuous video recording mode. In fixed-frame photography mode, the drone hovers at a preset waypoint and triggers the shutter; in continuous video recording mode, the drone flies at a constant speed along the flight path and continuously records video. The acquired raw data includes visible light images or video frames, and simultaneously records the time information, GPS location information, and drone attitude information corresponding to each frame as metadata, providing a spatiotemporal reference for subsequent processing.

[0053] The entire data collection process is automated through the drone's flight path planning, hovering, and automatic shooting functions, ensuring the standardization and repeatability of data collection and meeting the needs of large-scale, high-efficiency inspections at construction sites.

[0054] S2. Use a deep learning target detection model to identify and locate multiple construction elements in the inspection images, and obtain the bounding box coordinate information of each type of target; construction elements include at least foundation pits, lifting and hoisting equipment, paving equipment, protective fences and workers;

[0055] This step specifically involves using a deep learning object detection model to identify and locate multiple construction elements in the inspection images. This process is implemented by a multi-class object recognition module. Specifically, the YOLOv8 deep learning object detection model is selected as the core algorithm. This model possesses excellent detection accuracy and real-time performance, making it suitable for the simultaneous detection of multiple targets in complex construction site environments.

[0056] During the model training phase, a pre-labeled construction scene image dataset was used. The dataset's labeling categories strictly correspond to the key safety elements of the construction site, including at least the following five categories: foundation pit area, lifting equipment (such as truck cranes and crawler cranes), paving equipment, protective fencing facilities, and on-site workers. The annotation information for each target category includes a category label, precise location bounding box coordinates, and a confidence score. To improve the model's robustness to different lighting conditions, weather conditions, and shooting angles, various image enhancement strategies were introduced during training, including but not limited to data augmentation methods such as color perturbation, random rotation, and multi-scale scaling.

[0057] The model inference stage is deployed on GPU servers or edge computing devices with AI acceleration capabilities, supporting the processing of real-time video streams or offline image sequences. For an input single-frame inspection image, the model detects multiple types of targets in the image in parallel, outputting structured information for each identified target, including target category identifier, bounding box pixel coordinates (top-left and bottom-right corner coordinates), and detection confidence score. The bounding box coordinates accurately describe the target's position and extent in the image, providing the basic input for subsequent spatial logic judgments. The system supports the simultaneous identification of multiple objects of different categories in the same image, achieving true multi-class, multi-target detection capability.

[0058] The module maintains an inference speed of over 20fps on edge computing devices, meeting the application requirements of real-time inspection at construction sites. The model uses pre-trained weights for initialization and is fine-tuned for construction scenarios, ensuring high detection accuracy while reducing training data requirements. The detection layer and backend logic judgment module are deployed separately; the detection process does not rely on any hazard samples for training, performing only general target detection tasks, significantly reducing the system's dependence on labeled data.

[0059] The entire identification and positioning process is automated and requires no manual intervention. The detection results are transmitted to the downstream processing module through a standardized data interface, ensuring the accuracy and timeliness of data transmission.

[0060] S3. In the two-dimensional image pixel coordinate system, the corresponding spatial determination region is constructed by calculating the bounding box coordinate information through geometric formulas. The calculation process is entirely based on pixel-level geometric relationships and does not require camera parameter calibration or physical size conversion.

[0061] Specifically, this step involves calculating and constructing the corresponding spatial determination region in the two-dimensional image pixel coordinate system using geometric formulas based on the bounding box coordinate information. This process is implemented by the pixel-level spatial range calculation module. The construction of all spatial determination regions is entirely based on pixel-level geometric relationships. The entire calculation process does not require camera parameter calibration or physical size conversion, making it suitable for construction scenarios with varying scales and no need for precise measurements under drone aerial photography perspectives.

[0062] For excavation pit targets, based on the identified excavation pit boundary box, rectangular extension areas are constructed in the four directions (up, down, left, and right) as boundary bands. The specific construction method is as follows: based on the width and height values ​​of the boundary box, extend outwards by one-tenth of the original dimension both vertically and horizontally, forming four strip-shaped judgment areas of equal width. The vertical range of the upper boundary is determined by extending the original upper border upwards and downwards by one-tenth of the frame height, while the horizontal range remains consistent with the original frame width; the lower boundary band uses the same extension method applied to the lower border; the left boundary band extends horizontally to the left and right by one-tenth of the frame width, with the vertical range the same as the original frame height; the right boundary band extends horizontally based on the right side frame. These four boundary bands together constitute the safety protection judgment range around the excavation pit, used for subsequent assessment of the integrity of the protective fencing distribution.

[0063] For targets such as lifting and hoisting equipment, a circular warning zone is constructed based on the geometric center of the equipment's bounding box. First, the average coordinates of the two diagonal vertices of the bounding box are calculated to obtain the center point position. Then, the larger value between the width and height of the bounding box is taken as the radius parameter. A circular area is drawn with this center point as the center and the aforementioned radius as the distance metric. This circular area covers the maximum impact range of the hoisting operation and is used to determine the setting of surrounding warning facilities.

[0064] For targets such as paving equipment, a rectangular hazard buffer zone is formed by expanding outwards at equal intervals from the equipment's boundary frame. Specifically, the width and height of the boundary frame are used as expansion steps, adding a distance equal to the frame's own dimensions in each of the four directions of the original frame to obtain a larger rectangular area. This area represents the hazardous impact range of the machinery operation and is used to identify whether personnel have entered the hazardous work zone.

[0065] The construction of the three types of spatial determination regions described above all employs fixed geometric calculation formulas. All region coordinates are defined within the pixel coordinate system, without any physical unit conversion or 3D spatial reconstruction. The algorithms are low in complexity, making them suitable for real-time processing on edge computing devices and meeting the high-efficiency requirements of UAV inspection. The calculation results are output as standardized geometric region descriptions, providing a unified input interface for subsequent logical judgments.

[0066] S4. Based on the pixel-level inclusion relationship between the spatial determination area and the target center point, perform standardized spatial logic judgment to automatically identify safety hazards; the judgment logic includes judgment of the integrity of the foundation pit protection, judgment of the lack of hoisting and lifting warning, and judgment of the intrusion of paving operation personnel.

[0067] This step specifically involves: based on the pixel-level inclusion relationship between the spatial determination region and the target center point, performing standardized spatial logic judgment. This process is implemented by the spatial logic relationship judgment module. All judgment logic is completed within the two-dimensional image pixel coordinate system, replacing complex three-dimensional modeling or physical distance measurement with simple geometric inclusion operations, thus achieving efficient and unified automatic identification of potential hazards.

[0068] The judgment logic adopts a unified judgment criterion: extract the pixel coordinates of the center point of each target bounding box, and determine whether the center point falls within the corresponding spatial judgment region. If the coordinates of the target's center point are within the coordinate range of the judgment region, the target is considered to have a spatial relationship with the judgment region, satisfying the logical judgment condition; if the center point coordinates are outside the judgment region, the two are considered not to have a spatial relationship. This criterion applies to all types of spatial relationship judgments, ensuring the simplicity and consistency of logical operations.

[0069] For each identified foundation pit target, the integrity assessment checks the four boundary zones constructed around it sequentially. The system traverses the center points of all protective enclosure targets, determining whether each enclosure center point falls within any of the foundation pit's boundary zones. The number of enclosure center points contained in each boundary zone is counted. If each of the four boundary zones contains at least one enclosure center point, the foundation pit is considered intact and free of hidden dangers. If no enclosure center point is detected within any boundary zone, the foundation pit is considered to have a hidden danger of protection deficiency, and the specific direction of the missing boundary is recorded.

[0070] The system for determining the absence of hoisting and lifting safety precautions constructs a circular safety zone around each identified hoisting and lifting equipment target, using its center point as the reference and its maximum side length as the radius. The system iterates through the center points of all protective fencing targets, determining if any fencing center point falls within this circular safety zone. If no fencing center point is detected within the circular zone, the hoisting equipment is deemed to lack operational safety precautions, indicating a potential safety hazard in the hoisting area. If at least one fencing center point falls within the zone, the safety precautions are deemed adequate.

[0071] For each identified paving equipment target, the system constructs a rectangular hazard buffer zone that expands outward by one size based on its bounding box. The system iterates through the center points of all personnel targets, determining whether each person's center point falls within this buffer zone. If any personnel's center point is detected inside the buffer zone, a potential conflict is identified as a personnel entering the hazardous machinery operating area; if all personnel's center points are outside the buffer zone, the working environment is deemed safe.

[0072] The above judgment process is executed sequentially without interference. All calculations are based on pixel-level coordinate comparisons, resulting in high computational efficiency. When any judgment logic meets the hazard triggering condition, the system immediately generates the corresponding hazard judgment result, including the hazard type, associated target ID, and specific location information of the unmet condition, and passes the result to the downstream annotation module for visualization. If none of the judgments trigger the hazard condition, a normal status indicator is output, indicating that no safety hazards were found in the current inspection area.

[0073] S5. Output the hazard identification results, including hazard location markings, hazard category information, and structured data.

[0074] This step specifically involves the following: The hazard identification results are output by the hazard labeling and output module. This module performs image-level visual labeling and structured data output for the identified safety hazards. First, a highlighted label box is overlaid on the original inspection image to mark the location of the hazard. The shape of the label box is either rectangular or circular depending on the hazard type, and orange is used as the warning color to distinguish it from ordinary targets. For the hazard of missing foundation pit protection, an orange rectangular boundary band is drawn at the boundary of the unprotected foundation pit; for the hazard of missing hoisting and lifting warning, an orange circle is drawn at the edge of the circular warning area around the hoisting equipment; for the hazard of paving personnel intrusion, an orange rectangular box is drawn at the location of the intruder. A text label is attached above or near each label box, clearly indicating the hazard category. The label content includes type descriptions such as "missing foundation pit protection," "no warning in hoisting area," or "personnel intrusion in paving area." The text label is directly overlaid on the corresponding area of ​​the image to ensure that construction personnel can intuitively understand the risk point.

[0075] While visually labeling, the system generates a structured data file to support digital management. This file, encoded in JSON format, contains detailed attribute information for each hazard. The fields include at least: a unique identifier for the image, a hazard category label, coordinates of the target bounding box, coordinates of the hazard location, a detection confidence score, and metadata such as a timestamp. The JSON file and the labeled image are associated with the same filename for easy subsequent querying and archiving. The structured data can be transmitted wirelessly to a backend web visualization platform. After parsing the JSON content, the platform dynamically renders the hazard locations on the monitoring interface, allowing managers to view hazard distribution on an electronic map, filter specific types of risk events, replay historical inspection records, and generate statistical reports.

[0076] The entire output process is automated, requiring no manual intervention. Annotation and coding operations are completed within milliseconds, ensuring the continuity and efficiency of the inspection process. The output results can be stored locally as project archives or pushed in real time to a remote monitoring center for centralized management, meeting the application needs of construction projects of different sizes.

[0077] Furthermore, in step S3, the spatial determination region is calculated and constructed using the following geometric formula, and the spatial determination region construction process achieves a processing speed of over 20fps on the edge computing device:

[0078] For the foundation pit target, let its bounding box width be w and height be h. Then, expand it by 0.1 times the width or height pixel distance in four directions to form four rectangular boundary band regions. The upper boundary band has a vertical range of [y_min-0.1h, y_min+0.1h] and a horizontal range of [x_min, x_max]. The lower boundary band has a vertical range of [y_max-0.1h, y_max+0.1h] and a horizontal range of [x_min, x_max]. The left boundary band has a horizontal range of [x_min-0.1w, x_min+0.1w] and a vertical range of [y_min, y_max]. The right boundary band has a horizontal range of [x_max-0.1w, x_max+0.1w] and a vertical range of [y_min, y_max].

[0079] For a target lifting and hoisting equipment, let the coordinates of its bounding box center point be (x_c, y_c), where x_c = (x_min + x_max) / 2, y_c = (y_min + y_max) / 2, and the maximum side length r = max(w, h). Then, a circular warning area is constructed with the center point as the center and r as the radius. This area is defined by the equation (x - x_c)² + (y - y_c)² ≤ r².

[0080] For the paving equipment target, let its bounding box coordinates be (x_min, y_min, x_max, y_max). Then, expand outward by one size to form a rectangular danger buffer zone with coordinates (x_min-w, y_min-h, x_max+w, y_max+h).

[0081] Furthermore, in step S4, the spatial logic judgment adopts a unified judgment criterion: extract the pixel coordinates of the center point of the target bounding box, and determine whether the center point falls within the corresponding spatial judgment area. If it falls within the area, it is determined that the spatial association condition is met.

[0082] Furthermore, the deep learning target detection model is trained using a construction site image dataset, and the dataset is labeled with at least five categories: foundation pit, lifting and hoisting equipment, paving equipment, protective fencing, and workers. The target detection model is deployed separately from the backend logic judgment module, and the detection layer does not rely on hazard samples for training.

[0083] Furthermore, the hazard identification results are presented using a combination of visual annotation and JSON structured data output. The JSON data fields include at least the image number, hazard category, target bounding box coordinates, and hazard location coordinates.

[0084] Reference Figure 2 The system shown is an intelligent hazard identification system for highway construction drones based on spatial logic judgment, comprising:

[0085] The drone image acquisition module is used to acquire high-definition inspection images of the construction site;

[0086] The multi-category target recognition module, based on a deep learning target detection model, identifies and locates foundation pits, lifting equipment, paving equipment, protective fences, and workers in inspection images, and outputs bounding box coordinate information.

[0087] The pixel-level spatial range calculation module is connected to the target multi-category recognition module through a data interface. It is used to calculate and construct the spatial determination region in the two-dimensional image coordinate system based on the bounding box coordinate information and geometric formulas. The calculation process is entirely based on pixel-level geometric operations and does not require camera parameter calibration or physical size conversion.

[0088] The spatial logical relationship judgment module performs judgments on the integrity of foundation pit protection, the absence of hoisting and lifting warning, and the intrusion of paving workers, based on the pixel-level inclusion relationship between the target center point and the spatial judgment area.

[0089] The hazard labeling and output module is used to label images of identified hazards and output structured data.

[0090] In this embodiment, the UAV image acquisition module serves as the system's front-end data acquisition unit. It utilizes a multi-rotor UAV as its flight platform and is equipped with a high-definition visible light camera. The camera features a three-axis gimbal stabilization structure, supporting image stabilization and autofocus to ensure clear, unobstructed inspection images even under flight vibrations and environmental interference. The module supports two flight path settings: a vertical overhead view for the foundation pit work area, where the camera's optical axis is perpendicular to the ground to capture the overall view; and a slanted or surround view for the lifting and paving work area, where the camera's optical axis forms an angle with the horizontal plane to capture lateral spatial relationships. Data acquisition supports both frame-by-frame photography and continuous video recording. The acquired images are in general image or video formats, and the time information, GPS location information, and UAV attitude information for each frame are recorded simultaneously as metadata, which is then wirelessly transmitted to the back-end processing platform. This module features flight path planning, hovering, and automatic shooting functions, automating and standardizing the inspection process and ensuring data acquisition covers all high-risk areas of the construction site.

[0091] Multi-class Target Recognition Module: This module, built upon a deep learning target detection model, serves as the core of the system's intelligent perception, responsible for identifying and locating multiple construction elements in inspection images. The module employs the YOLOv8 target detection model as its basic algorithm framework, trained specifically on a construction site image dataset. The dataset's labeled categories strictly cover five key elements: foundation pit areas, lifting and hoisting equipment, paving equipment, protective fencing facilities, and on-site workers. Image enhancement strategies are incorporated during model training to improve robustness. Deployed on GPU servers or edge computing devices, it supports real-time video stream processing or offline batch image analysis. For the input inspection image, the module detects multiple target classes in parallel, outputting structured information for each identified target, including target category identifier, bounding box pixel coordinates, and detection confidence score. The module's inference speed on edge computing devices reaches over twenty frames per second, meeting the real-time requirements of engineering applications. The detection layer and backend logic judgment module are deployed separately, performing only general target detection tasks without relying on any hidden danger samples for training, significantly reducing the system's dependence on labeled data.

[0092] Pixel-level Spatial Range Calculation Module: This module connects to the target multi-class recognition module via a data interface. As the core of the system's spatial analysis, it is responsible for constructing spatial determination regions based on bounding box coordinate information in the two-dimensional image pixel coordinate system. The module implements the calculation process entirely based on pixel-level geometric operations, without requiring camera parameter calibration or physical size conversion. It is suitable for construction scenarios with variable scales and no need for precise measurement under UAV aerial photography perspectives. The module contains three parallel computing units: the foundation pit boundary zone calculation unit receives the foundation pit boundary box and expands it outward by one time the width or height in each of the four directions (up, down, left, and right) to form four rectangular boundary zones; the hoisting and lifting warning area calculation unit receives the hoisting equipment boundary box, calculates the center point coordinates, and takes the maximum side length as the radius to construct a circular warning area; the paving equipment hazard buffer area calculation unit receives the paving equipment boundary box and expands it outward by one time the size at equal intervals in all directions to form a rectangular buffer area. All calculations use fixed geometric formulas, and the region coordinates are defined within the pixel coordinate system. The algorithm has low complexity and achieves a real-time processing speed of more than 20 frames per second on edge computing devices. The calculation results are output as a standardized geometric region, providing a unified input interface for subsequent logical judgments.

[0093] Spatial Logic Relationship Judgment Module: This module serves as the system's decision-making core. Based on the pixel-level containment relationship between the target center point and the spatial judgment area, it performs standardized spatial logic judgments to automatically identify safety hazards. The module employs a unified judgment criterion: extracting the pixel coordinates of the center point of each target bounding box and determining whether each center point falls within the corresponding spatial judgment area. The module includes three parallel judgment units: a foundation pit protection integrity judgment unit checks whether each of the four boundary zones contains at least one protective fence center point; if any boundary zone lacks a fence, it is judged as a protection deficiency hazard; a lifting and hoisting warning deficiency judgment unit checks whether a protective fence center point exists within the circular warning area for each lifting device; if not, it is judged as a warning deficiency hazard; and a paving worker intrusion judgment unit checks whether a worker center point exists within the rectangular buffer area for each paving device; if present, it is judged as a personnel intrusion hazard. All judgment logic is completed within the pixel coordinate system, eliminating the need for complex 3D modeling or real-scale restoration. The judgment results include the hazard type, associated target ID, and specific location information of unmet conditions, which are then passed to the downstream annotation module for visualization processing.

[0094] Hazard Labeling and Output Module: This module serves as the system's result presentation unit, used to label identified hazards with images and output structured data, enabling the visualization and digital management of hazard information. The module receives raw inspection images and hazard assessment results, overlaying highlighted labels onto the original image to mark hazard locations. The shape of the label is either rectangular or circular, depending on the hazard type, and orange is used as the warning color. Text labels clearly describe the hazard category, including descriptions such as "lack of foundation pit protection," "no warning in hoisting area," or "personnel intrusion in paving area." Simultaneously, a JSON-formatted structured data file is generated, containing a unique image identifier, hazard category label, target bounding box coordinate description, hazard location coordinates, detection confidence score, and timestamps, among other metadata. The JSON file and labeled image are associated via the main filename for easy retrieval and archiving. The output results are transmitted wirelessly to a web visualization platform. After parsing the JSON content, the platform dynamically renders the hazard locations on the monitoring interface, allowing managers to view hazard distribution, filter risk events, replay historical records, and generate statistical reports. The entire output process is automated, with annotation and coding operations completed in milliseconds, ensuring the continuity and efficiency of the inspection process. The output results can be stored locally as project archives or pushed to a remote monitoring center in real time for centralized management.

[0095] Furthermore, the system is deployed on GPU servers or edge computing devices, supporting video stream processing or batch image analysis, and the spatial determination region calculation process achieves a processing speed of over 20fps on edge computing devices.

[0096] Furthermore, the pixel-level spatial range calculation module includes a foundation pit boundary zone calculation unit, a hoisting warning area calculation unit, and a paving buffer area calculation unit. Each calculation unit operates independently and outputs standardized geometric area description data through geometric formulas.

[0097] Furthermore, the system supports loading preset combinations of spatial judgment parameters for different construction scenarios. These parameter combinations are determined based on construction safety specifications and include the boundary zone expansion ratio, the warning zone radius coefficient, and the buffer zone expansion multiple.

[0098] Furthermore, the drone image acquisition module adopts a multi-rotor drone, equipped with a high-definition camera and gimbal stabilization structure, and supports flight path planning, fixed-point hovering and automatic shooting functions. The acquired image data is sent to the back-end processing platform wirelessly.

[0099] Example 1: Identification of Hidden Dangers Due to Inadequate Foundation Pit Protection

[0100] This embodiment describes the complete process by which the system inspects the foundation pit area at a highway construction site and identifies potential safety hazards due to lack of protection.

[0101] First, the drone flies to the airspace above the excavation pit working area along a preset vertical overhead flight path, adjusts its flight altitude and gimbal angle to ensure the camera's optical axis is perpendicular to the ground, and captures a clear image of the excavation pit area. The drone then collects high-resolution images while hovering, ensuring the complete visibility of the excavation pit's outline and surrounding protective facilities.

[0102] Image data is transmitted to an edge computing device via a wireless communication link. After receiving the image, the target multi-class recognition module uses the YOLOv8 model to detect the bounding boxes of the foundation pit and multiple protective fences in the image in parallel. The model outputs the class label, four-corner coordinates, and confidence score for each type of target.

[0103] After receiving the foundation pit boundary box, the pixel-level spatial range calculation module extracts its width and height values ​​and constructs boundary bands in four directions according to geometric formulas. The vertical range of the upper boundary is formed by extending the original upper border upwards and downwards by 0.1 times the frame height, while the horizontal range maintains the original width; the lower boundary band is applied to the lower border using the same extension method; the left boundary band extends horizontally to the left and right by 0.1 times the frame width, while the vertical range remains unchanged; the right boundary band performs a symmetrical horizontal extension. The four boundary bands together constitute the foundation pit protection determination area.

[0104] The spatial logic relationship judgment module traverses the center points of all protective fencing targets, calculates their coordinate positions one by one, and determines whether each center point falls within any of the four boundary zones. The system counts the number of fencing center points contained in each boundary zone. When it detects that no fencing center point exists within a certain boundary zone, it determines that protection in that direction is lacking. In this embodiment, the system detects that no fencing center point falls within the lower boundary zone of the foundation pit, triggering a hazard judgment.

[0105] The hazard labeling and output module overlays an orange rectangular highlight box at the lower boundary of the foundation pit in the original image, displays the label text "Foundation Pit Protection Missing," and generates a JSON structured data file containing fields such as image identifier, hazard type, and direction of the missing boundary. After receiving the data, the web visualization platform pops up an alert message on the monitoring interface to remind safety management personnel to conduct on-site verification.

[0106] Example 2: Identification of Missing Warning Points in Lifting and Hoisting Operations

[0107] This embodiment describes the complete process by which the system inspects the lifting and hoisting work area and identifies potential hazards caused by insufficient warnings.

[0108] The drone flies around the hoisting operation area using an oblique-view shooting method, with the camera's optical axis at a 45-degree angle to the horizontal plane, capturing the overall outline of the hoisting equipment and its surrounding environment from the side. The drone hovers at a fixed point outside the safe range of the hoisting equipment, continuously acquiring multiple frames of images to ensure complete coverage.

[0109] After the image data is input into the target multi-class recognition module, the model detects the bounding box of the truck crane and the bounding box of the nearby protective fence. The pixel-level spatial range calculation module calculates the center point coordinates of the bounding box of the lifting equipment as the center, takes the larger value between the width and height of the bounding box as the radius parameter, and constructs a circular warning area that covers the maximum impact range of the lifting operation.

[0110] The spatial logic relationship judgment module extracts the coordinates of the center points of all protective fencing targets, calculates the Euclidean distance from each center point to the center of the circle, and compares it with the radius of the circle. When the distance from the center point of all fencing targets to the center of the circle is greater than the radius, it is determined that there are no protective facilities within the circular warning area, indicating a potential hazard of insufficient warning in the hoisting operation area.

[0111] The hazard labeling and output module overlays orange circles around the edges of the circular warning area, displays the text label "No warning in hoisting area," and generates structured data output. The system pushes alarm information to the on-site safety officer via a web platform, recommending the immediate installation of warning barriers.

[0112] Example 3: Intrusion Detection of Paving Workers

[0113] This embodiment describes the complete process by which the system inspects the paver's work area and identifies potential human intrusion risks.

[0114] The drone flies in front of the paver in its direction of travel, maintaining a relatively fixed distance. Its camera uses an oblique angle to capture images of the work area in front of the paver, ensuring coverage of both the machine's operational range and the area where personnel are active. The drone continuously collects video streams during its flight and transmits them in real-time to the processing platform.

[0115] The target multi-category recognition module performs frame-by-frame detection of the video frames to identify the paver's bounding box and the operator's bounding box. The pixel-level spatial range calculation module expands the paver's bounding box outwards by a factor of one at equal intervals to construct a rectangular hazard buffer zone, which represents the hazardous impact range of the machinery's operation.

[0116] The spatial logic relationship judgment module traverses the center points of all worker targets and determines whether the coordinates of each center point fall within the paver's buffer area. When any worker's center point is detected to be within the buffer area, it is determined that there is a potential conflict of entry into the machinery's hazardous work area, and the target ID and location information of the intruder are recorded.

[0117] The hazard labeling and output module overlays a red rectangle and warning label at the location of the intruder, displaying the text "Paver intrusion," and simultaneously generates JSON data containing detailed information such as the intrusion time, the intruder's location coordinates, and the paver's location. The web platform displays the alarm screen in real time and triggers an audible and visual alarm to remind on-site workers to immediately evacuate the danger zone.

[0118] Example 4: Edge Device Deployment Performance

[0119] This embodiment describes the deployment and performance of the system on the NVIDIA Jetson Xavier NX edge computing device.

[0120] The hardware platform utilizes the Jetson Xavier NX embedded computing device, featuring GPU acceleration capabilities and power consumption kept below 15 watts, making it suitable for field operations. The system employs TensorRT to deeply optimize the YOLOv8 object detection model, including layer fusion, accuracy calibration, and automatic kernel tuning, significantly improving inference efficiency.

[0121] The processing flow is broken down into four stages: the object detection stage executes forward propagation of a convolutional neural network to extract multi-scale features and predict bounding boxes; the spatial range calculation stage executes three types of geometric formula operations in parallel, using the multi-threading capabilities of the GPU to accelerate matrix calculations; the logical judgment stage uses a lightweight loop traversal to quickly determine the center point inclusion relationship; and the annotation output stage calls the OpenCV library to perform image rendering and JSON encoding.

[0122] At high-definition image resolution, target detection takes less than 40 milliseconds, spatial range calculation takes about 10 milliseconds, logical judgment takes about 5 milliseconds, and annotation output takes about 3 milliseconds. The total processing time per frame is about 60 milliseconds, corresponding to a processing speed of over 20 frames per second, meeting the real-time inspection needs of construction sites. The system remains stable under continuous operation, without memory leaks or performance degradation, and its environmental temperature adaptability meets the requirements for outdoor construction.

[0123] Example 5: Extended Application of Parameter Configuration

[0124] This embodiment describes a system that uses a JSON configuration file to achieve multi-scenario adaptation and dynamic parameter adjustment.

[0125] The system includes pre-set configuration templates for various construction scenarios, such as bridge construction, roadbed construction, and pavement paving. The JSON configuration file contains scenario type identifiers and combinations of spatial determination parameters, including the pit boundary zone expansion ratio, the radius coefficient of the hoisting warning zone, and the expansion multiple of the paving equipment buffer zone. By default, the pit boundary zone expansion ratio is set to 0.1. For bridge high-pier construction scenarios, this can be adjusted to 0.15 to accommodate greater safety clearance requirements.

[0126] The configuration file supports both online modification via a visual interface and offline loading of local files. Safety administrators can select the construction scenario type through the web platform's parameter configuration page, after which the system automatically loads the corresponding parameter set. They can also manually fine-tune the values ​​of each parameter; adjustments take effect immediately without requiring a system restart or model reloading. The configuration process is logged for easy traceability and auditing.

[0127] In nighttime construction scenarios, the system expands to support infrared image input channels. By configuring image fusion weight parameters, visible light and infrared features are weighted and fused, improving the model's detection capabilities under low-light conditions. All configuration parameters are stored in plaintext JSON format, facilitating version management and batch deployment, ensuring the system's flexible adaptability to different construction environments and safety regulations.

[0128] By adopting the above-disclosed technical solution of this invention, the following beneficial effects are obtained:

[0129] This invention achieves intelligent hazard identification without the need for precise dimensional measurement by uniformly identifying multiple key elements at a construction site (such as foundation pits, lifting equipment, paving equipment, protective fencing, and workers) and establishing spatial logical relationships within the image pixel space. Compared with existing technologies that rely on single-class detection or end-to-end hazard classification, this invention has the following significant advantages:

[0130] (1) The identification mechanism is more universal and scalable.

[0131] Traditional methods rely on a large number of training samples labeled with potential hazards. This invention only requires regular target detection data to complete the identification task. It uses logical rules to infer potential hazards, which significantly reduces the amount of sample labeling. The model can be quickly transferred to different construction scenarios without retraining the hazard model.

[0132] (2) The computing structure is simplified and the system deployment is more efficient.

[0133] This invention replaces complex 3D modeling or precise calibration with "target detection + spatial logic judgment". The recognition logic relies only on pixel-level geometric relationships, reducing the algorithm complexity compared to traditional 3D reconstruction methods. It can achieve real-time inference on edge computing devices, meeting the actual needs of UAVs to recognize while flying.

[0134] (3) Improved accuracy and consistency in hazard identification

[0135] By employing unified spatial logic rules (such as the principle that the target's center point falls within the judgment area), this invention achieves standardized judgment of potential hazards such as lack of protection, lack of vigilance, and personnel intrusion. The logical judgment results are stable and highly consistent.

[0136] (4) Reduce manual intervention and improve inspection efficiency

[0137] Traditionally, a safety officer can inspect about 3 to 5 high-risk work sites per day on average. This invention combines drone inspection with automatic identification, which can process more than 2,000 images in the same amount of time, improving inspection efficiency and realizing automatic hazard labeling and report generation, reducing the workload of manual review.

[0138] (5) The system is highly adaptable and has low application costs.

[0139] This invention does not rely on specific camera parameters or scene scale, and is applicable to different brands of drones and various construction types (bridges, roadbeds, road paving, etc.). It can be quickly integrated into existing safety inspection platforms or mobile systems, and has good engineering feasibility and promotional value.

[0140] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for intelligent identification of potential hazards in highway construction drones based on spatial logic judgment, characterized in that, Includes the following steps: S1. Collect inspection images of highway construction sites using drones equipped with visible light cameras; S2. A deep learning target detection model is used to identify and locate multiple construction elements in the inspection image, and to obtain the bounding box coordinate information of each type of target; the construction elements include at least foundation pits, lifting and hoisting equipment, paving equipment, protective fences, and workers; S3. In the two-dimensional image pixel coordinate system, the corresponding spatial determination region is constructed by calculating the bounding box coordinate information using geometric formulas. The calculation process is entirely based on pixel-level geometric relationships and does not require camera parameter calibration or physical size conversion. S4. Based on the pixel-level inclusion relationship between the spatial determination area and the target center point, perform standardized spatial logic judgment to automatically identify safety hazards; the judgment logic includes judgment of the integrity of the foundation pit protection, judgment of the lack of hoisting and lifting warning, and judgment of intrusion by paving workers. S5. Output the hazard identification results, including hazard location markings, hazard category information, and structured data.

2. The method according to claim 1, characterized in that, The spatial determination region mentioned in step S3 is constructed by calculating the following geometric formula, and the spatial determination region construction process achieves a processing speed of more than 20 fps on the edge computing device: For the foundation pit target, let its bounding box width be w and height be h. Then, expand it by 0.1 times the width or height pixel distance in four directions to form four rectangular boundary band regions. The upper boundary band has a vertical range of [y_min-0.1h, y_min+0.1h] and a horizontal range of [x_min, x_max]. The lower boundary band has a vertical range of [y_max-0.1h, y_max+0.1h] and a horizontal range of [x_min, x_max]. The left boundary band has a horizontal range of [x_min-0.1w, x_min+0.1w] and a vertical range of [y_min, y_max]. The right boundary band has a horizontal range of [x_max-0.1w, x_max+0.1w] and a vertical range of [y_min, y_max]. For a target lifting and hoisting equipment, let the coordinates of its bounding box center point be (x_c, y_c), where x_c = (x_min + x_max) / 2, y_c = (y_min + y_max) / 2, and the maximum side length r = max(w, h). Then, a circular warning area is constructed with the center point as the center and r as the radius. This area is defined by the equation (x - x_c)² + (y - y_c)² ≤ r². For the paving equipment target, let its bounding box coordinates be (x_min, y_min, x_max, y_max). Then, expand outward by one size to form a rectangular danger buffer zone with coordinates (x_min-w, y_min-h, x_max+w, y_max+h).

3. The method according to claim 1, characterized in that, The spatial logic judgments described in step S4 all adopt a unified judgment criterion: extract the pixel coordinates of the center point of the target bounding box, and determine whether the center point falls within the corresponding spatial judgment area. If it falls within the area, it is determined that the spatial association condition is met.

4. The method according to any one of claims 1-3, characterized in that, The deep learning target detection model is trained using a construction site image dataset. The dataset is labeled with at least five categories: foundation pit, lifting and hoisting equipment, paving equipment, protective fencing, and workers. The target detection model is deployed separately from the backend logic judgment module, and the detection layer does not rely on hazard samples for training.

5. The method according to any one of claims 1-3, characterized in that, The hazard identification results are obtained by combining visual annotation with JSON structured data output. The JSON data fields include at least the image number, hazard category, target bounding box coordinates, and hazard location coordinates.

6. A smart hazard identification system for highway construction drones based on spatial logic judgment, characterized in that, include: The drone image acquisition module is used to acquire high-definition inspection images of the construction site; The multi-category target recognition module, based on a deep learning target detection model, identifies and locates foundation pits, lifting equipment, paving equipment, protective fences, and workers in inspection images, and outputs bounding box coordinate information. The pixel-level spatial range calculation module is connected to the target multi-category recognition module through a data interface. It is used to calculate and construct a spatial determination region in a two-dimensional image coordinate system based on the bounding box coordinate information and geometric formulas. The calculation process is entirely based on pixel-level geometric operations and does not require camera parameter calibration or physical size conversion. The spatial logical relationship judgment module performs judgments on the integrity of foundation pit protection, the absence of hoisting and lifting warning, and the intrusion of paving workers based on the pixel-level inclusion relationship between the target center point and the spatial judgment area. The hazard labeling and output module is used to label images of identified hazards and output structured data.

7. The system according to claim 6, characterized in that, The system is deployed on a GPU server or edge computing device, supports video stream processing or batch image analysis, and the spatial determination region calculation process achieves a processing speed of more than 20fps on the edge computing device.

8. The system according to claim 6, characterized in that, The pixel-level spatial range calculation module includes a foundation pit boundary zone calculation unit, a hoisting warning area calculation unit, and a paving buffer area calculation unit. Each calculation unit operates independently and outputs standardized geometric area description data through geometric formulas.

9. The system according to any one of claims 6-8, characterized in that, The system supports loading preset combinations of spatial judgment parameters for different construction scenarios. These parameter combinations are determined based on construction safety specifications and include boundary zone expansion ratio, warning zone radius coefficient, and buffer zone expansion multiple.

10. The system according to any one of claims 6-8, characterized in that, The UAV image acquisition module uses a multi-rotor UAV, equipped with a high-definition camera and gimbal stabilization structure, and supports flight path planning, fixed-point hovering and automatic shooting functions. The acquired image data is sent to the back-end processing platform wirelessly.

Images