Unmanned aerial vehicle patrol intelligent early warning method, device and equipment based on construction safety distance and medium

Abstract

The invention discloses an unmanned aerial vehicle patrol intelligent early warning method and device based on a construction safety distance, equipment and a medium, and the method comprises the steps: constructing an original backbone network, and carrying out the model training of the original backbone network based on a marked image data set, and obtaining an instance segmentation model; collecting an overhead video stream of the current construction site scene according to a preset route path through the unmanned aerial vehicle; calculating the minimum pixel distance between each person and the contour convex hull of each device based on the segmentation result, and converting the minimum pixel distance into an actual physical distance according to the known reference size; and judging whether to trigger early warning and the early warning level based on the effective safety distance threshold and the risk score, and executing a corresponding early warning response action according to the early warning level. Construction site personnel and construction equipment are recognized in real time through an intelligent algorithm, distance measurement analysis is carried out, safety distance violation behaviors are found in time, and the risk of collision accidents of personnel and equipment is effectively reduced.

Description

Technical Field

[0001] This invention relates to the field of drone patrol technology, and in particular to a method, device, equipment and medium for intelligent early warning of drone patrol based on construction safety distance. Background Technology

[0002] In construction site settings, managing the safe distance between personnel and construction equipment (such as excavators and cranes) is crucial to preventing accidents. If personnel and equipment are too close, there is a high risk of collision and other safety hazards.

[0003] Traditional manual inspections and supervision are inefficient and cannot achieve 24 / 7, all-around security monitoring. Using fixed camera equipment not only has limited coverage and cannot flexibly adjust monitoring positions, but also makes it difficult to adapt to dynamic construction environments due to obstructions from objects in the frame. While GPS positioning systems can provide location information for personnel and equipment, their accuracy is limited in complex environments, making it difficult to meet the requirements for determining safe distances.

[0004] With the development of drone technology, it has become possible to use drones for construction site inspections. Drones are characterized by high flexibility, wide coverage, and high real-time performance. Combined with computer vision and artificial intelligence technologies, they can be used to monitor the safe distance between construction personnel and equipment in real time and issue timely warnings to avoid potential accidents.

[0005] Therefore, there is an urgent need for an intelligent early warning method for drone patrols based on construction safety distance. Summary of the Invention

[0006] In view of the above problems, the present invention is proposed to provide a method, device, equipment and medium for intelligent early warning of drone patrol based on construction safety distance to overcome or at least partially solve the above problems.

[0007] Other features and advantages of the invention will become apparent from the following detailed description, or may be learned in part by practice of the invention.

[0008] According to a first aspect of the present invention, a method for intelligent early warning of unmanned aerial vehicle (UAV) patrols based on construction safety distances is provided, comprising: By using drones to collect video data in various construction site scenarios, an aerial image dataset of personnel and construction equipment is constructed, and pixel-level masking is performed on the personnel and construction equipment in the image dataset. An original backbone network is constructed, and an instance segmentation model is obtained by training the original backbone network based on the labeled image dataset. The drone collects video streams of the current construction site scene from the air following a preset flight path, which includes flight altitude, speed, and zoom parameters. The instance segmentation model is used to perform real-time analysis on the returned video stream, identify and segment personnel and construction equipment in the video stream, and obtain segmentation results. Based on the segmentation results, the minimum pixel distance between each person and the convex hull of each device is calculated, and converted into the actual physical distance according to the known reference size; Based on the actual physical distance and combined with a preset rule base, the effective safe distance threshold and risk score of the current construction site scenario are dynamically calculated. Based on the effective safe distance threshold and risk score, it is determined whether to trigger an early warning and the warning level. According to the warning level, the corresponding early warning response action is executed.

[0009] In some embodiments of the present invention, the step of constructing an original backbone network and training the original backbone network to obtain an instance segmentation model based on the labeled image dataset includes: Based on the target-based instance segmentation model CenterMask, an original backbone network VoVNetV2 consisting of multiple OSA modules is constructed. A convolutional block attention module (CBAM) is inserted after the aggregation layer of each OSA module in the original backbone network VoVNetV2 to obtain the basic model. A scale prediction module, SizePredict, is added to the detection head of the base model. The SizePredict module is used to dynamically select the feature layer level of the Feature Pyramid Network (FPN) based on the target size in the image dataset. The SAG-MASK module is set in the base model. The SAG-MASK module is used to perform instance segmentation of the target in the base model, progressively upsample the region of interest feature map extracted by the RoI Align method to the target resolution, and use the PointRend method to refine the target segmentation edge upsampled to the target resolution. Add a boundary loss term, BoundaryLoss, to the loss function of the base model to calculate the boundary loss; The image dataset is divided into a training set and a validation set according to a predetermined ratio. After training the base model, the instance segmentation model is obtained.

[0010] In some embodiments of the present invention, the scale prediction module SizePredict dynamically selects the feature layer level of the Feature Pyramid Network (FPN) based on the target size in the image dataset, including: Define the feature layer level allocation rules of the Feature Pyramid Network (FPN), wherein the allocation rules characterize the feature layer level corresponding to targets of different sizes in the image dataset; The target size in the image dataset is determined by calculation; The feature layer level is dynamically selected based on the target size, and features from adjacent layers are fused for targets that are close to the feature layer level boundary.

[0011] In some embodiments of the present invention, the progressive upsampling of the region of interest feature map extracted by the RoI Align method to the target resolution includes: Determine the target resolution based on the target characteristics; The feature map of the region of interest is extracted using the RoI Align method; The first feature map is obtained by upsampling the feature map of the region of interest using a deconvolution library function. The first feature map is then upsampled using the deconvolution library function, and this operation is repeated until the resolution of the feature map of the region of interest reaches the target resolution.

[0012] In some embodiments of the present invention, the step of calculating the minimum pixel distance between the contour convex hull of each person and each device, and converting it into an actual physical distance based on a known reference size, includes: Extract the convex hulls corresponding to each identified personnel contour and construction equipment contour to obtain the personnel convex hull point set and the equipment convex hull point set; Calculate the minimum pixel distance between the convex hull point set of each person and the convex hull point set of each device; Obtain the maximum pixel width Pw of the person's convex hull along the horizontal axis, and use the preset reference shoulder width value as the known reference size to calculate the scale factor Scale = reference shoulder width value / Pw; Multiplying the minimum pixel distance by the scale factor yields the actual physical distance between the person and the device.

[0013] In some embodiments of the present invention, the step of dynamically calculating the effective safe distance threshold of the current construction site scenario in conjunction with a preset rule base includes: The effective safe distance threshold for ESD is obtained by the following formula: ESD = BSR × SM; Wherein, BSR is the basic safety radius set in the preset rule base based on the type of construction equipment, and SM is the state coefficient set in the preset rule base based on the operating status of the construction equipment.

[0014] In some embodiments of the present invention, the step of dynamically calculating the risk score of the current construction site scenario by combining a preset rule base includes: The risk score RS is obtained using the following formula: RS = (ESD / (d + S)) × BW; Where d is the actual physical distance, S is a constant, and BW is a behavior weighting factor set in the preset rule base based on the personnel's behavior status.

[0015] According to a second aspect of the present invention, a drone-based intelligent early warning device for construction safety distance patrols is provided, the device comprising: The data processing module is used to collect video data using drones in various construction site scenarios, construct an aerial image dataset of personnel and construction equipment, and perform pixel-level masking annotation on personnel and construction equipment in the image dataset. The model building module is used to build the original backbone network and train the original backbone network based on the labeled image dataset to obtain an instance segmentation model. The image data acquisition module is used to acquire video streams of the airspace above the current construction site scene via a drone following a preset flight path, wherein the flight path includes flight altitude, speed, and zoom parameters. The image segmentation module is used to perform real-time analysis on the returned video stream using the instance segmentation model, identify and segment personnel and construction equipment in the video stream, and obtain segmentation results. The distance calculation module is used to calculate the minimum pixel distance between the outline convex hull of each person and each device based on the segmentation results, and convert it into the actual physical distance according to the known reference size; The early warning processing module is used to dynamically calculate the effective safe distance threshold and risk score of the current construction site scene based on the actual physical distance and in combination with the preset rule base, and to determine whether to trigger an early warning and the early warning level based on the effective safe distance threshold and risk score, and to execute the corresponding early warning response action according to the early warning level.

[0016] According to a third aspect of the present invention, a computer device is provided, including a processor and a memory, the memory storing computer program instructions executable by the processor, wherein when the processor executes the computer program instructions, it implements the instructions as described in any of the above methods.

[0017] According to a fourth aspect of the present invention, a computer-readable storage medium is provided, wherein computer program instructions are stored therein, the computer program instructions being loaded and executed by a processor to perform the operations performed by the method described in any of the preceding claims.

[0018] The technical solutions provided in the embodiments of the present invention have at least the following technical effects or advantages: This invention provides an intelligent early warning method, device, equipment, and medium for drone patrol based on construction safety distance. The method utilizes the high mobility and wide field of view of drones to identify construction personnel and equipment in real time through intelligent algorithms and perform distance measurement analysis. It can promptly detect violations of safety distance and provide an immediate early warning and intervention basis for construction site safety management. This invention significantly improves the monitoring efficiency and response speed of safety distance, and is especially suitable for high-risk operation areas, effectively reducing the risk of collisions between personnel and equipment.

[0019] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 A flowchart illustrating an intelligent early warning method for drone patrol based on construction safety distance, provided as an embodiment of the present invention; Figure 2 A schematic diagram illustrating the process of training the original backbone network to obtain an instance segmentation model; Figure 3 A schematic diagram of the principle structure of an intelligent early warning device for drone patrol based on construction safety distance provided in an embodiment of the present invention; Figure 4 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0022] Exemplary embodiments of this application will now be described in more detail with reference to the accompanying drawings.

[0023] The accompanying drawings illustrate various structural schematics according to embodiments of this application. These drawings are not to scale, and some details have been enlarged for clarity, and some details may have been omitted. The shapes of the various regions and layers shown in the drawings, as well as their relative sizes and positional relationships, are merely exemplary and may deviate from reality due to manufacturing tolerances or technical limitations. Furthermore, those skilled in the art can design regions / layers with different shapes, sizes, and relative positions as needed.

[0024] It should be noted that the terms "first," "second," etc., used in this application can be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from the second element. The terms "comprising" and "having," and any variations thereof, used in this application, are intended to cover non-exclusive inclusion. The term "multiple" used in this application refers to two or more. In the context of this application, similar or identical parts may be represented by the same or similar reference numerals.

[0025] To better understand the above technical solutions, the following will describe the above technical solutions in detail with reference to specific implementation methods. It should be understood that the embodiments of this application and the specific features in the embodiments are detailed descriptions of the technical solutions of the present invention, rather than limitations on the technical solutions of the present invention. In the absence of conflict, the embodiments of the present invention and the technical features in the embodiments can be combined with each other.

[0026] Figure 1 This is a flowchart illustrating an intelligent early warning method for drone patrols based on construction safety distance, provided by an embodiment of the present invention. Figure 1 As shown, the intelligent early warning method for drone patrol based on construction safety distance includes the following steps: S1. Use drones to collect video data in various construction site scenarios, construct an aerial view image dataset of personnel and construction equipment, and perform pixel-level masking annotation on the personnel and construction equipment in the image dataset. In this embodiment of the invention, the construction site scene includes, but is not limited to, municipal engineering, bridge construction, and residential construction under low-altitude circling and fixed-point hovering conditions; the construction of the aerial perspective image dataset of personnel and construction equipment includes: using drones to collect video data in various construction site scenes, extracting video frames from the video data according to a predetermined sampling frequency, manually deleting frames that do not contain the target object to be detected after extraction to obtain usable frames, and constructing an aerial perspective image dataset of personnel and construction equipment (such as excavators, pile drivers, drilling machines, cranes, bulldozers, road rollers, mixers, heavy trucks, etc.).

[0027] After obtaining the image dataset, this embodiment of the invention uses an annotation tool to perform pixel-level mask annotation on the personnel and construction equipment in the image dataset. The annotation tool is, for example, X-AnyLabeling. X-AnyLabeling can automatically and quickly annotate by loading some models (such as the YOLOE(ye) model, which is used to identify any object model in real time and can perform detection and segmentation tasks based on text and visual enhancement). After loading the YOLOE(ye) model, the annotator can click on the target area or select the target by box. Based on this visual cues, the YOLOE(ye) model will automatically generate the complete outline of the target. Finally, the annotator manually adjusts (corrects edges, etc.), checks for errors and omissions, generates a JSON label file, and saves it.

[0028] S2. Construct the original backbone network, and train the original backbone network based on the labeled image dataset to obtain an instance segmentation model; Because there are many types of construction equipment in a construction site, it is necessary to extract more representative features for different types of construction equipment. However, there is a huge range in size between people and construction equipment. For smaller people, it is necessary to finely segment the human body outline, while for larger construction equipment, it is necessary to accurately understand the overall shape and structural features. Based on this, the embodiments of the present invention need an instance segmentation model that can pay attention to both local details and grasp global information.

[0029] See Figure 2 As shown, in this embodiment of the invention, an original backbone network is constructed, and an instance segmentation model is obtained by training the original backbone network based on the labeled image dataset. This includes: S21. Based on the target-based instance segmentation model CenterMask, construct the original backbone network VoVNetV2 consisting of multiple OSA modules. Insert a convolutional block attention module (CBAM) after the aggregation layer of each OSA module in the original backbone network VoVNetV2 to obtain the basic model. In this embodiment of the invention, the instance segmentation model CenterMask is used as the target model, and a pre-constructed original backbone network VoVNetV2 composed of multiple OSA (One-Shot Aggregation) modules is built. The original backbone network VoVNetV2 is the backbone network of the instance segmentation model CenterMask. The OSA module can achieve efficient feature fusion by aggregating all intermediate features at once. The CBAM (Convolutional Block Attention Module) is used for the attention mechanism of convolutional neural networks, and performs attention calculation on the feature map in two dimensions: channel and space.

[0030] In this embodiment of the invention, the convolutional block attention module (CBAM) inserted after the aggregation layer of each OSA module in the original backbone network VoVNetV2 includes: performing global average pooling and global max pooling on the output x of each OSA aggregation layer; inputting the two pooling results into two fully connected layers respectively; summing the output features and passing them through an activation function to generate channel attention weights Wc; multiplying the output x with the channel attention weights Wc to obtain a new output y; performing average pooling and max pooling on the output y along the channel dimension; concatenating the two pooling results into a feature map; compressing the channels to 1 through a convolutional layer; generating spatial attention weights Ws through an activation function; and multiplying y with Ws to obtain the final output z. Thus, through the synergistic effect of channel attention ca and spatial attention sa, the ability to focus on key equipment parts (such as crane hooks) can be enhanced, and category-specific features (such as excavator buckets and bulldozer blades) can be amplified.

[0031] S22. Add a scale prediction module SizePredict to the detection head of the basic model. The scale prediction module SizePredict is used to dynamically select the feature layer level of the Feature Pyramid Network (FPN) according to the target size in the image dataset. To address the issue of significant size differences between personnel and construction equipment, this invention introduces a Dynamic ScalePrediction (DSP) mechanism, which dynamically selects feature maps based on the size of the target. Currently, the Feature Pyramid Network (FPN) in CenterMask assigns targets according to fixed rules (feature layer level P3: small target, feature layer level P4: medium target, feature layer levels P4-P7: large target).

[0032] By adding a scale prediction module SizePredict to the detection head of the base model, where SizePredict refers to size prediction mechanisms in different contexts, in the embodiments of the present disclosure, the scale prediction module SizePredict dynamically selects the feature layer levels of the Feature Pyramid Network (FPN) based on the target sizes in the image dataset, including: defining the distribution rules for the feature layer levels of the FPN, where the distribution rules characterize the feature layer levels corresponding to targets of different sizes in the image dataset; calculating and determining the target sizes in the image dataset; dynamically selecting the feature layer levels according to the target sizes, and fusing the features of adjacent levels for targets close to the boundaries of the feature layer levels.

[0033] Specifically, calculating and determining the target sizes in the image dataset is to calculate the diagonal length d of the target = , and then dynamically allocate it to the most suitable FPN level (P3 - P7) according to the range of d. For example: When d < thr1, select the feature layer level P3 to match the target worker; When thr1 ≤ d < thr2, select the feature layer level P4 to match the target drill rig; When thr2 ≤ d < thr3, select the feature layer level P5 to match the target bulldozer and roller; When thr3 ≤ d < thr4, select the feature layer level P6 to match the target excavator, mixer, and pile driver; When thr4 ≤ d, select the feature layer level P7 to match the target crane and heavy truck; Among them, thr1 = 64, thr2 = 128, thr3 = 256, thr4 = 512, and the unit is pixel.

[0034] When the target size is close to the boundary of the feature layer level, for example, when d is 256, fuse P5 and P6 at the same time. For personnel targets, fuse the feature layer level P3 (details) and the feature layer level P4 (semantics) to avoid background interference; for cranes, fuse the feature layer level P5 (main body) and the feature layer level P6 (long structural features) to avoid fragmentation.

[0035] S23. Set a SAG - MASK module in the base model. The SAG - MASK module is used to perform instance segmentation on the targets in the base model, progressively upsample the region of interest feature map extracted by the RoI Align method to the target resolution, and refine the target segmentation edges upsampled to the target resolution using the PointRend method; The SAG-MASK (Spatial Attention Guide Mask) module can perform fine pixel-level segmentation of each target region. It extracts region of interest (RoI) features from the corresponding layer's FPN feature map at a resolution of 14×14 using the RoI Align method. To avoid the problem of blurred edges in the generated mask due to the limited resolution of RoI and the lack of effective modeling of edge details, this embodiment of the invention progressively upsamples the extracted region of interest feature map to the target resolution and uses the PointRend method to refine the target segmentation edges upsampled to the target resolution.

[0036] Specifically, the progressive upsampling of the region of interest feature map extracted by the RoI Align method to the target resolution includes: determining the target resolution based on the target characteristics; extracting the region of interest feature map using the RoI Align method; upsampling the region of interest feature map using a deconvolution library function to obtain a first feature map; upsampling the first feature map again using the deconvolution library function, and repeating this operation until the resolution of the region of interest feature map reaches the target resolution.

[0037] In this embodiment of the invention, the target resolution is first determined based on the target characteristics (or computational cost). For example, the target resolution is 56×56, while the default resolution output by the instance segmentation model CenterMask is 14×14. The extracted region of interest feature map can be progressively upsampled to the target resolution using the PyTorch library function nn.ConvTranspose2d(). nn.ConvTranspose2d() is a PyTorch library function. A deconvolution library function module is provided to implement two-dimensional transposed convolution (also known as deconvolution or upsampling convolution). The deconvolution library function `nn.ConvTranspose2d()` is used to upsample the feature map of the region of interest to obtain a first feature map, i.e., upsampling a 14×14 feature map to generate a 28×28 feature map. The first feature map is then upsampled again using the deconvolution library function, and this operation is repeated until the resolution of the feature map of the region of interest reaches the target resolution. Specifically, the deconvolution library function `nn.ConvTranspose2d()` is used to upsample the 28×28 feature map to generate a 56*56 feature map, thus achieving the target resolution for the feature map.

[0038] To effectively recover the spatial details and edge information lost due to the reduction in RoI size, this embodiment of the invention further refines the edge regions using the PointRend (point-based rendering edge refinement technique) method. First, the mask output from the above steps, achieving the target resolution, is used as input. Pixels with high uncertainty (such as areas with mask values ​​close to 0.5, which are often where edges are located) are selected. Local features are extracted from the original feature map for these selected keypoints (i.e., pixels with high uncertainty), and then fed into a lightweight sub-network for refined prediction. The final classification result for each keypoint is output. Finally, the original mask is updated based on the refined keypoint prediction results to obtain a more refined segmentation result.

[0039] S24. Add a boundary loss term to the loss function of the basic model to calculate the boundary loss; This invention adds a boundary loss term, Boundary Loss, to the loss function of the base model for calculating boundary loss. Based on this Boundary Loss term, the pixel-level deviation between the predicted boundary and the true boundary is quantified, effectively optimizing the target contour details. Specifically, distance transformation operations are performed on the true mask gt_mask (e.g., the data annotated with pixel-level masks in step S1) and the predicted mask pred_mask (e.g., the data output by the instance segmentation model CenterMask), generating two distance transformation maps. The true mask gt_dist records the distance of each pixel to the nearest true boundary, and the predicted mask pred_dist represents the distance of each pixel to the nearest predicted boundary. The boundary loss is calculated based on this and the predicted mask pred_mask, using the following formula: boundaryloss=<(pred_mask·gt_dist)>+<(1-pred_mask ) ·gt_dist>; In the formula, <> represents averaging over all elements, and · represents multiplying element by element.

[0040] S25. Divide the image dataset into a training set and a validation set according to a predetermined ratio, and train the base model to obtain the instance segmentation model.

[0041] In this embodiment of the invention, the image dataset is divided into a training set and a validation set according to a predetermined ratio (e.g., 5:1). The base model is trained. For the first training, the weights of the CenterMask instance segmentation model pre-trained on the COCO dataset can be selected to initialize the backbone network parameters. Subsequently, the weights of the instance segmentation models for personnel and construction scenes are continuously iterated and optimized. During the model training process, the loss value of each training cycle is calculated and monitored in real time, and the changing trends of the center loss and mask loss are observed. At the same time, the model performance is evaluated using the validation set after each training cycle. Finally, based on the loss convergence and performance evaluation results, the model with the strongest generalization ability and the best performance is selected as the instance segmentation model for deployment on the UAV system.

[0042] S3. Collect video streams of the airspace above the current construction site scene using a drone following a preset flight path, wherein the flight path includes flight altitude, speed, and zoom parameters; This invention allows for adaptive pre-setting of flight paths based on actual scenarios, ensuring that the video stream captured by the drone conforms to the recognition conditions of the current construction site scene. This avoids false detections of non-target areas and reduces computational resource consumption. For example, if the drone hangar is some distance from the construction site, during mission execution, the drone can be set to fly rapidly along the flight path at an altitude of 110 meters and a speed of 15 m / s to the current construction site area. After reaching the pre-planned waypoint, the flight altitude is reduced to 40 meters, and the zoom lens (z zoom set to 2) is activated to fly along the construction site flight path while maintaining a vertical overhead shooting operation. In other application scenarios, the flight path can be determined according to actual application requirements, and this invention does not impose any limitations on this.

[0043] S4. The returned video stream is analyzed in real time using the instance segmentation model to identify and segment the personnel and construction equipment in the video stream and obtain the segmentation result. S5. Based on the segmentation results, calculate the minimum pixel distance between the contour convex hull of each person and each device, and convert it into the actual physical distance according to the known reference size; In step S5, the embodiment of the present invention calculates the minimum pixel distance between the contour convex hull of each person and each piece of equipment, and converts it into the actual physical distance based on the known reference size, including: extracting the convex hull corresponding to each identified person contour and construction equipment contour to obtain the person convex hull point set and the equipment convex hull point set; calculating the minimum pixel distance between each person convex hull point set and each equipment convex hull point set; obtaining the maximum pixel width Pw of the person convex hull along the horizontal axis, and using a preset reference shoulder width value as the known reference size, calculating the scale factor Scale = reference shoulder width value / Pw; multiplying the minimum pixel distance by the scale factor to obtain the actual physical distance between the person and the equipment.

[0044] In this embodiment of the invention, a convex hull is found for the contours of each detected person and each piece of construction equipment. A convex hull point set for each person and a convex hull point set for each piece of construction equipment are constructed. The convex hull can be extracted, for example, using the `convexHull()` function in OpenCV. For each detected person, the maximum width Pw (in pixels) of their corresponding convex hull along the horizontal axis is calculated. Since the detection image is an orthophoto from a UAV, this maximum width can be approximated as the average shoulder width of an adult, 0.5m. Therefore, the reference shoulder width value is 0.5. This yields the conversion relationship Scale = ... (the distance from one pixel in the image to the real-world distance in meters). For this individual, calculate the minimum distance between the point set of the convex hull of their personnel and the point set of the convex hull of each construction device in the image. (The unit is pixels) ={ ,… , …, }, where i represents the personnel ID and j represents the corresponding type of construction equipment ID; then... Convert the obtained scale factor into actual physical distance (Unit is meters), the formula is as follows: = .

[0045] S6. Based on the actual physical distance and combined with the preset rule base, dynamically calculate the effective safe distance threshold and risk score of the current construction site scenario, and determine whether to trigger an early warning and the early warning level based on the effective safe distance threshold and risk score. Execute the corresponding early warning response action according to the early warning level.

[0046] In step S6, the embodiment of the present invention dynamically calculates the effective safe distance threshold for the current construction site scenario in conjunction with a preset rule base, including: The effective safe distance threshold for ESD is obtained by the following formula: ESD = BSR × SM; Wherein, BSR is the basic safety radius set in the preset rule base based on the type of construction equipment, and SM is the state coefficient set in the preset rule base based on the operating status of the construction equipment.

[0047] In this embodiment of the invention, the basic safety radius set in the preset rule base based on the type of construction equipment is shown in Table 1 for example: Table 1. Basic safety radius set for construction equipment types.

[0048] In this embodiment of the invention, the state coefficients set in the preset rule base based on the operating status of the construction equipment are shown in Table 2 for reference: Table 2 State coefficients for setting the operating status of construction equipment

[0049] For example, the effective safe distance threshold (ESD) for excavator operation is BSR × 1.0 = 5m. If the excavator is stationary, then ESD = 5 × 0.6 = 3m.

[0050] In step S6, the embodiment of the present invention dynamically calculates the risk score of the current construction site scenario in conjunction with a preset rule base, including: The risk score RS is obtained using the following formula: RS = (ESD / (d + S)) × BW; Where d is the actual physical distance, S is a constant, and BW is a behavior weighting factor set in the preset rule base based on the personnel's behavior status.

[0051] In this embodiment of the invention, the behavior weighting factors set based on the personnel's behavior status in the preset rule base are shown in Table 3 below: Table 3. Behavioral weighting factors for personnel behavior status settings

[0052] To avoid division by zero, this embodiment of the invention also provides a small constant S, for example, 0.1.

[0053] This invention embodiment determines whether to trigger an alert and the alert level based on the effective safe distance threshold and risk score. The conditions for triggering an alert are, for example: The warning level is Level 1 (Alert). The conditions for triggering the warning are d < 1.2 × ESD and behavior = "approaching". The warning response actions are, for example, the drone's local voice: "Attention, danger zone ahead!", and the App push notification to the user and the team leader. Warning level 2 (strong warning), the condition for triggering the warning is d. <esd 且 behavior="loitering" （>(10s), early warning response actions include activating ground-based audible and visual alarms, automatically recording and archiving video, and issuing pop-up alerts in the safety officer's app; The warning level is three (emergency intervention). The conditions for triggering the warning are d < 0.8 × ESD and behavior = "running through" or Risk Score > 2.5. Warning response actions include sending a pause signal to the equipment PLC (if supported), linking the broadcast system to announce the warning to the whole site, and generating a work order for accident hazards.

[0054] The intelligent early warning method for drone patrol based on construction safety distance described in this invention utilizes the high mobility and wide field of vision of drones. Through intelligent algorithms, it identifies construction personnel and equipment in real time and performs distance measurement analysis to promptly detect violations of safety distance. This provides an immediate early warning and intervention basis for construction site safety management. This invention significantly improves the monitoring efficiency and response speed of safety distance, and is especially suitable for high-risk operation areas (such as the rotation range of crane towers, excavator operation areas, etc.), effectively reducing the risk of collision accidents between personnel and equipment.

[0055] Based on the above embodiments, as a supplement to the above... Figure 1 The present invention provides an embodiment of an intelligent early warning device for drone patrol based on construction safety distance, which is similar to the method shown. Figure 1 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices, see reference. Figure 3 As shown, the intelligent early warning device for drone patrol based on construction safety distance includes: The data processing module 100 is used to collect video data using drones in various construction site scenarios, construct an aerial view image dataset of personnel and construction equipment, and perform pixel-level masking annotation on personnel and construction equipment in the image dataset. The model building module 200 is used to build the original backbone network and train the original backbone network based on the labeled image dataset to obtain an instance segmentation model. The image data acquisition module 300 is used to acquire video streams of the airspace above the current construction site scene by means of a drone following a preset flight path, wherein the flight path includes flight altitude, speed, and zoom parameters. The image segmentation module 400 is used to perform real-time analysis on the returned video stream using the instance segmentation model, identify and segment personnel and construction equipment in the video stream, and obtain segmentation results. The distance calculation module 500 is used to calculate the minimum pixel distance between the outline convex hull of each person and each device based on the segmentation results, and convert it into the actual physical distance according to the known reference size; The early warning processing module 600 is used to dynamically calculate the effective safe distance threshold and risk score of the current construction site scene based on the actual physical distance and in combination with the preset rule base, and to determine whether to trigger an early warning and the early warning level based on the effective safe distance threshold and risk score, and to execute the corresponding early warning response action according to the early warning level.

[0056] The modules in the aforementioned intelligent early warning device for drone patrols based on construction safety distances can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0057] The intelligent early warning device for drone patrol based on construction safety distance described in this embodiment of the invention can execute the intelligent early warning method for drone patrol based on construction safety distance provided in the above embodiment. The intelligent early warning device for drone patrol based on construction safety distance has the corresponding functional steps and beneficial effects of the intelligent early warning method for drone patrol based on construction safety distance described in the above embodiment. For details, please refer to the embodiment of the intelligent early warning method for drone patrol based on construction safety distance described above. The embodiments of the present invention will not be repeated here.

[0058] In one exemplary embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 4 As shown, the computer device includes a processor, memory, input / output interfaces, a communication interface, a display unit, and an input device. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interfaces. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface, such as a network interface card, is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, Near Field Communication (NFC), or other technologies. When the computer program is executed by the processor, it implements a congestion control method. The display unit is used to form a visually visible image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0059] Those skilled in the art will understand that Figure 4 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0060] In one exemplary embodiment, a chip is provided, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps of the method in any of the above embodiments.

[0061] In one exemplary embodiment, a network interface card is provided, including a chip as described in any of the above embodiments and multiple interfaces, wherein the chip communicates externally through the interfaces.

[0062] In one embodiment, a computer device is also provided, including a processor, a chip in any of the above embodiments, or a network interface card in any of the above embodiments, wherein the chip or the network interface card is used to schedule packets to the processor or the chip or the network interface card itself for processing, and the processor is used to process the packets scheduled by the chip or the network interface card.

[0063] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0064] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0065] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0066] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0067] Similarly, it should be understood that, for the purpose of simplification and aiding understanding of one or more aspects of the invention, various features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof in the description of exemplary embodiments of the invention above. Other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the disclosure herein. This application is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein. The specification and embodiments are to be considered exemplary only, and it should be noted that the above embodiments are illustrative of the invention and not restrictive, and that alternative embodiments can be devised by those skilled in the art without departing from its scope.< / esd>

Claims

1. A method for intelligent early warning of unmanned aerial vehicle (UAV) patrols based on construction safety distance, characterized in that, Includes the following steps: By using drones to collect video data in various construction site scenarios, an aerial image dataset of personnel and construction equipment is constructed, and pixel-level masking is performed on the personnel and construction equipment in the image dataset. An original backbone network is constructed, and an instance segmentation model is obtained by training the original backbone network based on the labeled image dataset. The drone collects video streams of the current construction site scene from the air following a preset flight path, which includes flight altitude, speed, and zoom parameters. The instance segmentation model is used to perform real-time analysis on the returned video stream, identify and segment personnel and construction equipment in the video stream, and obtain segmentation results. Based on the segmentation results, the minimum pixel distance between each person and the convex hull of each device is calculated, and converted into the actual physical distance according to the known reference size; Based on the actual physical distance and combined with a preset rule base, the effective safe distance threshold and risk score of the current construction site scenario are dynamically calculated. Based on the effective safe distance threshold and risk score, it is determined whether to trigger an early warning and the warning level. According to the warning level, the corresponding early warning response action is executed.

2. The intelligent early warning method for UAV patrol based on construction safety distance according to claim 1, characterized in that, The process of constructing the original backbone network and training the original backbone network to obtain the instance segmentation model based on the labeled image dataset includes: Based on the target-based instance segmentation model CenterMask, an original backbone network VoVNetV2 consisting of multiple OSA modules is constructed. A convolutional block attention module (CBAM) is inserted after the aggregation layer of each OSA module in the original backbone network VoVNetV2 to obtain the basic model. A scale prediction module, SizePredict, is added to the detection head of the base model. The SizePredict module is used to dynamically select the feature layer level of the Feature Pyramid Network (FPN) based on the target size in the image dataset. The SAG-MASK module is set in the base model. The SAG-MASK module is used to perform instance segmentation of the target in the base model, progressively upsample the region of interest feature map extracted by the RoI Align method to the target resolution, and use the PointRend method to refine the target segmentation edge upsampled to the target resolution. Add a boundary loss term to the loss function of the base model to calculate the boundary loss; The image dataset is divided into a training set and a validation set according to a predetermined ratio. After training the base model, the instance segmentation model is obtained.

3. The intelligent early warning method for UAV patrol based on construction safety distance according to claim 2, characterized in that, The size prediction module SizePredict dynamically selects the feature layer level of the Feature Pyramid Network (FPN) based on the target size in the image dataset. include: Define the feature layer level allocation rules of the Feature Pyramid Network (FPN), wherein the allocation rules characterize the feature layer level corresponding to targets of different sizes in the image dataset; The target size in the image dataset is determined by calculation; The feature layer level is dynamically selected based on the target size, and features from adjacent layers are fused for targets that are close to the feature layer level boundary.

4. The intelligent early warning method for UAV patrol based on construction safety distance according to claim 2, characterized in that, The progressive upsampling of the region of interest feature map extracted by the RoI Align method to the target resolution includes: Determine the target resolution based on the target characteristics; The feature map of the region of interest is extracted using the RoI Align method; The first feature map is obtained by upsampling the feature map of the region of interest using a deconvolution library function. The first feature map is then upsampled using the deconvolution library function, and this operation is repeated until the resolution of the feature map of the region of interest reaches the target resolution.

5. The intelligent early warning method for UAV patrol based on construction safety distance according to claim 1, characterized in that, The calculation of the minimum pixel distance between each person and the convex hull of each device, and the conversion to the actual physical distance based on the known reference dimensions, includes: Extract the convex hulls corresponding to each identified personnel contour and construction equipment contour to obtain the personnel convex hull point set and the equipment convex hull point set; Calculate the minimum pixel distance between the convex hull point set of each person and the convex hull point set of each device; Obtain the maximum pixel width Pw of the person's convex hull along the horizontal axis, and use the preset reference shoulder width value as the known reference size to calculate the scale factor Scale = reference shoulder width value / Pw; Multiplying the minimum pixel distance by the scale factor yields the actual physical distance between the person and the device.

6. The intelligent early warning method for UAV patrol based on construction safety distance according to claim 1, characterized in that, The dynamic calculation of the effective safe distance threshold for the current construction site scenario based on a preset rule base includes: The effective safe distance threshold for ESD is obtained by the following formula: ESD = BSR × SM; Wherein, BSR is the basic safety radius set in the preset rule base based on the type of construction equipment, and SM is the state coefficient set in the preset rule base based on the operating status of the construction equipment.

7. The intelligent early warning method for UAV patrol based on construction safety distance according to claim 6, characterized in that, The dynamic calculation of the risk score for the current construction site scenario based on a preset rule base includes: The risk score RS is obtained using the following formula: RS = (ESD / (d + S)) × BW; Where d is the actual physical distance, S is a constant, and BW is a behavior weighting factor set in the preset rule base based on the personnel's behavior status.

8. A drone-based intelligent early warning device for construction safety distance inspection, applied to the method described in any one of claims 1-7, characterized in that, The device includes: The data processing module is used to collect video data using drones in various construction site scenarios, construct an aerial image dataset of personnel and construction equipment, and perform pixel-level masking annotation on personnel and construction equipment in the image dataset. The model building module is used to build the original backbone network and train the original backbone network based on the labeled image dataset to obtain an instance segmentation model. The image data acquisition module is used to acquire video streams of the airspace above the current construction site scene via a drone following a preset flight path, wherein the flight path includes flight altitude, speed, and zoom parameters. The image segmentation module is used to perform real-time analysis on the returned video stream using the instance segmentation model, identify and segment personnel and construction equipment in the video stream, and obtain segmentation results. The distance calculation module is used to calculate the minimum pixel distance between the outline convex hull of each person and each device based on the segmentation results, and convert it into the actual physical distance according to the known reference size; The early warning processing module is used to dynamically calculate the effective safe distance threshold and risk score of the current construction site scene based on the actual physical distance and in combination with the preset rule base, and to determine whether to trigger an early warning and the early warning level based on the effective safe distance threshold and risk score, and to execute the corresponding early warning response action according to the early warning level.

9. A computer device comprising a processor and a memory, characterized in that, The memory stores computer program instructions that can be executed by the processor, and when the processor executes the computer program instructions, it implements the instructions of the method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer program instructions that are loaded and executed by a processor to perform the operations described in any one of claims 1-7.

Images