Construction site illegal behavior real-time detection method and system and airborne computing power terminal

Abstract

The invention discloses a construction site illegal behavior real-time detection method, a construction site illegal behavior real-time detection system and an airborne computing power terminal in the technical field of embedded edge computing and computer vision crossing. The method comprises the following steps: performing preliminary feature extraction on an obtained high-altitude construction site image by using a feature extraction backbone network to obtain a second enhanced feature, a third enhanced feature and a multi-scale feature; according to the third enhanced feature and the multi-scale feature, performing feature fusion enhancement by using a feature connection fusion module to obtain a fourth enhanced feature and an output of the feature connection fusion module; according to the output of the second enhanced feature, the multi-scale feature, the fourth enhanced feature and the feature connection fusion module, shallow, middle and deep feature extraction is performed by using a shallow feature detection branch, a middle feature detection branch and a deep feature detection branch to obtain shallow, middle and deep detection results; and non-maximum suppression operation based on intersection-to-union ratio is carried out on the three detection results to obtain a final target bounding box and an illegal behavior category.

Description

Technical Field

[0001] This invention relates to the field of embedded edge computing and computer vision, and in particular to a method, system and airborne computing terminal for real-time detection of construction site violations. Background Technology

[0002] With the acceleration of urbanization, the number and scale of construction sites continue to grow, posing increasing challenges to construction site safety management. Traditional safety management relies on manual inspections, which suffers from core defects such as poor real-time monitoring and low coverage, making it difficult to completely eliminate safety hazards. In recent years, the rapid development of drone and artificial intelligence technologies has provided new technological pathways for construction site safety monitoring. Drones, with their flexibility and maneuverability, can carry high-resolution cameras to acquire real-time images of construction sites from the air. Combined with artificial intelligence algorithms such as target detection, they hold promise for automatically detecting and issuing early warnings of violations on construction sites, thereby improving the level of intelligent safety management.

[0003] Currently, while drone technology offers a global perspective, existing air-to-ground monitoring solutions still face several technical challenges. For example, drone-borne terminals (such as embedded devices like Raspberry Pis) are limited by space, weight, power consumption, and computing resources, placing stringent lightweight requirements on the running algorithm models. However, the original models are highly complex, with a large number of parameters and computational demands, making direct deployment insufficient for real-time processing needs; targeted lightweight optimization of the models is required. Secondly, hardware integration presents numerous technical challenges. The collaborative operation of the drone platform and embedded computing units involves key aspects such as power management, data transmission, and physical interface adaptation: drone batteries are limited and must provide a stable and compatible power output for the embedded onboard terminal; image data needs to be transmitted at high speed and low latency between the drone and the computing unit.

[0004] Therefore, to address the aforementioned problems in existing construction site safety monitoring technologies, this invention proposes developing an air-to-ground behavior detection system based on embedded hardware and an optimized, high-efficiency target detection model. This system aims to achieve real-time and accurate detection of violations at construction sites, thus solving a pressing technical problem in the field of construction site safety management. (Invention Content)

[0005] The purpose of this invention is to overcome the shortcomings of the prior art and provide a method, system and airborne computing terminal for real-time detection of construction site violations. Through the integration of a lightweight target detection architecture and customized hardware, real-time detection of construction site violations can be achieved.

[0006] To solve the above-mentioned technical problems, the present invention is implemented using the following technical solution:

[0007] In a first aspect, the present invention provides a method for real-time detection of construction site violations, comprising:

[0008] Based on the acquired aerial images of the construction site, a trained lightweight object detection model is used to detect violations at the construction site.

[0009] The high-altitude construction site image is initially extracted using a feature extraction backbone network to obtain the second enhanced feature, the third enhanced feature, and multi-scale features.

[0010] Based on the third enhanced feature and the multi-scale feature, feature fusion enhancement is performed using the feature connection and fusion module to obtain the fourth enhanced feature and the output of the feature connection and fusion module.

[0011] Based on the output of the second enhanced feature and feature fusion module, shallow feature extraction is performed using the shallow feature detection branch to obtain the fifth enhanced feature and shallow detection result;

[0012] Based on the fourth and fifth enhancement features, the middle-layer feature extraction is performed using the middle-layer feature detection branch to obtain the sixth enhancement feature and the middle-layer detection result.

[0013] Based on the multi-scale features and the sixth enhanced feature, deep feature extraction is performed using the deep feature detection branch to obtain deep detection results;

[0014] The shallow, medium, and deep detection results are input into the nonmaximum suppression module to perform nonmaximum suppression based on cross-union ratio, thereby obtaining the final target bounding box and violation category.

[0015] Optionally, the preliminary feature extraction of the high-altitude construction site image using the feature extraction backbone network to obtain second enhanced features, third enhanced features, and multi-scale features includes:

[0016] The EcoConv convolution module is used to perform two feature extraction operations on the high-altitude construction site image to obtain the second convolution feature.

[0017] The second convolutional feature is enhanced using an attention enhancement module to obtain the first enhanced feature.

[0018] The EcoConv convolution module is used to perform a feature extraction operation on the first enhanced feature to obtain the third convolution feature.

[0019] The third convolutional feature is enhanced using an attention enhancement module to obtain the second enhanced feature.

[0020] The second enhanced feature is extracted once using the EcoConv convolution module to obtain the fourth convolution feature.

[0021] The fourth convolutional feature is enhanced using an attention enhancement module to obtain the third enhanced feature.

[0022] The third enhanced feature is extracted once using the EcoConv convolution module to obtain the fifth convolution feature.

[0023] The fifth convolutional feature is enhanced using an attention enhancement module to obtain the fourth enhanced feature;

[0024] The fourth enhanced feature is subjected to multi-scale max pooling operation using a multi-scale pooling module to obtain multi-scale features.

[0025] Optionally, the step of performing feature fusion enhancement using the feature concatenation and fusion module based on the third enhanced feature and multi-scale features to obtain the fourth enhanced feature and the output of the feature concatenation and fusion module includes:

[0026] The multi-scale features are upsampled and then concatenated with the third enhanced features to obtain the first concatenated feature.

[0027] The first concatenated feature is enhanced using an attention enhancement module to obtain a fourth enhanced feature.

[0028] The fourth enhanced feature is upsampled to obtain the output of the feature connection and fusion module.

[0029] Optionally, the step of extracting shallow features using the shallow feature detection branch based on the output of the second enhanced feature and feature fusion module to obtain the fifth enhanced feature and shallow detection result includes:

[0030] The output of the second enhanced feature is concatenated with the output of the feature connection and fusion module to obtain shallow concatenated features;

[0031] The shallow splicing features are enhanced using an attention enhancement module to obtain a fifth enhanced feature.

[0032] The fifth enhancement feature is input into the shallow detection head to obtain the shallow detection result;

[0033] The step of extracting mid-level features using the mid-level feature detection branch based on the fourth and fifth enhancement features to obtain the sixth enhancement feature and mid-level detection results includes:

[0034] The fifth enhancement feature is convolved once using the EcoConv convolution module and then concatenated with the fourth enhancement feature to obtain the middle layer concatenated feature;

[0035] The attention enhancement module is used to enhance the mid-layer splicing features to obtain the sixth enhanced feature;

[0036] The sixth enhanced feature is input into the middle layer detection head to obtain the middle layer detection result;

[0037] The step of extracting deep features using the deep feature detection branch based on the multi-scale features and the sixth enhanced feature to obtain deep detection results includes:

[0038] After performing a convolution operation on the sixth enhanced feature using the EcoConv convolution module, it is concatenated with the multi-scale feature to obtain the deep concatenated feature;

[0039] The deep splicing features are enhanced using an attention enhancement module to obtain the seventh enhanced feature;

[0040] The seventh enhanced feature is input into the deep detection head to obtain the deep detection result;

[0041] The shallow, medium, and deep detection heads each include two EcoConv convolutional modules, used for detecting bounding boxes and violation categories, respectively.

[0042] Optionally, the EcoConv convolutional module includes a convolutional layer Conv, a depthwise convolutional layer DWConv, a splicing layer, a batch normalization layer, and a ReLU activation layer;

[0043] The data processing flow of the EcoConv convolutional module includes:

[0044] Based on the input of the EcoConv convolutional module, intrinsic features are extracted using the Conv convolutional layer to obtain... An intrinsic feature map;

[0045] Deep feature extraction is performed on each intrinsic feature map using deep convolutional layers to obtain... A number of derived feature maps; among them... This represents the total number of derived feature maps for each intrinsic feature map;

[0046] Using splicing layers An intrinsic feature map and The derived feature maps are concatenated to obtain the concatenated feature map. ;

[0047] The stitched feature map is processed using a batch normalization layer. Perform batch normalization to obtain the batch normalization result. ;

[0048] The batch normalization results are processed using a ReLU activation layer. Perform non-linear activation to obtain the output of the EcoConv convolutional module;

[0049] The intrinsic feature map is obtained by the following formula:

[0050] ,

[0051] in, Represents the set of intrinsic feature maps. This represents the first intrinsic feature map. This represents the second intrinsic feature map. Indicates the first An intrinsic feature map, This represents the convolution operation. This represents the input to the EcoConv convolutional module;

[0052] The derived feature map is obtained through the following formula:

[0053] ,

[0054] in, Indices representing the index of the intrinsic feature map, Indicates the index of the derived feature map. This represents a depthwise convolution operation. Indicates the first An intrinsic feature map, Indicates the first The first intrinsic feature map A derived feature map;

[0055] The splicing feature map It can be obtained through the following formula:

[0056] ,

[0057] in, This indicates a splicing operation. This represents the first derived feature map of the first intrinsic feature map. This represents the second derived feature map of the first intrinsic feature map. Represents the first intrinsic feature map. A derived feature map, This represents the first derived feature map of the second intrinsic feature map. Indicates the first The first intrinsic feature map A derived feature map;

[0058] The expression for batch normalization is as follows:

[0059] ,

[0060] in, Indicates the batch normalization result The One characteristic, and For learnable parameters, Represents the splicing feature map The One characteristic, Represents the splicing feature map The mean, , Represents the splicing feature map variance , This represents a preset parameter to prevent the denominator from being zero.

[0061] The expression for the ReLU activation layer is as follows:

[0062] ,

[0063] in, This represents the output of the EcoConv convolutional module. Represents the ReLU activation function. This indicates the operation of selecting the maximum value.

[0064] Optionally, the attention enhancement module includes a spatial attention branch, a channel attention branch, and a fusion module;

[0065] The expression for the spatial attention branch is as follows:

[0066] ,

[0067] in, This represents the output of the spatial attention branch. This represents the input to the attention enhancement module. This represents the softmax activation function. This represents the sigmoid activation function. express Convolution operation, Indicates an upsampling operation. This represents the shallow, high-resolution features of the backbone network.

[0068] The expression for the channel attention branch is as follows:

[0069] ,

[0070] in, This represents the activation function. This represents the output of the channel attention branch. This indicates a two-layer fully connected operation. Indicates global average pooling. Indicates global max pooling;

[0071] The expression for the fusion module is as follows:

[0072] ,

[0073] in, This represents the output of the attention enhancement module. and These are learnable parameters.

[0074] Optionally, the expression for the multi-scale pooling module is as follows:

[0075] ,

[0076] in, This represents the output of the multi-scale pooling module. This represents the EcoConv convolutional module. This indicates a splicing operation. This indicates a max pooling operation. This represents the input to the multi-scale pooling module.

[0077] Optionally, the trained lightweight target detection model is subjected to 8-bit integer quantization to compress the model weights from 32-bit floating-point numbers to 8-bit integers.

[0078] The 8-bit integer quantization operation is implemented using the following formula:

[0079] ,

[0080] in, Represents the quantized integer. This indicates the rounding operation. Represents floating-point numbers. Indicates the scaling factor. It represents zero point.

[0081] Secondly, the present invention provides a real-time detection system for construction site violations, comprising:

[0082] The feature extraction backbone module is used to: perform preliminary feature extraction on the high-altitude construction site image using the feature extraction backbone network to obtain the second enhanced feature, the third enhanced feature, and the multi-scale feature;

[0083] The feature fusion module is used to: perform feature fusion enhancement based on the third enhanced feature and the multi-scale feature, and obtain the fourth enhanced feature and the output of the feature fusion module;

[0084] The shallow feature extraction module is used to: extract shallow features using the shallow feature detection branch based on the output of the second enhanced feature and feature fusion module, and obtain the fifth enhanced feature and shallow detection result;

[0085] The intermediate feature extraction module is used to: extract intermediate features based on the fourth and fifth enhanced features using the intermediate feature detection branch to obtain the sixth enhanced feature and the intermediate detection result;

[0086] The deep feature extraction module is used to: extract deep features using the deep feature detection branch based on the multi-scale features and the sixth enhanced features, and obtain deep detection results;

[0087] The non-maximum suppression module is used to: input the shallow detection results, middle detection results and deep detection results into the non-maximum suppression module, perform non-maximum suppression operation based on cross-union ratio, and obtain the final target bounding box and violation category.

[0088] Thirdly, the present invention provides an airborne computing terminal, comprising: an E-Port adapter board, a serial port module, a network port module, and a development board;

[0089] The airborne computing terminal is connected to the UAV via an E-Port adapter board to acquire real-time aerial construction site data captured by the UAV.

[0090] The serial port module connects the E-Port adapter board and the development board, and is used to transmit and receive UAV status data, control commands and violation detection results between the E-Port adapter board and the development board;

[0091] The network port module connects the E-Port adapter board and the development board, and is used to realize the transmission of video stream data captured by the drone in real time between the E-Port adapter board and the development board;

[0092] The development board is equipped with a lightweight target detection model to implement the real-time detection method for construction site violations as described in any of the first aspects.

[0093] Compared with existing technologies, the beneficial effects achieved by this invention are as follows:

[0094] 1. A real-time detection method for construction site violations is proposed. By introducing the EcoConv high-efficiency convolution module, the computational complexity and number of parameters of the model are significantly reduced. At the same time, by combining the attention enhancement module and the multi-scale feature fusion mechanism, the model parameters are reduced while maintaining high detection accuracy. It is particularly suitable for detecting violations of multi-scale targets and complex background interference in construction site scenes from a high-altitude top view.

[0095] 2. A real-time detection system for construction site violations is proposed. By setting up a feature extraction backbone module, a feature connection and fusion module, a shallow feature extraction module, a mid-level feature extraction module, a deep feature extraction module, and a non-maximum suppression module, a complete multi-scale feature extraction and detection process is formed. It can simultaneously take into account the accurate perception of small targets and the semantic judgment of large targets, which significantly improves the robustness and real-time performance of the system for detecting various construction site violations and has good application prospects.

[0096] 3. An airborne computing terminal is proposed. Through the customized E-Port adapter board and the collaborative design of serial port and network port modules, a seamless physical connection and stable data interaction with the UAV platform are achieved, supporting low-latency transmission of video streams and control commands. The terminal is lightweight, low-power, easy to install, and has an automatic startup function, providing a reliable hardware carrier for intelligent air-to-ground construction site supervision. The development board is equipped with a lightweight target detection model, realizing real-time inference deployment on embedded devices. Attached Figure Description

[0097] Figure 1 This is a structural diagram of a lightweight target detection model provided according to an embodiment of the present invention;

[0098] Figure 2 This is a structural diagram of a lightweight multi-scale pooling module provided according to an embodiment of the present invention;

[0099] Figure 3 This is a diagram showing the detection results of construction site violations according to an embodiment of the present invention.

[0100] Figure 4 This is a structural diagram of an airborne computing terminal hardware device provided according to an embodiment of the present invention;

[0101] Figure 5 This is a flowchart of a method for real-time detection of construction site violations using an airborne computing terminal, provided by an embodiment of the present invention. Figure 6 This is a structural diagram of the EcoConv convolutional module provided according to an embodiment of the present invention. Detailed Implementation

[0102] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments and specific features in the embodiments are detailed descriptions of the technical solution of the present application, rather than limitations thereof. In the absence of conflict, the embodiments and technical features in the embodiments can be combined with each other.

[0103] It should be noted that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0104] Example 1:

[0105] This invention discloses a method for real-time detection of construction site violations, which specifically includes the following steps:

[0106] Based on the acquired aerial images of the construction site, a trained lightweight object detection model is used to detect violations at the construction site.

[0107] S1, the feature extraction backbone network is used to perform preliminary feature extraction on the high-altitude construction site image to obtain the second enhanced feature, the third enhanced feature and the multi-scale feature;

[0108] S2, based on the third enhanced feature and the multi-scale feature, feature fusion enhancement is performed using the feature connection and fusion module to obtain the fourth enhanced feature and the output of the feature connection and fusion module;

[0109] S3, based on the output of the second enhanced feature and feature connection fusion module, shallow feature extraction is performed using the shallow feature detection branch to obtain the fifth enhanced feature and shallow detection result;

[0110] S4, based on the fourth and fifth enhancement features, perform mid-level feature extraction using the mid-level feature detection branch to obtain the sixth enhancement feature and mid-level detection result;

[0111] S5, based on the multi-scale features and the sixth enhanced features, deep feature extraction is performed using the deep feature detection branch to obtain the deep detection result;

[0112] S6. Input the shallow detection results, middle detection results and deep detection results into the non-maximum suppression module to perform non-maximum suppression operation based on cross-union ratio to obtain the final target bounding box and violation category.

[0113] To better achieve object detection with limited computing power, this embodiment constructs a lightweight object detection model, referencing... Figure 1 As shown, the model includes a feature extraction backbone network, a feature fusion module, a shallow feature detection branch, a mid-level feature detection branch, a deep feature detection branch, and a non-maximum suppression module.

[0114] In the feature maps generated by typical CNN convolutional modules, there is a lot of redundancy, and many feature maps are very similar to each other. Therefore, this embodiment specifically designs an EcoConv convolutional module to replace the general convolution operation. Unlike ordinary convolution, the EcoConv convolutional module does not perform a complete and expensive convolution operation for each output channel, but only calculates a small portion of the feature maps. In this embodiment, the EcoConv convolutional module is applied to various structures of the lightweight object detection model, which greatly improves the overall computational efficiency of the model.

[0115] Specifically, the EcoConv convolutional module includes one convolutional layer (Conv), one depthwise convolutional layer (DWConv), one concatenation layer, one batch normalization layer, and one ReLU activation layer. For detailed connection relationships, please refer to [reference needed]. Figure 6 As shown, its workflow includes:

[0116] Based on the input of the EcoConv convolutional module, intrinsic features are extracted using the Conv convolutional layer to obtain... One intrinsic feature map:

[0117] ,

[0118] in, Represents the set of intrinsic feature maps. This represents the first intrinsic feature map. This represents the second intrinsic feature map. Indicates the first An intrinsic feature map, This represents a convolution operation using a regular convolution kernel. This represents the input to the EcoConv convolutional module, with channels. , ;

[0119] Deep feature extraction is performed by applying a series of inexpensive and fast linear operations to each intrinsic feature map using deep convolutional layers. Derived feature map:

[0120] ,

[0121] in, Indices representing the index of the intrinsic feature map, Indicates the index of the derived feature map. This indicates a depthwise convolution operation (DWConv, Depthwise Convolution). Indicates the first An intrinsic feature map, Indicates the first The first intrinsic feature map A derived feature map; This represents the total number of derived feature maps for each intrinsic feature map. DepthwiseConv performs spatial convolution independently on each input channel, which is much less computationally expensive than ordinary convolution. Linear operations (such as DW Conv) are very efficient and can simulate spatial transformations between feature maps (such as smoothing and edge extraction). These transformations can approximate the generation process of redundant feature maps.

[0122] Using splicing layers An intrinsic feature map and The derived feature maps are concatenated to obtain the concatenated feature map. :

[0123] ,

[0124] in, This indicates a splicing operation. This represents the first derived feature map of the first intrinsic feature map. This represents the second derived feature map of the first intrinsic feature map. Represents the first intrinsic feature map. A derived feature map, This represents the first derived feature map of the second intrinsic feature map. Indicates the first The first intrinsic feature map EcoConv significantly reduces the number of parameters and computational cost because most feature maps are obtained through linear transformation. Furthermore, experiments have shown that replacing ordinary convolutional modules results in only a minimal decrease in accuracy.

[0125] The stitched feature map is processed using a batch normalization layer. Perform batch normalization to obtain the batch normalization result. :

[0126] ,

[0127] in, Indicates the batch normalization result The One characteristic, and For learnable parameters, Represents the splicing feature map The One characteristic, Represents the splicing feature map The mean, , Represents the splicing feature map variance , This represents a preset parameter to prevent the denominator from being zero; batch normalization layers can accelerate the convergence speed during model training, making the model training process more stable and avoiding gradient explosion or gradient vanishing.

[0128] The batch normalization results are processed using a ReLU activation layer. After performing non-linear activation, the output of the EcoConv convolutional module is obtained:

[0129] ,

[0130] in, This represents the output of the EcoConv convolutional module. Represents the ReLU activation function. It represents the operation of selecting the maximum value; compared with other activation functions, the ReLU function has a simple mathematical form and is easy to calculate.

[0131] This embodiment also incorporates attention enhancement modules in various structures of the model to enhance features, aiming to improve the model's performance in multi-scale feature extraction. The attention enhancement module combines spatial-channel dual attention with feature fusion, specifically designed for addressing small targets and occlusion issues in complex construction site scenes. The attention enhancement module includes a spatial attention branch, a channel attention branch, and a fusion module; the expression for the spatial attention branch is as follows:

[0132] ,

[0133] in, This represents the output of the spatial attention branch. This represents the input to the attention enhancement module. This represents the softmax activation function. This represents the sigmoid activation function. express Convolution operation, Indicates an upsampling operation. This represents the shallow high-resolution features of the backbone network. The shallow high-resolution features guide the generation of a spatial weight matrix, which enhances the spatial localization of small targets. After element-wise multiplication and Softmax spatial normalization, dynamic focusing on the small target region is achieved.

[0134] The expression for the channel attention branch is as follows:

[0135] ,

[0136] in, This represents the activation function. This represents the output of the channel attention branch. This indicates a two-layer fully connected operation. Indicates global average pooling. This indicates global max pooling, with the channel attention branch used to detect the target category;

[0137] The expression for the fusion module is as follows:

[0138] ,

[0139] in, This represents the output of the attention enhancement module. and These are learnable parameters.

[0140] In step S1, a feature extraction backbone network is used to perform preliminary feature extraction on the high-altitude construction site image, resulting in second enhanced features, third enhanced features, and multi-scale features, including:

[0141] The EcoConv convolution module is used to perform two feature extraction operations on the high-altitude construction site image to obtain the second convolution feature.

[0142] The second convolutional feature is enhanced using an attention enhancement module to obtain the first enhanced feature.

[0143] The EcoConv convolution module is used to perform a feature extraction operation on the first enhanced feature to obtain the third convolution feature.

[0144] The third convolutional feature is enhanced using an attention enhancement module to obtain the second enhanced feature.

[0145] The second enhanced feature is extracted once using the EcoConv convolution module to obtain the fourth convolution feature.

[0146] The fourth convolutional feature is enhanced using an attention enhancement module to obtain the third enhanced feature.

[0147] The third enhanced feature is extracted once using the EcoConv convolution module to obtain the fifth convolution feature.

[0148] The fifth convolutional feature is enhanced using an attention enhancement module to obtain the fourth enhanced feature;

[0149] The fourth enhanced feature is subjected to multi-scale max pooling operation using a multi-scale pooling module to obtain multi-scale features.

[0150] After stacking EcoConv and attention enhancement modules, complex patterns are learned through multi-layer nonlinear transformations, gradually abstracting features from low to high levels. Finally, pooling layers are used to perform pooling operations on the image. Due to the high-altitude overhead view, some construction site violations, such as not wearing safety helmets, are extremely small in scale. Traditional single-scale pooling easily loses details, and the fixed receptive field is difficult to adapt to changes in target scale. Therefore, this embodiment designs a multi-scale pooling module, referencing... Figure 2 As shown, by performing max pooling operations at different scales on the input feature map, feature maps of multiple scales are obtained. These feature maps are then concatenated to form a feature vector containing multi-scale information. This feature vector contains both the spatial information of the original feature map and feature information at different scales, thereby improving the richness and representativeness of the features. By concatenating multiple small-sized pooling kernels, the effect of parallel large-sized pooling kernels is equivalently simulated. Furthermore, the EcoConv convolution module mentioned above is used to replace the convolution operation, further reducing the parameters required for computation. The expression of the multi-scale pooling module is as follows:

[0151] ,

[0152] in, This represents the output of the multi-scale pooling module. This represents the EcoConv convolutional module. This indicates a splicing operation. This indicates a max pooling operation. This represents the input to the multi-scale pooling module.

[0153] In step S2, based on the third enhanced feature and the multi-scale feature, feature fusion enhancement is performed using the feature concatenation and fusion module to obtain the fourth enhanced feature and the output of the feature concatenation and fusion module, including:

[0154] The multi-scale features are upsampled and then concatenated with the third enhanced features to obtain the first concatenated feature.

[0155] The first concatenated feature is enhanced using an attention enhancement module to obtain a fourth enhanced feature.

[0156] The fourth enhanced feature is upsampled to obtain the output of the feature connection and fusion module.

[0157] The lightweight target detection model in this embodiment adopts a multi-scale prediction architecture. By designing shallow feature detection branches, mid-level feature detection branches, and deep feature detection branches, it addresses the challenge of huge differences in the target scale of illegal activities at construction sites from an aerial perspective. The three detection heads correspond to feature layers of different depths in the feature extraction backbone network and are used to detect targets of different scales.

[0158] In step S3, the shallow feature detection branch is responsible for sensing and locating small-scale targets in the image, such as those not wearing helmets or smoking. This detection head utilizes high-resolution shallow feature maps from the feature pyramid network, which are rich in spatial detail and beneficial for accurate location regression of small targets. Based on the output of the second enhanced feature and feature fusion module, shallow feature extraction is performed using the shallow feature detection branch to obtain the fifth enhanced feature and shallow detection results, including:

[0159] The output of the second enhanced feature is concatenated with the output of the feature connection and fusion module to obtain shallow concatenated features;

[0160] The shallow splicing features are enhanced using an attention enhancement module to obtain a fifth enhanced feature.

[0161] The fifth enhanced feature is input into the shallow detection head to obtain the shallow detection result.

[0162] In step S4, the mid-level feature detection branch is responsible for detecting medium-scale targets, such as construction workers fighting. This layer achieves a good balance between spatial detail and semantic information. Based on the fourth and fifth enhancement features, the mid-level feature detection branch is used to extract mid-level features, resulting in the sixth enhancement feature and mid-level detection results, including:

[0163] The fifth enhancement feature is convolved once using the EcoConv convolution module and then concatenated with the fourth enhancement feature to obtain the middle layer concatenated feature;

[0164] The attention enhancement module is used to enhance the mid-layer splicing features to obtain the sixth enhanced feature;

[0165] The sixth enhanced feature is input into the middle layer detection head to obtain the middle layer detection result.

[0166] In step S5, the deep feature detection branch is responsible for detecting large-scale targets, such as uncovered bare soil areas; the deep detection head utilizes highly abstracted feature maps, which are more accurate in classifying large targets; based on the multi-scale features and the sixth enhanced feature, the deep feature detection branch performs deep feature extraction to obtain deep detection results, including:

[0167] After performing a convolution operation on the sixth enhanced feature using the EcoConv convolution module, it is concatenated with the multi-scale feature to obtain the deep concatenated feature;

[0168] The deep splicing features are enhanced using an attention enhancement module to obtain the seventh enhanced feature;

[0169] The seventh enhanced feature is input into the deep detection head to obtain the deep detection result.

[0170] In this embodiment, the shallow detection head, the middle detection head, and the deep detection head each include two EcoConv convolutional modules, which are used to detect bounding boxes and violation categories, respectively.

[0171] In step S6, the predicted bounding boxes generated by the three detectors undergo non-maximum suppression (NMS) based on intersection-over-union (IoU) before being sent to the final output. This ensures that only one optimal predicted bounding box is retained for the same target. The specific steps are as follows:

[0172] First, the confidence scores of all predicted boxes generated by all detection heads are sorted in descending order. Then, iterative selection and suppression are performed, selecting the predicted box with the highest confidence score as the baseline and adding it to the final output list. Subsequently, the IoU between the baseline box and all other predicted boxes in the list is calculated. If the IoU value of a box exceeds a preset threshold, it is considered a redundant detection of the same target and is removed. The above selection and suppression steps are repeated for the remaining predicted boxes until all boxes have been processed.

[0173] After NMS processing, redundant prediction boxes that are highly overlapping in spatial location are effectively suppressed, and only the prediction box with the highest confidence and the corresponding violation category are retained.

[0174] During the training of the lightweight object detection model, the optimizer used is AdamW, and the initial learning rate is set to 1. The training employs a cosine annealing scheduler for learning rate decay, and focuses on monitoring the loss of small targets in each epoch to ensure stable convergence. A multi-GPU training strategy with a batch size of 16 per GPU is used, combined with a gradient accumulation mechanism to address memory limitations. To prevent overfitting, detection accuracy is evaluated on the validation set every 10 epochs. If the validation metrics do not improve for five consecutive epochs, an early stopping mechanism is triggered, ensuring both efficient and robust model training.

[0175] In this embodiment, after the lightweight target detection model is trained, an 8-bit integer quantization operation is performed on the model to further reduce the model parameters and improve the detection speed. The essence of quantization is to compress the model weights / activation values ​​from 32-bit floating-point (FP32) to 8-bit integer (INT8).

[0176] ,

[0177] in, Represents the quantized integer. This indicates the rounding operation. Represents floating-point numbers. This represents the scaling factor. It represents the zero point.

[0178] The FP32 model trained with aerial construction site data is used as the basis for quantization. After quantization, the model is calibrated to construct a calibration dataset. The calibration dataset contains 2,000 aerial construction site images, covering various lighting conditions and weather scenarios, to ensure that the quantization parameters are adapted to the actual business environment.

[0179] Example 2:

[0180] Based on the same inventive concept as Embodiment 1, this embodiment of the invention discloses a real-time detection system for construction site violations, comprising:

[0181] The feature extraction backbone module is used to: perform preliminary feature extraction on the high-altitude construction site image using the feature extraction backbone network to obtain the second enhanced feature, the third enhanced feature, and the multi-scale feature;

[0182] The feature fusion module is used to: perform feature fusion enhancement based on the third enhanced feature and the multi-scale feature, and obtain the fourth enhanced feature and the output of the feature fusion module;

[0183] The shallow feature extraction module is used to: extract shallow features using the shallow feature detection branch based on the output of the second enhanced feature and feature fusion module, and obtain the fifth enhanced feature and shallow detection result;

[0184] The intermediate feature extraction module is used to: extract intermediate features based on the fourth and fifth enhanced features using the intermediate feature detection branch to obtain the sixth enhanced feature and the intermediate detection result;

[0185] The deep feature extraction module is used to: extract deep features using the deep feature detection branch based on the multi-scale features and the sixth enhanced features, and obtain deep detection results;

[0186] The non-maximum suppression module is used to: input the shallow detection results, middle detection results and deep detection results into the non-maximum suppression module, perform non-maximum suppression operation based on cross-union ratio, and obtain the final target bounding box and violation category.

[0187] The specific functions of each module described above are explained in the relevant content of the method in Embodiment 1, and will not be repeated here.

[0188] Example 3:

[0189] Based on the same inventive concept as Embodiment 1, this embodiment of the invention discloses an airborne computing terminal, with reference to... Figure 4 As shown, it includes: an E-Port adapter board, a serial port module, a network port module, and a development board;

[0190] The airborne computing terminal is connected to the UAV via an E-Port adapter board to acquire real-time aerial construction site data captured by the UAV.

[0191] The serial port module connects the E-Port adapter board and the development board, and is used to transmit and receive UAV status data, control commands and violation detection results between the E-Port adapter board and the development board;

[0192] The network port module connects the E-Port adapter board and the development board, and is used to realize the transmission of video stream data captured by the drone in real time between the E-Port adapter board and the development board;

[0193] The development board is equipped with a lightweight target detection model, which is used to implement the real-time detection method for construction site violations as described in any of the embodiments.

[0194] Specifically, the airborne computing terminal consists of an E-Port adapter board, a serial port module, a network port module, and a Raspberry Pi 4b development board. The drone connects to the E-Port adapter board, while the E-Port adapter board sends and receives data with the development board via the serial port module. Simultaneously, the drone's video stream is transmitted to the development board via the network card module. The terminal integrates DJI's official SDK, customizing and optimizing the communication protocol between the drone and the terminal: it adopts a frame-synchronized data packet transmission mechanism, attaching a timestamp and checksum to each frame of video data. After receiving the data, the terminal quickly verifies its integrity using the checksum. If packet loss is detected, an immediate retransmission request is triggered to ensure the continuity of the video stream. Simultaneously, the data parsing process is optimized, coordinating the video decoding module in the SDK with the terminal's local hardware decoding unit to reduce the time spent from receiving to decoding video frames, keeping the single-frame processing latency below 50ms, meeting the low-latency requirements of real-time monitoring.

[0195] The airborne computing terminal has a built-in detection program that automatically starts when connected to the DJI drone's E-Port interface. It's a simple, one-click operation that's easy to learn. It also uses a bracket and nut structure, making the terminal weigh only 200g. Installation is simple: the nut secures it to the drone's fuselage through pre-drilled holes, and a single person can complete the installation in under 30 seconds.

[0196] refer to Figure 3 The image shown is a result of the detection of violations at the construction site. When violations are detected, the detection results meet the following conditions: confidence level ≥ 0.6, violation duration ≥ 5 frames, and target pixel size ≥ 15*15 pixels. The terminal then automatically operates the drone via the DJI SDK to take photos and transmit the data back to the DJI cloud platform and the control terminal. Figure 5As shown. Compared to the single-frame detection scheme, this scheme significantly reduces the false trigger rate. If the drone is equipped with a loudspeaker or lighting module, it can warn the violator when a violation is detected.

[0197] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0198] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0199] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0200] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0201] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims. All of these forms are within the protection scope of the present invention.

Claims

1. A method for real-time detection of construction site violations, characterized in that, include: Based on the acquired aerial images of the construction site, a trained lightweight object detection model is used to detect violations at the construction site. The high-altitude construction site image is initially extracted using a feature extraction backbone network to obtain the second enhanced feature, the third enhanced feature, and multi-scale features. Based on the third enhanced feature and the multi-scale feature, feature fusion enhancement is performed using the feature connection and fusion module to obtain the fourth enhanced feature and the output of the feature connection and fusion module. Based on the output of the second enhanced feature and feature fusion module, shallow feature extraction is performed using the shallow feature detection branch to obtain the fifth enhanced feature and shallow detection result; Based on the fourth and fifth enhancement features, the middle-layer feature extraction is performed using the middle-layer feature detection branch to obtain the sixth enhancement feature and the middle-layer detection result. Based on the multi-scale features and the sixth enhanced feature, deep feature extraction is performed using the deep feature detection branch to obtain deep detection results; The shallow, medium, and deep detection results are input into the nonmaximum suppression module to perform nonmaximum suppression based on cross-union ratio, thereby obtaining the final target bounding box and violation category.

2. The real-time detection method for construction site violations according to claim 1, characterized in that, The preliminary feature extraction of the high-altitude construction site image using the feature extraction backbone network yields second enhanced features, third enhanced features, and multi-scale features, including: The EcoConv convolution module is used to perform two feature extraction operations on the high-altitude construction site image to obtain the second convolution feature. The second convolutional feature is enhanced using an attention enhancement module to obtain the first enhanced feature. The EcoConv convolution module is used to perform a feature extraction operation on the first enhanced feature to obtain the third convolution feature. The third convolutional feature is enhanced using an attention enhancement module to obtain the second enhanced feature. The second enhanced feature is extracted once using the EcoConv convolution module to obtain the fourth convolution feature. The fourth convolutional feature is enhanced using an attention enhancement module to obtain the third enhanced feature. The third enhanced feature is extracted once using the EcoConv convolution module to obtain the fifth convolution feature. The fifth convolutional feature is enhanced using an attention enhancement module to obtain the fourth enhanced feature; The fourth enhanced feature is subjected to multi-scale max pooling operation using a multi-scale pooling module to obtain multi-scale features.

3. The real-time detection method for construction site violations according to claim 1, characterized in that, The step of performing feature fusion enhancement based on the third enhanced feature and multi-scale features using the feature concatenation and fusion module to obtain the fourth enhanced feature and the output of the feature concatenation and fusion module includes: The multi-scale features are upsampled and then concatenated with the third enhanced features to obtain the first concatenated feature. The first concatenated feature is enhanced using an attention enhancement module to obtain a fourth enhanced feature. The fourth enhanced feature is upsampled to obtain the output of the feature connection and fusion module.

4. The real-time detection method for construction site violations according to claim 1, characterized in that, The step involves extracting shallow features using a shallow feature detection branch based on the output of the second enhanced feature and feature fusion module, to obtain the fifth enhanced feature and shallow detection results, including: The output of the second enhanced feature is concatenated with the output of the feature connection and fusion module to obtain shallow concatenated features; The shallow splicing features are enhanced using an attention enhancement module to obtain a fifth enhanced feature. The fifth enhancement feature is input into the shallow detection head to obtain the shallow detection result; The step of extracting mid-level features using the mid-level feature detection branch based on the fourth and fifth enhancement features to obtain the sixth enhancement feature and mid-level detection results includes: The fifth enhancement feature is convolved once using the EcoConv convolution module and then concatenated with the fourth enhancement feature to obtain the middle layer concatenated feature; The attention enhancement module is used to enhance the mid-layer splicing features to obtain the sixth enhanced feature; The sixth enhanced feature is input into the middle layer detection head to obtain the middle layer detection result; The step of extracting deep features using the deep feature detection branch based on the multi-scale features and the sixth enhanced feature to obtain deep detection results includes: After performing a convolution operation on the sixth enhanced feature using the EcoConv convolution module, it is concatenated with the multi-scale feature to obtain the deep concatenated feature; The deep splicing features are enhanced using an attention enhancement module to obtain the seventh enhanced feature; The seventh enhanced feature is input into the deep detection head to obtain the deep detection result; The shallow, medium, and deep detection heads each include two EcoConv convolutional modules, used for detecting bounding boxes and violation categories, respectively.

5. The real-time detection method for construction site violations according to claim 2, characterized in that, The EcoConv convolutional module includes a convolutional layer Conv, a deep convolutional layer DWConv, a splicing layer, a batch normalization layer, and a ReLU activation layer; The data processing flow of the EcoConv convolutional module includes: Based on the input of the EcoConv convolutional module, intrinsic features are extracted using the Conv convolutional layer to obtain... An intrinsic feature map; Deep feature extraction is performed on each intrinsic feature map using deep convolutional layers to obtain... A number of derived feature maps; among them... This represents the total number of derived feature maps for each intrinsic feature map; Using splicing layers An intrinsic feature map and The derived feature maps are concatenated to obtain the concatenated feature map. ; The stitched feature map is processed using a batch normalization layer. Perform batch normalization to obtain the batch normalization result. ; The batch normalization results are processed using a ReLU activation layer. Perform non-linear activation to obtain the output of the EcoConv convolutional module; The intrinsic feature map is obtained by the following formula: ， in, Represents the set of intrinsic feature maps. This represents the first intrinsic feature map. This represents the second intrinsic feature map. Indicates the first An intrinsic feature map, This represents the convolution operation. This represents the input to the EcoConv convolutional module; The derived feature map is obtained through the following formula: ， in, Indices representing the index of the intrinsic feature map, Indicates the index of the derived feature map. This represents a depthwise convolution operation. Indicates the first An intrinsic feature map, Indicates the first The first intrinsic feature map A derived feature map; The splicing feature map It can be obtained through the following formula: ， in, This indicates a splicing operation. This represents the first derived feature map of the first intrinsic feature map. This represents the second derived feature map of the first intrinsic feature map. Represents the first intrinsic feature map. A derived feature map, This represents the first derived feature map of the second intrinsic feature map. Indicates the first The first intrinsic feature map A derived feature map; The expression for batch normalization is as follows: ， in, Indicates the batch normalization result The One characteristic, and For learnable parameters, Represents the splicing feature map The One characteristic, Represents the splicing feature map The mean, , Represents the splicing feature map variance , This represents a preset parameter to prevent the denominator from being zero. The expression for the ReLU activation layer is as follows: ， in, This represents the output of the EcoConv convolutional module. Represents the ReLU activation function. This indicates the operation of selecting the maximum value.

6. The real-time detection method for construction site violations according to claim 2, characterized in that, The attention enhancement module includes a spatial attention branch, a channel attention branch, and a fusion module; The expression for the spatial attention branch is as follows: ， in, This represents the output of the spatial attention branch. This represents the input to the attention enhancement module. This represents the softmax activation function. This represents the sigmoid activation function. express Convolution operation, Indicates an upsampling operation. This represents the shallow, high-resolution features of the backbone network. The expression for the channel attention branch is as follows: ， in, This represents the activation function. This represents the output of the channel attention branch. This indicates a two-layer fully connected operation. Indicates global average pooling. Indicates global max pooling; The expression for the fusion module is as follows: ， in, This represents the output of the attention enhancement module. and These are learnable parameters.

7. The real-time detection method for construction site violations according to claim 2, characterized in that, The expression for the multi-scale pooling module is as follows: ， in, This represents the output of the multi-scale pooling module. This represents the EcoConv convolutional module. This indicates a splicing operation. This indicates a max pooling operation. This represents the input to the multi-scale pooling module.

8. The method for real-time detection of construction site violations according to claim 1, characterized in that, The trained lightweight object detection model is subjected to 8-bit integer quantization to compress the model weights from 32-bit floating-point numbers to 8-bit integers. The 8-bit integer quantization operation is implemented using the following formula: ， in, Represents the quantized integer. This indicates the rounding operation. Represents floating-point numbers. Indicates the scaling factor. It represents zero point.

9. A real-time detection system for construction site violations, characterized in that, include: The feature extraction backbone module is used to: perform preliminary feature extraction on the high-altitude construction site image using the feature extraction backbone network to obtain the second enhanced feature, the third enhanced feature, and the multi-scale feature; The feature fusion module is used to: perform feature fusion enhancement based on the third enhanced feature and the multi-scale feature, and obtain the fourth enhanced feature and the output of the feature fusion module; The shallow feature extraction module is used to: extract shallow features using the shallow feature detection branch based on the output of the second enhanced feature and feature fusion module, and obtain the fifth enhanced feature and shallow detection result; The intermediate feature extraction module is used to: extract intermediate features based on the fourth and fifth enhanced features using the intermediate feature detection branch to obtain the sixth enhanced feature and the intermediate detection result; The deep feature extraction module is used to: extract deep features using the deep feature detection branch based on the multi-scale features and the sixth enhanced features, and obtain deep detection results; The non-maximum suppression module is used to: input the shallow detection results, middle detection results and deep detection results into the non-maximum suppression module, perform non-maximum suppression operation based on cross-union ratio, and obtain the final target bounding box and violation category.

10. An airborne computing terminal, characterized in that, include: E-Port adapter board, serial port module, network port module and development board; The airborne computing terminal is connected to the UAV via an E-Port adapter board to acquire real-time aerial construction site data captured by the UAV. The serial port module connects the E-Port adapter board and the development board, and is used to transmit and receive UAV status data, control commands and violation detection results between the E-Port adapter board and the development board; The network port module connects the E-Port adapter board and the development board, and is used to realize the transmission of video stream data captured by the drone in real time between the E-Port adapter board and the development board; The development board is equipped with a lightweight target detection model for implementing the real-time detection method for construction site violations as described in any one of claims 1 to 8.

Images