A method and system for processing drone monitoring data based on smart construction sites

Abstract

This invention belongs to the field of image processing technology, specifically relating to a method and system for processing drone monitoring data based on smart construction sites. The method includes: first, using a YOLO model to obtain a set of target bounding boxes of workers in the original image; then, obtaining the instantaneous motion blur degree from the pixel gradient abrupt changes within a single target bounding box; next, obtaining candidate bounding boxes by the intersection-union ratio (IUU) of the target bounding box and the predicted bounding box of the same worker; and obtaining a candidate matching bounding box sequence by comparing the Euclidean distance between the center points of the target bounding box and the candidate bounding boxes with a threshold; then, obtaining the continuous motion blur degree based on the average movement speed and instantaneous blur degree of the workers in the candidate sequence within a time window; obtaining the blur kernel direction by weighting the target gradient map with its normalized result, and obtaining the blur kernel length through positive correlation mapping to form the motion blur kernel; and finally, restoring the image through Wiener filtering to obtain a clear video stream.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method and system for processing drone monitoring data based on smart construction sites. Background Technology

[0002] In the construction of modern smart construction sites, the use of drones for high-altitude, large-scale safety inspections has become an efficient management method. Drones can quickly acquire image data of the construction site and automatically identify unsafe behaviors of workers through computer vision technology, such as not wearing safety helmets as required. This automated monitoring method greatly improves the response speed and coverage of safety management and effectively reduces the risk of safety accidents.

[0003] However, drones face severe challenges in practical applications of security inspections. Due to the complex operating environment, drones are often affected by factors such as wind disturbances, mechanical vibrations of the aircraft itself, rapid target tracking, and automatic zoom operations, which inevitably lead to motion blur in the acquired images. This degradation in image quality severely interferes with the performance of subsequent computer vision-based intelligent analysis algorithms, directly resulting in a significant decrease in the accuracy of identifying key security details.

[0004] Specifically, when a drone captures a densely populated work area containing multiple workers, each worker may be in a different state of motion and have different relative displacements with the drone. This results in multiple motion blurs of different directions and degrees existing in a single image. If the entire image is restored using these motion blur parameters, it will not only fail to effectively eliminate the blur, but may even introduce new artifacts, making it even more blurry. Ultimately, this will severely reduce the accuracy of identifying whether each worker in the densely populated work area is wearing a safety helmet. Summary of the Invention

[0005] To address the problem that existing technologies in multi-target dense scenarios cannot differentiate the processing of motion blur of each target, resulting in poor image restoration and affecting the accuracy of subsequent recognition, this invention provides a method and system for processing UAV monitoring data based on smart construction sites.

[0006] In a first aspect, the present invention provides a method for processing drone monitoring data based on smart construction sites, comprising: calling a pre-trained YOLO deep learning object detection network model to obtain a set of target bounding boxes corresponding to each worker in all original images; obtaining the instantaneous motion blur of a single target bounding box based on the gradient abrupt changes of pixels within the single target bounding box in the original image; obtaining candidate bounding boxes based on the intersection-union ratio (IUU) between the target bounding box and all predicted bounding boxes corresponding to the same worker in the original image; obtaining a sequence of candidate matching bounding boxes corresponding to the target bounding box based on the comparison results of the Euclidean distance and distance threshold between the corresponding center point coordinates of the target bounding box and the candidate bounding boxes; obtaining the continuous motion blur of the worker corresponding to each target bounding box based on the average movement speed of the worker within the time window length in the candidate matching bounding box sequence and the instantaneous motion blur of the target bounding box; weighting the target gradient map based on the normalization result of the continuous motion blur to obtain the blur kernel direction; obtaining the blur kernel length based on the positive correlation mapping relationship; constructing a motion blur kernel using the blur kernel direction and blur kernel length, and performing image restoration on the original image using the Wiener filtering algorithm, thereby obtaining a clear video stream collected by the drone.

[0007] This invention uses YOLO to accurately acquire the bounding box of the worker target, and combines pixel gradient mutations with worker motion trajectory analysis to obtain targeted motion blur kernel parameters. This makes Wiener filtering restoration more consistent with the actual blur situation, effectively improving the clarity of UAV video streams. At the same time, it takes into account both instantaneous and continuous motion blur features, reducing misjudgments due to single factors. The restored clear video can better meet the image quality requirements of scenarios such as worker operation monitoring and behavior analysis, and improve the reliability of related applications.

[0008] Preferably, obtaining the instantaneous motion blur degree of a single target bounding box based on the gradient abrupt changes of pixels within the single target bounding box in the original image includes: In the formula, It is the original image. target bounding box The corresponding instantaneous motion blur level; It is the target bounding box The average gradient magnitude of all pixels within the range; It is the original image. The maximum value of the average gradient magnitude of the pixels contained within the bounding boxes of all targets; It is the target bounding box The entropy value of the gradient direction of all pixels within the area; It is the original image. The maximum value of the entropy of the gradient direction corresponding to the pixels contained within the bounding box of all targets; It is the index of the original image; It is the original image. The index of the target bounding box; It is the first weighting coefficient.

[0009] This method effectively avoids the limitations of a single indicator by combining the average gradient magnitude and gradient direction entropy of pixels within the target bounding box, reducing misjudgments caused by differences in lighting and texture. Simultaneously, the introduction of a first weighting coefficient allows for adjustment of the importance of the two indicators based on actual working conditions, improving adaptability. Furthermore, normalization of the maximum value of the corresponding indicator within the image ensures uniform and controllable blur levels, facilitating comparison of blurring in different worker bounding boxes within the same image and providing a foundation for subsequent fusion with other indicators. Moreover, the calculation focuses on a single bounding box, accurately assessing the blurring state of each worker region and avoiding interference from other regions, providing a reliable basis for targeted image restoration.

[0010] Preferably, obtaining the candidate matching bounding box sequence corresponding to the target bounding box includes: selecting from all predicted bounding boxes corresponding to the same worker the candidate matching bounding box. The predicted bounding box with the highest intersection-union ratio is selected as the candidate bounding box. If the original image Middle target bounding box With candidate bounding boxes Euclidean distance between their respective center point coordinates Less than or equal to the distance threshold Then the original image is considered target bounding box With candidate bounding boxes If they belong to the same worker, they are considered to belong to different workers; otherwise, they are considered not to belong to the same worker. Based on the judgment result, the bounding box sequence of the same worker in the current frame and the previous consecutive frames is obtained, which constitutes the candidate matching bounding box sequence.

[0011] This method selects the predicted bounding box with the largest intersection-union ratio (IoU) with the target box from the predicted bounding boxes of the same worker as candidate boxes, and then verifies whether they belong to the same worker by checking whether the Euclidean distance between their center points is less than or equal to a threshold. This double screening greatly improves the matching accuracy and provides accurate and continuous basic data for subsequent calculation of the worker's average movement speed and the degree of continuous motion blur, reducing subsequent analysis bias and further ensuring the reliability of the overall image restoration process.

[0012] Preferably, obtaining the continuous motion blur degree of the worker corresponding to each target bounding box includes: In the formula, It is the target bounding box The degree of ambiguity corresponding to the continuous motion of the worker; It is the target bounding box Corresponding worker within the time window length The average velocity within; It is the original image. In the middle, all target bounding boxes correspond to workers within the time window length. The maximum value of the average velocity within the range; It is the original image. target bounding box The corresponding instantaneous motion blur level; It is the index of the original image; It is the second weighting coefficient.

[0013] This method integrates the average movement speed of the worker within a time window with the instantaneous motion blur of the bounding box using a second weighting coefficient. This balances the temporal continuity of the worker's movement, avoiding the one-sidedness of relying solely on instantaneous blur while ignoring the movement trend, and preserves the details of the instantaneous blur state. Furthermore, the second weighting coefficient can flexibly adapt to different motion scenarios, such as adjusting the emphasis on speed or instantaneous blur depending on the worker's speed. The resulting continuous motion blur can more comprehensively and accurately characterize the blur features caused by the worker's movement, providing a reliable basis for constructing an accurate motion blur kernel for subsequent weighted target gradient map construction and achieving efficient image restoration.

[0014] Preferably, obtaining the fuzzy kernel length based on the positive correlation mapping relationship includes: fixing the UAV inspection altitude to match the actual inspection scene, which can be set to 50 meters; the original image acquisition frequency is 1 image per second, and the camera exposure time is 0.1 seconds; setting the range of multiple known motion speeds to 0.5. up to 3 The simulation targets are then analyzed. The average value of the continuous motion ambiguity of each simulation target is calculated, and a simulation set of all simulation targets is constructed. The simulation set is fitted with linear fitting to obtain a mapping function in the form of a linear function. By adjusting the scaling factor and constant term of this mapping function, when the relative error between the calculated speed and the actual speed after fitting is less than or equal to 5%, the mapping function is the positive correlation mapping relationship in this scenario.

[0015] This method uses a fixed drone altitude, image acquisition frequency, and camera exposure time consistent with the actual inspection scenario, and sets a coverage of 0.5. up to 3 Simulated worker targets with common operating speeds are used to construct a simulation set by calculating the average value of their continuous motion blur. Then, a linear fitting is performed and the parameters are adjusted until the relative error between the fitted speed and the actual speed is less than or equal to 5% to determine a positive correlation mapping relationship. This ensures that the mapping relationship is highly adapted to the real inspection scenario, and the mapping accuracy is ensured by covering the key speed range of the simulated target and strict error control. This provides a solid support for the subsequent reliable acquisition of the blur kernel length and the realization of efficient image restoration.

[0016] Secondly, the present invention provides a drone monitoring data processing system based on smart construction sites, including a processor and a memory. The memory stores computer program instructions, and when the computer program instructions are executed by the processor, the above-mentioned drone monitoring data processing method based on smart construction sites is implemented.

[0017] By adopting the above technical solution, a computer program is generated from the above-mentioned method for processing drone monitoring data based on smart construction sites and stored in a memory so that it can be loaded and executed by a processor. This allows for the creation of terminal devices based on the memory and processor, making them convenient to use.

[0018] The beneficial effects of this invention are as follows: It accurately locates the worker's target bounding box using the YOLO model, laying an accurate object foundation for subsequent fuzzy analysis; it calculates the instantaneous fuzziness level based on pixel gradient mutations within the bounding box, enabling real-time capture of the fuzzy state generated by worker movement; it combines cross-union ratio (CUI) screening of candidate bounding boxes with Euclidean distance matching sequences to effectively eliminate irrelevant interference, ensuring the continuity and accuracy of worker trajectory tracking; it fuses the average velocity and instantaneous fuzziness within a time window to obtain the continuous motion fuzziness level, taking into account both temporal dynamic features and immediate fuzziness information, making the calculation of fuzzy kernel parameters more consistent with actual motion scenarios; it obtains the fuzzy kernel direction through normalized weighted gradient maps and the fuzzy kernel length based on positive correlation mapping, constructing a highly targeted motion fuzzy kernel; and it restores the image through Wiener filtering, significantly improving the clarity of the UAV video stream. This not only provides clearer visual evidence for worker operation monitoring but also improves the accuracy and reliability of subsequent applications such as behavior analysis and safety warnings, reducing the risk of misjudgment caused by image fuzziness. Attached Figure Description

[0019] Figure 1 This is a flowchart illustrating a method for processing drone monitoring data based on smart construction sites according to the present invention. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0022] This invention discloses a method for processing drone monitoring data based on smart construction sites, referring to... Figure 1 This includes steps S1 to S4:

[0023] S1. Call a pre-trained YOLO deep learning object detection network model to obtain the set of object bounding boxes for all original images.

[0024] It should be noted that this step aims to quickly locate candidate regions containing individual workers, i.e., the target bounding box set, from the original image set that may have motion blur. Due to the motion blur of the images, the localization results of this step are allowed to have inaccuracies in position and size. Its main purpose is to provide independent input for subsequent differential blur analysis for each region.

[0025] Specifically, technicians operate drones over the smart construction site to capture raw images containing multiple workers at a rate of one image per second, obtaining a raw image set, denoted as... ,in, It is the total number of original images. It is the index of the original image.

[0026] Furthermore, a pre-trained YOLO deep learning object detection network model is obtained. This model's acquisition involves the following training process: collecting a large number of smart construction site scene images containing workers and accurately annotating them to generate target bounding boxes for each worker; applying techniques such as motion blur, Gaussian blur, and brightness adjustment to the images in the dataset to simulate various complex situations captured by drones, significantly improving the model's robustness to blurred targets; using the aforementioned dataset, performing end-to-end training of the YOLO network in the PyTorch deep learning framework; and continuously iterating and updating the network weight parameters by optimizing the loss function until the model's performance on the validation set meets the preset requirements, thereby obtaining a pre-trained YOLO object detection neural network model.

[0027] It should be noted that in the specific solution described in this invention, the core is to use a pre-trained YOLO deep learning model for object detection; this model already has strong noise resistance and feature extraction capabilities, and there is no need for, nor should there be, additional traditional preprocessing steps such as median filtering.

[0028] Specifically, each original image in the original image set is directly input into a pre-trained YOLO deep learning object detection network; the YOLO model performs forward propagation on the input original images, automatically outputting the bounding box information of all initially detected targets and their confidence scores; to eliminate redundant detections of the same worker, a non-maximum suppression algorithm is applied to filter overlapping redundant detection boxes, suppressing those redundant boxes whose overlap with the highest-scoring box exceeds a set threshold; finally, a series of accurate and unique target bounding boxes are output: for example, the original images are... The set of target bounding boxes detected in the data is denoted as . ;in, Represents the original image The total number of detected target bounding boxes; The index represents the target bounding box; each target bounding box contains the length and width in the original image; the position corresponding to half the length and half the width of each target bounding box is taken as the center point of each target bounding box, thus obtaining the center point coordinates of each target bounding box.

[0029] After the above processing, the set of target bounding boxes in all original images is obtained and used as input for subsequent steps; the structure, training and forward propagation process of the YOLO model are existing technologies and will not be described in detail here.

[0030] S2. Based on the gradient abrupt changes of pixels within a single target bounding box in the original image, obtain the instantaneous motion blur degree of the single target bounding box.

[0031] It should be noted that this step aims to quantify the degree of motion blur experienced by each detected worker; motion blur leads to loss of image details and edge directional blurring; by analyzing the gradient abrupt changes in the target region within a single frame image, the instantaneous blur level can be assessed; if the image corresponding to the target bounding box is clear, there should be a large number of sharp edges within the target bounding box, these sharp edges have high amplitude values, and the gradient direction distribution is relatively chaotic and disordered; if the image corresponding to the target bounding box has motion blur, there should be a large number of blurred edges within the target bounding box, these blurred edges have low amplitude values, and the gradient direction distribution is less chaotic and more ordered.

[0032] Specifically, the original image set and the set of target bounding boxes corresponding to each original image are taken as input, and the original images are used as input. Set of target bounding boxes detected in Any target bounding box in Let's take the original image as an example for analysis: First, let's analyze the original image... Extract the corresponding target bounding box from the middle. Use the Sobel gradient operator to obtain the target bounding box. Gradient magnitude map and gradient direction map; target bounding box The average gradient magnitude and the entropy of the gradient direction of all pixels are denoted as follows: , Further, obtain the original image. For each target bounding box, the average gradient magnitude and the entropy value along the gradient direction are considered. The maximum values ​​of the average gradient magnitude and the entropy value along the gradient direction are selected and denoted as follows: , Then the target bounding box The corresponding instantaneous motion blur level is:

[0033] ;

[0034] In the formula, It is the original image. target bounding box The corresponding instantaneous motion blur level; It is the target bounding box The average gradient magnitude of all pixels within the range; It is the original image. The maximum value of the average gradient magnitude of the pixels contained within the bounding boxes of all targets; It is the target bounding box The entropy value of the gradient direction of all pixels within the area; It is the original image. The maximum value of the entropy of the gradient direction corresponding to the pixels contained within the bounding box of all targets; It is the index of the original image; It is the original image. The index of the target bounding box; It is the first weighting coefficient.

[0035] In the formula, Represents the original image Middle target bounding box The average gradient magnitude relative to the original image The relative intensity of the clearest target bounding box; when the target bounding box When the image is relatively blurry, the ratio is small. Subtracting this small ratio from 1 will result in a large value close to 1, representing a high degree of blur. The opposite is also true. Represents the original image Middle target bounding box The degree of disorder in the gradient direction relative to the original image The relative intensity of the most chaotic target bounding box; when the target bounding box When the image is relatively blurry, the ratio is small, reflecting that the gradient direction is relatively ordered. Subtracting this small ratio from 1 will result in a large value close to 1, which also represents a high degree of blurriness, and vice versa.

[0036] It should be noted that when drones conduct large-scale, high-speed inspections, the main source of ambiguity is linear relative motion. In this case, the decrease in gradient directional entropy is an extremely stable and clear signal. To improve the sensitivity to this type of ambiguity, the gradient directional entropy loss term should be given a higher weight.

[0037] Among them, the first weight coefficient It is a range of values The hyperparameters between these parameters are used to adjust the normalized gradient magnitude loss term. and normalized gradient direction entropy loss term The contribution of the instantaneous motion blur degree corresponding to the target bounding box in the original image; the first weighting coefficient. The specific value can be set based on prior analysis of the application scenario. For example, in this invention, the first weighting coefficient is... Set as This ensures that the algorithm has the highest sensitivity to the orderliness of gradient directions caused by linear motion.

[0038] Normalized gradient magnitude loss term and normalized gradient direction entropy loss term When the average value is larger, the original image In the middle, the target bounding box Corresponding instantaneous motion blur The larger it is, the smaller it is, and vice versa.

[0039] Based on the above operations, the original image is obtained. The instantaneous motion blur degree corresponding to each target bounding box in the image is obtained, thereby obtaining the instantaneous motion blur degree corresponding to each target bounding box in each original image.

[0040] S3. Based on the intersection-union ratio of the target box and the predicted worker trajectory box, and the comparison between the Euclidean distance between their center points and the threshold, a sequence of candidate matching bounding boxes is obtained; then, by combining the average speed of the worker's time window and the instantaneous motion blur of the target box in the sequence, the continuous motion blur is obtained.

[0041] It should be noted that step S3, based on quantifying the instantaneous blur of the target within a single frame, combines the positional changes of the same worker target between consecutive frames to reflect the worker's movement speed and comprehensively evaluate its continuous motion blur in the time series, so as to more comprehensively reflect the cumulative blur effect caused by the worker's violent and continuous movement, thereby providing a basis for subsequent decision-making.

[0042] Specifically, when a worker is detected for the first time, a new trajectory is created and the Kalman filter is initialized; for each existing worker trajectory in the system, its corresponding Kalman filter is used to predict the position where the trajectory should appear in the current original image, resulting in a predicted bounding box; for example, for existing worker trajectories in the system... Using its corresponding Kalman filter, the trajectory is predicted in the original image. The original image is obtained by finding the position where it should appear in the image. workers Corresponding predicted bounding box ;in, It is an index for workers.

[0043] Furthermore, using the original image target bounding box and predicted bounding boxes Let's take an example to analyze: Calculate the original image In the middle, the target bounding box The intersection-union ratio (IU) with each predicted bounding box is calculated using the following formula:

[0044] ;

[0045] In the formula, It is the original image. Middle target bounding box With predicted bounding box The intersection and union ratio; It is the original image. The first in One target bounding box; It is the original image. workers The corresponding predicted bounding box; It is the target bounding box With predicted bounding box The intersection of the two bounding boxes reflects the pixel area of ​​the overlapping region. It is the target bounding box With predicted bounding box The union of the two bounding boxes reflects the area of ​​all pixel regions that can be covered by the two bounding boxes together.

[0046] Select the bounding box that matches the target bounding box from all predicted bounding boxes corresponding to the same worker. The predicted bounding box with the highest intersection-union ratio is selected as the candidate bounding box. ; Calculate the original image Middle target bounding box With candidate bounding boxes The Euclidean distance between the coordinates of their respective center points is denoted as . Set distance thresholds based on the drone's inspection altitude. If the original image Middle target bounding box With candidate bounding boxes Euclidean distance between their respective center point coordinates Less than or equal to the distance threshold Then the original image is considered target bounding box With candidate bounding boxes Belonging to the same worker, and using this detection box to update the trajectory. The Kalman filter is used to determine whether the worker belongs to the same worker or not. Conversely, if the worker does not belong to the same worker, the bounding box sequence of the same worker in the current frame and previous consecutive frames is obtained, forming a candidate matching bounding box sequence, denoted as: ,in, It is the length of the time window.

[0047] It should be noted that the above operation does not directly match the detection box of the previous frame. This ensures that even if a detection was missed in the previous frame, as long as the trajectory is within the time window, it can still be correlated. It should also be noted that the top-left corner of the original image is the origin of the coordinate system, with the horizontal axis extending horizontally to the right and the vertical axis extending vertically downwards. Therefore, the original image... Middle target bounding box With candidate bounding boxes The x-coordinate of each center point is the x-coordinate of the top-left corner of the bounding box plus half its length, and the y-coordinate is the y-coordinate of the top-left corner of the bounding box plus half its width.

[0048] Specifically, in terms of trajectory Corresponding candidate matching bounding box sequence Taking this as an example: calculate the Euclidean distance between the center points of every two adjacent candidate matching bounding boxes in the sequence, and denot the sum as... Then the trajectory Corresponding workers In time window length The average velocity within is: Similarly, obtain the original image. In the middle, all target bounding boxes correspond to workers within a preset continuous tracking window length. The maximum average velocity within the range is denoted as . Then the target bounding box The degree of ambiguity corresponding to the continuous motion of the worker is:

[0049] ;

[0050] In the formula, It is the target bounding box The degree of ambiguity corresponding to the continuous motion of the worker; It is the target bounding box Corresponding worker within the time window length The average velocity within; It is the original image. In the middle, all target bounding boxes correspond to workers within the time window length. The maximum value of the average velocity within the range; It is the original image. target bounding box The corresponding instantaneous motion blur level; It is the index of the original image; It is the second weighting coefficient.

[0051] When the target bounding box The corresponding worker within the time window length The greater the average motion speed within the image, the better. target bounding box The greater the degree of instantaneous motion blur, the larger the target bounding box. The greater the ambiguity of the continuous movement of the corresponding worker, the less ambiguity there is, and vice versa.

[0052] It should be noted that the second weighting coefficient It is a range of values The hyperparameters between these parameters are used to adjust the normalized average velocity. and instantaneous motion blur The contribution of the continuous motion blur degree corresponding to the target bounding box in the original image; the second weighting coefficient. The specific value can be set based on prior analysis of the application scenario. For example, the present invention uses the second weighting coefficient. Set as It can enhance the impact of temporal motion speed on continuous fuzzy evaluation and more accurately capture the cumulative motion fuzziness caused by the worker's continuous and violent displacement.

[0053] Based on the above operations, the continuous motion blur degree of each target bounding box in each original image is obtained.

[0054] S4. The ratio of the continuous motion blur of each target box to the sum of all targets is used as the blur weight to obtain the weighted target gradient map, thereby obtaining the motion blur kernel. After Wiener filtering, a clear video stream of the UAV is obtained.

[0055] It should be noted that, based on the continuous motion blur of each target bounding box in each original image obtained in step S3, the worker area with the most severe blur is focused on, and the key parameters of motion blur are accurately estimated: blur kernel length and blur kernel direction. Then, the entire image is restored in a targeted manner to avoid the effect deviation caused by uniform processing, and finally a clear video stream is stitched together.

[0056] Specifically, all worker target bounding boxes are extracted from the original image, and a blur weight is assigned to each target bounding box according to its continuous motion blur degree. The following conditions must be met: the sum of the assigned blur weights is 1, and the blur weight assigned to each target bounding box is the ratio of the continuous motion blur degree of each target bounding box to the sum of the continuous motion blur degrees of all target bounding boxes in the original image. This setting can ensure that worker target bounding boxes with high continuous motion blur degree contribute more.

[0057] Furthermore, Sobel gradient calculation is performed on the original image to obtain the gradient magnitude map of the original image; the gradient magnitude within each target bounding box is multiplied by its blur weight, and the gradient magnitude of the background region is set to 0 to generate a weighted target gradient map. This map retains only the gradient information of the worker target, providing high-quality input for subsequent parameter estimation.

[0058] It should be noted that motion blur is essentially the trailing image formed on the image when a target undergoes linear displacement during the exposure time. Its core descriptive parameters include the blur kernel length and the blur kernel direction, which together constitute the blur kernel function. The direction of the trailing image caused by motion blur is the target's motion direction. The target's motion direction is parallel to the gradient direction of the edge, so the gradient direction angle of the pixel can be directly used as its corresponding blur direction angle. When the camera exposure time is fixed, the blur kernel length is equal to the product of the target's relative motion speed and the exposure time. Since the degree of continuous motion blur is positively correlated with the target's relative motion speed, the weighted average of the degree of continuous motion blur is first calculated using the blur weights of each target bounding box as weights. Then, the target's relative motion speed is obtained through the positive correlation mapping relationship. The obtained target relative motion speed is multiplied by the exposure time to obtain the blur kernel length.

[0059] In one embodiment, the positive correlation mapping is obtained as follows: A smart construction site drone inspection scenario is simulated, with the drone inspection height fixed to match the actual inspection scenario, which can be set to 50 meters; the original image acquisition frequency is 1 frame per second, and the camera exposure time is 0.1 seconds; under the above fixed scenario, multiple known motion speeds are set to a range of 0.5. up to 3 The simulated worker targets are within a range that covers the common movement speed range of construction workers; the average value of the continuous motion ambiguity of each simulated worker target is calculated according to step S3, and a simulation set of all simulated worker targets is constructed: ,in, It is a simulated worker target The known velocity of motion, It is a simulated worker target The weighted average of the continuous motion blur levels; a linear fitting is used to fit the simulated set to obtain a mapping function in the form of a linear function: In the formula, This is the proportionality coefficient. As a constant term, by adjusting the scaling factor and constant term of this linear function mapping function, when the relative error between the calculated speed and the actual speed after fitting is less than or equal to 5%, the accuracy requirement is met. At this time, the mapping function is the positive correlation mapping relationship in this scenario.

[0060] Furthermore, the specific method for obtaining the blur kernel length and blur kernel direction is as follows: First, extract significant edge features from the weighted target gradient map and generate a gradient direction map based on the extraction results; directly use the gradient direction angle of each pixel in the gradient direction map as the blur direction angle of the corresponding pixel, and normalize it to the range of 0 to 180 degrees; use the gradient magnitude of the pixel as the weight, perform weighted histogram statistics on the normalized blur direction angle, and take the angle corresponding to the maximum peak value in the histogram as the blur kernel direction; use the blur weight of each target bounding box as the weight, calculate the weighted average of the blur degree of continuous motion, and then obtain the relative motion speed of the target through a positive correlation mapping relationship. Multiply the obtained relative motion speed of the target by the exposure time to obtain the blur kernel length.

[0061] Furthermore, based on the aforementioned blur kernel direction and blur kernel length, the motion blur kernel corresponding to the original image is obtained. Based on this motion blur kernel, the Wiener filtering algorithm is used to perform deconvolution operation on the original image to achieve image restoration; finally, a clear image after removing motion blur is output.

[0062] Repeat the above operations to obtain a set of clear images; since the original images were acquired at a rate of 1 frame per second, the set of clear images can be stitched together in index order to obtain a clear video stream acquired by the drone.

[0063] This invention also discloses a drone monitoring data processing system based on smart construction sites, including a processor and a memory. The memory stores computer program instructions, and when the computer program instructions are executed by the processor, a drone monitoring data processing method based on smart construction sites according to the present invention is implemented.

[0064] The system also includes other components well known to those skilled in the art, such as communication buses and communication interfaces, the settings and functions of which are known in the art and will not be described in detail here.

Claims

1. A method for processing drone monitoring data based on smart construction sites, characterized in that, include: Call a pre-trained YOLO deep learning object detection network model to obtain the set of target bounding boxes corresponding to each worker in all the original images; Based on the gradient abrupt changes of pixels within a single target bounding box in the original image, the instantaneous motion blur degree of the single target bounding box is obtained, including: In the formula, It is the original image. target bounding box The corresponding instantaneous motion blur level; It is the target bounding box The average gradient magnitude of all pixels within the range; It is the original image. The maximum value of the average gradient magnitude of the pixels contained within the bounding boxes of all targets; It is the target bounding box The entropy value of the gradient direction of all pixels within the area; It is the original image. The maximum value of the entropy of the gradient direction corresponding to the pixels contained within the bounding box of all targets; It is the index of the original image; It is the original image. Index of the target bounding box; It is the first weighting coefficient; Candidate bounding boxes are obtained based on the intersection-union ratio (IUU) between the target bounding box and all predicted bounding boxes corresponding to the same worker in the original image; a sequence of candidate matching bounding boxes corresponding to the target bounding box is obtained based on the comparison between the Euclidean distance and the distance threshold between the center point coordinates of the target bounding box and the candidate bounding boxes. Based on the average movement speed of the corresponding worker within the time window in the candidate matching bounding box sequence, and the instantaneous motion blur of the target bounding box, the continuous motion blur of the worker corresponding to each target bounding box is obtained; the target gradient map is weighted according to the normalization result of the continuous motion blur to obtain the blur kernel direction; the blur kernel length is obtained according to the positive correlation mapping relationship; the blur kernel direction and blur kernel length constitute the motion blur kernel, and the Wiener filtering algorithm is used to restore the original image, thereby obtaining a clear video stream captured by the UAV.

2. The method for processing UAV monitoring data based on smart construction sites according to claim 1, characterized in that, The methods for obtaining all predicted bounding boxes corresponding to the same worker include: When a worker is detected for the first time, a new trajectory is created and the Kalman filter is initialized. For each existing worker trajectory in the system, its corresponding Kalman filter is used to predict the location where the trajectory should appear in the current original image, thus obtaining the predicted bounding box.

3. The method for processing UAV monitoring data based on smart construction sites according to claim 1, characterized in that, The method for obtaining the intersection-union ratio (IUU) between the target bounding box in the original image and all predicted bounding boxes corresponding to the same worker includes: ； In the formula, It is the original image. Middle target bounding box With the original image Chinese workers Corresponding predicted bounding box The intersection and union ratio; It is the original image. The first in One target bounding box; It is the original image. workers The corresponding predicted bounding box.

4. The method for processing UAV monitoring data based on smart construction sites according to claim 1, characterized in that, The process of obtaining the candidate matching bounding box sequence corresponding to the target bounding box includes: Select the bounding box that matches the target bounding box from all predicted bounding boxes corresponding to the same worker. The predicted bounding box with the highest intersection-union ratio is selected as the candidate bounding box. If the original image Middle target bounding box With candidate bounding boxes Euclidean distance between their respective center point coordinates Less than or equal to the distance threshold Then the original image is considered target bounding box With candidate bounding boxes If they belong to the same worker, they are considered to belong to different workers; otherwise, they are considered not to belong to the same worker. Based on the judgment result, the bounding box sequence of the same worker in the current frame and the previous consecutive frames is obtained, which constitutes the candidate matching bounding box sequence.

5. The method for processing UAV monitoring data based on smart construction sites according to claim 1, characterized in that, Obtaining the coordinates of the center points of the target bounding box and the candidate bounding box respectively includes: The top left corner of the original image is the origin of the coordinate system. The horizontal axis extends horizontally to the right from the origin, and the vertical axis extends vertically downwards. The x-coordinate of the center point of the target bounding box and the candidate bounding box in the original image is the x-coordinate of the top left corner of the bounding box plus half its length, and the y-coordinate is the y-coordinate of the top left corner of the bounding box plus half its width.

6. The method for processing UAV monitoring data based on smart construction sites according to claim 1, characterized in that, The process of obtaining the continuous motion blur degree of the worker corresponding to each target bounding box includes: ； In the formula, It is the target bounding box The degree of ambiguity corresponding to the continuous motion of the worker; It is the target bounding box Corresponding worker within the time window length The average velocity within; It is the original image. In the middle, all target bounding boxes correspond to workers within the time window length. The maximum value of the average velocity within the range; It is the original image. target bounding box The corresponding instantaneous motion blur level; It is the index of the original image; It is the second weighting coefficient.

7. The method for processing UAV monitoring data based on smart construction sites according to claim 1, characterized in that, The step of weighting the target gradient map based on the normalization result of the continuous motion ambiguity to obtain the ambiguity kernel direction includes: First, significant edge features are extracted from the weighted target gradient map, and a gradient direction map is generated based on the extraction results. The gradient direction angle of each pixel in the gradient direction map is directly used as the blur direction angle of the corresponding pixel, and it is normalized to the range of 0 to 180 degrees. Using the gradient magnitude of the pixel as the weight, a weighted histogram is calculated on the normalized blur direction angle, and the angle corresponding to the maximum peak value in the histogram is used as the blur kernel direction.

8. The method for processing UAV monitoring data based on smart construction sites according to claim 1, characterized in that, The step of obtaining the fuzzy kernel length based on the positive correlation mapping relationship includes: The fixed drone inspection altitude matches the actual inspection scenario and can be set to 50 meters; the raw image acquisition frequency is 1 frame per second, and the camera exposure time is 0.1 seconds; the range of multiple known motion speeds is set to 0.

5. up to 3 The simulation targets are then analyzed. The average value of the continuous motion ambiguity of each simulation target is calculated, and a simulation set of all simulation targets is constructed. The simulation set is fitted with linear fitting to obtain a mapping function in the form of a linear function. By adjusting the scaling factor and constant term of this mapping function, when the relative error between the calculated speed and the actual speed after fitting is less than or equal to 5%, the mapping function is the positive correlation mapping relationship in this scenario.

9. A drone monitoring data processing system based on smart construction sites, characterized in that, include: A processor and a memory, wherein the memory stores computer program instructions that, when executed by the processor, implement a UAV monitoring data processing method based on a smart construction site according to any one of claims 1-8.

Images