Road-occupying management intelligent monitoring method based on YOLO algorithm and unmanned aerial vehicle cooperation

Abstract

The invention relates to a lane-occupying management intelligent monitoring method based on cooperation of a YOLO algorithm and an unmanned aerial vehicle. The method comprises the following steps: acquiring image data of a lane-occupying operation scene according to a preset gridding path through an unmanned aerial vehicle, and preprocessing the image data; training a YOLO algorithm based on the preprocessed data, and generating and optimizing a target detection model; performing target detection and time sequence behavior analysis on a video stream acquired by the unmanned aerial vehicle in real time by using the target detection model, and judging whether a lane-occupying operation behavior exists or not; and according to a determination result, triggering a grading alarm, recording and processing a road-occupying operation behavior, storing a violation event as structured data, supporting query and analysis, and completing a closed-loop management process. According to the method, through YOLO algorithm optimization and cooperation with the unmanned aerial vehicle, high-precision (mAP at 50 > = 92%) and low-delay (< = 200ms) detection is realized, grading response and edge calculation are combined, more than 80% of manual patrol amount is reduced, the method adapts to complex scenes such as day and night, haze and the like, and the automation and refinement level of city management can be improved.

Description

Technical Field

[0001] The present application relates to the field of intelligent urban governance methods, and in particular to an intelligent monitoring method, device, computer-readable storage medium, and electronic device for road occupation business based on the YOLO algorithm and the collaboration of drones. Background Art

[0002] With the acceleration of smart city construction, the need for intelligent monitoring of illegal street occupation is becoming increasingly urgent. Currently, street occupation monitoring technology based on the YOLO (You Only Look Once) algorithm and drones has become a key research direction in this field and is gradually being implemented in some urban governance scenarios. This technology uses drones as aerial perception platforms and combines them with the YOLO series of object detection algorithms to provide real-time identification and early warning of street occupation (such as street vendor carts and product displays). This has initially achieved improved monitoring efficiency and optimized traditional manual inspection methods.

[0003] However, existing technologies still have many limitations in practical applications, specifically: (1) At the algorithm level, the YOLO series of algorithms, as a single-stage detection framework, has the advantage of real-time performance (typical FPS can reach 30-60), but its performance is limited in small target detection scenarios. For small targets such as vendor carts, the missed detection rate is high due to the limited feature extraction capabilities. In complex scenarios such as dense crowds and target deformation (such as when a cart is folded), the detection confidence may drop by more than 30%, affecting the reliability of recognition.

[0004] (2) At the data level, existing models are highly dependent on manually labeled structured data sets for training. The identification of new road-blocking tools (such as electric food trucks, mobile shelves, etc.) requires re-collection and labeling of data, resulting in insufficient model generalization ability and difficulty in adapting to dynamically changing road-blocking business scenarios.

[0005] (3) At the hardware and deployment level, the training and inference of high-precision YOLO models require a lot of computing power (typically requiring a GPU with ≥8GB of video memory), which conflicts with the computing power limitations of edge devices such as drones. At the same time, drones are limited by battery capacity, and their single-time flight time is usually ≤40 minutes, which limits the continuity and coverage of monitoring tasks.

[0006] (4) In terms of adaptability to application scenarios, extreme environments (such as strong light reflection, mirror occlusion, etc.) can easily cause false detections, resulting in redundant alarms. When deployed across regions, the model or inspection strategy needs to be repeatedly adjusted according to parameters such as different road widths and building layouts, which significantly increases the system deployment and maintenance costs.

[0007] To alleviate the above problems, existing technologies attempt to optimize model performance through training strategies such as dynamic label allocation and Mosaic data enhancement, and combine inference acceleration frameworks such as TensorRT to achieve edge deployment, reducing target recognition and alarm response time to less than 200ms; at the same time, through drone grid inspections (flight altitude 30-50 meters, speed ≤5m / s) and multimodal data fusion (video, voiceprint, air quality sensors) to improve scene adaptability, a detection accuracy rate of more than 92% was achieved in the pilot area, and a closed-loop management was formed through a hierarchical governance mechanism (automatic voice warnings, AR-assisted law enforcement), effectively reducing the workload of manual inspections by more than 80%.

[0008] Despite this, the core bottlenecks of existing technologies in terms of robustness of small target detection, data generalization capability, edge computing power and endurance constraints, and cross-scenario adaptability have not yet been completely broken through. It is urgent to propose a more efficient and intelligent method for monitoring road occupation to further improve the level of automation and refinement of urban management. Summary of the Invention

[0009] In order to address the above problems, this application proposes a new intelligent monitoring method for road occupation business based on the YOLO algorithm and drone collaboration.

[0010] The present invention mainly adopts the following technical strategies: 1. Dynamic target detection and optimization based on YOLO algorithm 1) Model lightweighting and precision improvement: YOLOv8 (or an improved version) is selected as the core detection algorithm. Adaptive anchor box clustering (generating nine groups of initial anchor boxes based on the K-means algorithm) and Mosaic data augmentation (using four-image stitching) are used to improve the detection accuracy of road-blocking business targets (mAP@50 ≥ 92%) while ensuring real-time performance (inference latency ≤ 200ms).

[0011] 2) Time series behavior analysis: Combined with DeepSORT or FairMOT algorithm to track the target trajectory, if the target is detected within 5 consecutive frames (time window 10 seconds) in a fixed area and the coverage area is ≥ 2m 2 , it will be determined as occupying the road for business.

[0012] 2. UAV dynamic inspection and multimodal data fusion 1) Adaptive Path Planning: The drone is equipped with an RTK positioning module and flies along a 100m x 100m grid path (speed ≤ 5m / s, altitude 30-50m). It integrates thermal imaging and low-light enhancement technology to achieve all-weather monitoring.

[0013] 2) Cross-modal verification: Integrate drone video streams, GPS coordinates, voiceprint sensors (for detecting tweeter noise), and air quality data (for identifying oil smoke concentration), and use Kalman filtering to achieve spatiotemporal trajectory matching to reduce false alarm rates.

[0014] 3. Edge Computing and Real-time Response Mechanism 1) Lightweight deployment: Using edge computing units such as NVIDIA Jetson Xavier to load the compressed YOLO model (using TensorRT / ONNX format), low-latency inference (<200ms) is achieved, supporting multi-machine collaboration on 5G networks.

[0015] 2) Tiered Alerts: Level 1 alerts trigger cloud broadcasts and rectification notifications; level 2 alerts dispatch patrol cars and AR glasses with overlaid detection frames to assist law enforcement, creating a closed loop of "AI early warning - manual handling - data archiving."

[0016] 4. Multimodal Data Privacy Protection 1) Desensitizing sensitive information: Edge computing is used for local processing. Sensitive information such as facial features is uploaded to the cloud only after desensitization, meeting the requirements of the Personal Information Protection Law.

[0017] 2) Hierarchical authority management: Law enforcement records and merchant information are stored separately, and federated learning is used to implement cross-regional model updates to prevent the risk of data abuse.

[0018] In order to achieve the above objectives, this application provides the following technical solutions: The first aspect of the present application provides an intelligent monitoring method for road occupation based on the YOLO algorithm and the collaboration of a drone, the method comprising: S1: Use drones to collect image data of road occupation business scenes along a preset grid path and pre-process the data; S2: Train the YOLO algorithm based on the preprocessed data to generate and optimize the target detection model; S3: Use the optimized target detection model to perform target detection and time series behavior analysis on the video stream collected by the drone in real time to determine whether there is any road occupation behavior; S4: Trigger graded alarms based on the judgment results, record and process road-blocking business behaviors, and store violation records as structured data to support query and analysis, completing the closed-loop management process.

[0019] Furthermore, in the method of the present application, the step S1 described in which the image data of the road-occupying business scene is collected by a drone according to a preset grid path includes: the drone is equipped with an RTK positioning module, and the image data of the road-occupying business scene is collected according to a preset 100m×100m grid path, the drone speed is ≤5m / s, the flight altitude is 30-50m, and the collected image resolution is ≥1920×1080.

[0020] Furthermore, in the method of the present application, the data preprocessing in step S1 includes: S11: Enhance images in low-light or hazy scenes by using histogram equalization combined with gamma correction to restore contrast; S12: Based on Zhang Zhengyou’s calibration method, the radial distortion and tangential distortion of the drone lens are corrected. The radial distortion correction formula is:

[0021] Where, is the radial distortion coefficient, , is the lateral distortion coordinate, is the longitudinal distortion coordinate, x is the image horizontal coordinate, and y is the image vertical coordinate; The tangential distortion correction formula is:

[0022] Where, , , is the longitudinal distortion coordinate, , x is the horizontal coordinate of the image, y is the vertical coordinate of the image; S13: The scene data with accurate geometric features is described by the intrinsic parameter matrix. The formula of the intrinsic parameter matrix is: K=

[0023] Where, is the focal length, The main point coordinates.

[0024] Furthermore, in the method of the present application, the training and optimization of the target detection model in step S2 includes: S21: Collect images of roadside business operations under different lighting conditions, including daytime, nighttime, sunny, and rainy days, and label small, medium, and large objects in the YOLO standard format. S22: The labeled data is divided into a training set and a validation set. Mosaic data augmentation (four-image stitching) and random HSV perturbation (hue ±10%, saturation ±20%, lightness ±15%) are used to improve the robustness of the model. S23: YOLOv8s is used as the base model, with an input size of 640×640 and a CSPDarknet53 backbone network. Nine groups of initial anchor boxes are generated through K-means clustering (Euclidean distance metric) and assigned to the P3, P4, and P5 detection layers of YOLOv8s to match the scale distribution of small, medium, and large objects, respectively. S24: Use a dynamic label assignment strategy, combined with cosine annealing learning rate scheduling and SGD optimizer to train the model, with 50-100 epochs and a batch size of 16 (when GPU memory ≥ 8GB).

[0025] Furthermore, in the method of this application, the three types of targets, small, medium, and large, mentioned in step S21 include: Small targets include vendor carts, pavement tables and chairs, food stalls, temporary pavement stalls, and electric tricycles; Medium-sized targets include road-blocking cargo, small vans, and small trucks; Large targets include illegal billboards, vans, and medium-sized trucks.

[0026] Furthermore, in the method of the present application, the three detection layers of YOLOv8s described in step S23, wherein P3 is downsampled 8 times, P4 is downsampled 16 times, and P5 is downsampled 32 times; The dynamic label allocation strategy described in step S24 is high weight when IoU>0.6 and low weight when IoU≤0.6; the initial learning rate of the cosine annealing learning rate scheduling is 0.01, and the cosine annealing is reduced from the initial value to the minimum value of 0.001; the momentum of the SGD optimizer is 0.9, and the weight decay is 5e-4.

[0027] Furthermore, in the method of this application, step S3 includes: S31: Deploy the optimized object detection model to an edge computing unit (such as the NVIDIA Jetson XavierNX) and achieve low-latency inference (≤200ms) through TensorRT acceleration. S32: Target detection is performed on the video stream collected by the drone in real time, and the target category, position (x, y, w, h), confidence level and pixel area are output; S33: Track the target trajectory based on the DeepSORT or FairMOT algorithm and calculate the actual coverage area. The area conversion formula is:

[0028] S34: Determine whether there is any business activity occupying the road. The determination method is a joint time-space determination. The determination rule is: if the target is detected in a fixed area for 5 consecutive frames (time window ≤ 10 seconds) or the stay time is ≥ 10 seconds, and the average actual coverage area is ≥ 2m², then it is determined that there is business activity occupying the road.

[0029] Furthermore, in the method of the present application, the graded alarm in step S4 includes: Level 1 Alert: triggers a cloud broadcast to play a warning message (such as "Please do not occupy the road for business") and pushes a screenshot of the violation to the urban management platform; if not handled within 15 minutes, it is upgraded to Level 2 Alert; Level 2 alert: Dispatch patrol cars equipped with AR glasses overlay the illegal target location frame and historical records in real time to assist law enforcement officers in handling the situation.

[0030] A second aspect of the present application provides an intelligent monitoring device for road occupation based on the YOLO algorithm and the collaboration of a drone, the device comprising: The data acquisition module is used to collect image data of road occupation business scenes using drones along a preset grid path and pre-process the data; Model training module, used to train the YOLO algorithm based on preprocessed data to generate and optimize the target detection model; The detection and analysis module uses the optimized target detection model to perform target detection and time-series behavior analysis on the video stream collected by the drone in real time to determine whether there is any road occupation behavior; The alarm management module is used to trigger graded alarms based on the judgment results, record and process road-blocking business behaviors, and store violation records as structured data, supporting query and analysis to complete the closed-loop management process.

[0031] When running, the device implements the steps of the aforementioned intelligent monitoring method for road occupation business based on the YOLO algorithm and the collaboration of the drone.

[0032] A third aspect of the present application provides an electronic device, comprising: a memory and a processor; Memory: used to store computer programs; Processor: used to execute the computer program to implement the steps of the aforementioned intelligent monitoring method for road occupation based on the YOLO algorithm and the collaboration of drones.

[0033] The fourth aspect of the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the aforementioned intelligent monitoring method for road occupation business based on the YOLO algorithm and the collaboration of drones are implemented.

[0034] In summary, compared with existing monitoring tools, the intelligent monitoring method for road occupation based on the YOLO algorithm and drone collaboration proposed in this application has the following advantages: 1. Real-time and high efficiency 1) Fast Response and High-Precision Detection: The YOLOv5 / YOLOv8 algorithms are used for single-stage object detection, achieving an inference speed of 30-60 FPS. Combined with real-time video transmission from a drone at an altitude of 30-50 meters (resolution ≥ 1920×1080), this system can detect road-blocking behaviors within 10 seconds. Through a dynamic label allocation strategy and optimized CIoU loss function, the model achieves a mean average precision (mAP@50) exceeding 92% in typical scenarios (using a measured scenario in Shanghai's Pudong New Area as an example, the accuracy reached 94.3%).

[0035] 2) Full-time coverage and blind spot elimination: Drones equipped with edge computing modules (such as the NVIDIA Jetson series) support 24 / 7 autonomous patrol. Preset 100m x 100m grid inspection routes effectively cover traditional surveillance blind spots. Combined with thermal imaging and low-light enhancement technology, detection rates remain above 90% even at night or in low-light conditions such as fog and haze.

[0036] 2. Multimodal Fusion and Intelligent Analysis 1) Cross-modal data linkage: This system integrates multi-source data, including drone-collected video streams, GPS positioning data, voiceprint sensors (for detecting tweeter noise), and air quality sensors (for identifying oil smoke concentration), to verify the authenticity and persistence of road-blocking behavior through spatiotemporal matching.

[0037] 2) Behavioral dynamic modeling: Based on target trajectory tracking technology, when the time a road-occupying subject stays in a specific area exceeds a threshold (for example, the target is detected in five consecutive frames within 10 seconds), a graded alarm mechanism is automatically triggered.

[0038] 3. Intelligent Closed-Loop Management and Resource Optimization 1) Tiered Response Mechanism: A two-tiered alert response strategy is implemented. Level 1 alerts automatically play warnings (e.g., "Please do not occupy the road for business") via cloud broadcasting, and simultaneously push rectification notifications to the offending merchants' mobile phones. If no action is taken within 15 minutes, the system automatically upgrades to level 2 response. The level 2 response involves patrol cars and AR glasses, overlaying a real-time violation target location frame and historical violation records to assist law enforcement officers in precise handling.

[0039] 2) Cost and Efficiency Optimization: Compared to traditional manual inspections, this solution can reduce manual inspection workload by over 80% (according to Fuya Intelligent's field data). A single drone inspection can replace the on-site inspections of 3-5 urban management personnel. Furthermore, by leveraging federated learning technology to enable cross-regional model iteration and update, the recognition rate for small samples of new vendor tools (such as electric food carts) can be increased by over 18%.

[0040] Other features and advantages of this application will be described in detail in the following description, or will be understood through the implementation of the relevant technical solutions of this application. The objectives and other advantages of this application can be achieved through the technical features and technical means clearly indicated in the description, claims, and drawings, and obtained through the implementation of these technical contents. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] To more clearly illustrate the technical solutions of the embodiments of the present application, the following briefly introduces the drawings involved in the description of the embodiments. It should be noted that the drawings only illustrate some embodiments of the present application. Those skilled in the art can deduce other relevant drawings based on these drawings without engaging in creative work.

[0042] Figure 1 This is the overall implementation flow chart of the intelligent monitoring method for road occupation business in this application.

[0043] Figure 2 This is a flowchart of the YOLO algorithm optimization method in this embodiment.

[0044] Figure 3 Schematic diagram of multimodal data fusion in the method of this embodiment.

[0045] Figure 4 This is a structural diagram of the intelligent monitoring device for road occupation business in this application.

[0046] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It should be understood that the described embodiments are only some embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0048] In this document, the term "including" and any variations thereof (such as "including," "comprising," etc.) are open-ended expressions and should be understood as meaning "including but not limited to," meaning that the listed contents are not exhaustive and may include other contents not explicitly mentioned. The term "based on" should be understood as meaning "based at least in part on," meaning that the basis or condition referred to may not be the only factor and may also involve other relevant factors. The term "one embodiment" should be understood as meaning "at least one embodiment," meaning that the described embodiment is not the only possible implementation method and that other similar embodiments may exist.

[0049] In this application, the terms "a" and "a plurality" are used to modify related elements or features in an illustrative, non-restrictive manner. Unless the context clearly indicates otherwise, "a" should be understood as meaning "at least one," and "a plurality" should be understood as meaning "at least two." Those skilled in the art should interpret these terms appropriately based on the semantics and logical relationships of the context to ensure that they encompass the possibility of "one or more."

[0050] Figure 1 The following is the overall implementation process of the intelligent monitoring method for road occupation based on the YOLO algorithm and drone collaboration provided by this application, including the following steps: S1: Use drones to collect image data of road occupation business scenes along a preset grid path and pre-process the data; S2: Train the YOLO algorithm based on the preprocessed data to generate and optimize the target detection model; S3: Use the optimized target detection model to perform target detection and time series behavior analysis on the video stream collected by the drone in real time to determine whether there is any road occupation behavior; S4: Trigger graded alarms based on the judgment results, record and process road-blocking business behaviors, and store violation records as structured data to support query and analysis, completing the closed-loop management process.

[0051] In order to more clearly illustrate the technical solution of the present application, the following will further illustrate it through embodiments of specific scenarios.

[0052] This application proposes an intelligent monitoring method and system for road occupation business that combines the YOLO target detection algorithm with dynamic inspection by drones, achieving high-precision recognition and real-time response through multimodal data fusion and adaptive parameter optimization.

[0053] The core steps and key technical parameters of this method are as follows: Step 1: UAV data collection and preprocessing Inspection route planning: The drone is equipped with an RTK positioning module and flies according to a preset grid path (100m×100m grid). The flight altitude is controlled at 30-50m, the speed is ≤5m / s, and the image resolution is ensured to be ≥1920×1080.

[0054] Image Dehazing and Enhancement: Enhance low-light conditions or remove haze from drone-collected data. Through histogram equalization and contrast enhancement, this technology provides rapid enhancement in light fog conditions. It performs contrast-limited adaptive histogram equalization on images, adjusts global contrast through gamma correction, and restores natural colors through color balancing.

[0055] Image distortion correction: Using the Zhang Zhengyou calibration method, wide-angle distortion of drone lenses is corrected to improve the accuracy of target geometric features. The Zhang Zhengyou calibration method is a camera calibration method based on a planar checkerboard calibration plate. By capturing images of the calibration plate from multiple angles, the camera's intrinsic and extrinsic parameter matrices, as well as distortion coefficients, are calculated. Its advantages lie in its simplicity and high accuracy, making it suitable for drone lens calibration.

[0056] 1) Radial distortion: caused by the shape of the lens, manifested as the edges of the image expanding outward (barrel distortion) or shrinking inward (pincushion distortion).

[0057]

[0058] Parameter meaning: is the radial distortion coefficient, , is the lateral distortion coordinate, is the longitudinal distortion coordinate, x is the image horizontal coordinate, and y is the image vertical coordinate.

[0059] 2) Tangential distortion: caused by the installation deviation between the lens and the sensor, which appears as a tilted image.

[0060] Parameter meaning: , , is the longitudinal distortion coordinate, , x is the horizontal coordinate of the image, and y is the vertical coordinate of the image.

[0061] 3) Internal parameter matrix: contains focal length , principal point coordinates Parameters, describing the geometric characteristics of camera imaging:

[0062] Step 2: YOLO model training and optimization 1. Data collection and annotation 1) Collection requirements: Scenario coverage: covers road occupation business scenarios under different time periods (day / night), weather (sunny / rainy), and lighting conditions.

[0063] 2) Target category: Target 0: vendor carts (such as roadside tables and chairs, snack stalls, temporary roadside stalls, electric tricycles and other small targets); Target 1: Goods occupying the road (such as small vans, small trucks and other medium-sized targets); Target No. 2: Illegal billboards (such as billboards, vans, medium-sized trucks and other large targets).

[0064] 3) Data volume: Process at least 2000 labeled images, each containing 2-5 objects.

[0065] 4) Marking specifications: Use the LabelImg tool and the labeling format is the YOLO standard as follows: <class_id><x_center_normalized><y_center_normalized><width_normalized><height_normalized> 2. Dataset Division Ratio: training set (80%), validation set (20%), to ensure that the two types of data are distributed consistently (for example, the ratios for different scenarios are the same).

[0066] 3. Data augmentation strategy Mosaic enhancement is used: 4 images are randomly selected and stitched into 1, simulating the distribution of multiple targets in complex scenes, improving the model's detection ability for small targets and occluded targets.

[0067] 4. Random HSV perturbation Hue ±10%: Adapts to color changes under different lighting conditions.

[0068] Saturation ±20%: Enhances robustness to low-saturation target images.

[0069] Value ±15%: Simulates the appearance of a target in strong light or shadow.

[0070] 5. Other enhancements Random rotation (±15°), translation (±20%), and scaling (0.8-1.2x).

[0071] Adds Gaussian noise (simulates low-light camera noise).

[0072] 6. Model Configuration Choose YOLOv8s: It balances accuracy and speed and is suitable for deployment on edge devices (such as drone onboard computers).

[0073] Input size: 640×640 pixels, taking into account both detail preservation and computational efficiency.

[0074] Backbone network: CSPDarknet53 (cross-stage local network), optimized gradient flow.

[0075] 7. Hyperparameter settings Learning rate (LR): The initial value is 0.01.

[0076] Scheduling strategy: Cosine annealing (from the initial value to the minimum value of 0.001), formula:

[0077] Training rounds: 50-100 epochs (dynamically adjusted based on the convergence of the validation set mAP).

[0078] Batch size: Batch Size = 16 (GPU memory ≥ 8GB is required. If the GPU memory is insufficient, it can be reduced to 8 and the learning rate can be reduced accordingly).

[0079] Optimizer: SGD with momentum=0.9, weight decay=5e-4.

[0080] 8. Model training command yolo train \ data=dataset.yaml #Dataset configuration file model=yolov8s.yaml #Model structure configuration epochs=100 # Maximum number of training rounds batch=16 # batch size imgsz=640 # Input size lr0=0.01 # Initial learning rate weight_decay=5e-4 # weight decay cos_lr=True # Enable Cosine annealing pretrained=True # Load pretrained weights 9. Dataset Construction Collect images of street-side business scenes (including vendor carts, goods occupying the road, illegal billboards, etc.), and annotate them in the YOLO standard format (<class_id><x_center><y_center><width> <height>), the ratio of training set to validation set is 8:2, and data enhancement uses Mosaic (four-image stitching) and random HSV color perturbation (hue ±10%, saturation ±20%, lightness ±15%).

[0081] 10. Model training Use YOLOv8s (lightweight version), input size 640×640, initial learning rate 0.01, Cosine annealing strategy, training 50-100 epochs, Batch Size = 16 (GPU memory ≥ 8GB).

[0082] 11. Adaptive anchor box optimization According to the actual scale distribution of road occupation business targets, the preset anchor frame size is optimized, and nine groups of initial anchor frames (such as [11, 13], [16, 35], etc.) are generated through K-means clustering to match the target scale distribution.

[0083] A. Extract the annotation box size: Read the normalized width and height (width, height) from all annotations in the training set.

[0084] B. K-means clustering: Using the Euclidean distance metric, clustering is done into 9 groups of anchor boxes (YOLO has 3 detection layers by default, with 3 anchor boxes in each layer).

[0085] C. Calculate the intersection over union (IOU): Verify the matching degree between the clustered anchor box and the real box, and the goal is to minimize the IoU loss.

[0086] D. Assign the 9 anchor boxes to the three detection layers (P3, P4, P5) of YOLOv8s, corresponding to small, medium and large targets respectively: a) P3 layer (downsampled 8 times): Detects small objects such as tables and chairs on the road, snack stalls, temporary stalls on the road, and electric tricycles (such as anchor boxes [11, 13], [16, 30]).

[0087] b) P4 layer (downsampled 16 times): detects medium-sized objects such as minivans and minivans (such as anchor boxes [32, 22], [25, 61]).

[0088] c) P5 layer (downsampled 32 times): detects large objects such as billboards, vans, and medium-sized trucks (such as anchor boxes [53, 66], [80, 100]).

[0089] 12. Verification and Tuning Monitor training on the road occupation data, monitoring training loss, validation set, and category precision and recall. If there is no improvement on the validation set for 10 consecutive rounds, automatically terminate training. Save the best validation set. If training loss fluctuates: reduce the learning rate (e.g., 0.01 → 0.005) or increase the batch size. If the validation set is overfitting: increase data augmentation or add a dropout layer.

[0090] Figure 2 The YOLO algorithm optimization flow chart in this embodiment includes the following main processes: 1) Data preprocessing: A collaborative strategy of Mosaic enhancement and HSV perturbation is used to improve robustness in rainy / foggy scenes.

[0091] 2) Adaptive Anchor Frame Clustering: K-means generates nine groups of anchor frames to detect small objects such as tables and chairs on the road, snack stalls, temporary roadside stalls, and electric tricycles; medium-sized objects such as small vans and small trucks; and the aspect ratio characteristics of large objects such as vans and medium-sized trucks.

[0092] 3) Dynamic label assignment: A dual-weighted label assignment mechanism is used: IoU > 0.6 → high weight (confidence-weighted), and IoU < 0.6 → low weight, reducing the missed detection rate of small objects by 23%.

[0093] 4) Reasoning and behavior analysis: spatiotemporal joint judgment (stay time ≥ 10 seconds, area ≥ 2m 2 ), filter out false alarms of temporarily stopped pedestrians.

[0094] Through the above steps, a high-precision road-blocking business detection model can be trained to accurately identify carts, goods, and billboard targets, which is suitable for automated inspection scenarios in urban management.

[0095] Step 3: Dynamic Detection and Behavior Analysis 1. Real-time inference module (edge ​​computing deployment) Hardware Configuration Edge device parameters: NVIDIA Jetson Xavier NX (32GB eMMC, 8-core ARM CPU, 384-core Volta GPU, 48 Tensor Cores) Performance benchmark parameters: 21 TOPS computing power at FP16 precision, power consumption ≤15W, meeting the drone's endurance requirements.

[0096] 2. Model deployment process # Step 1: Model format conversion (PyTorch → TensorRT) yolo export model=best.pt format=engine device=0 half=True # Step 2: Deploy the inference engine import torch from ultralytics import YOLO model = YOLO("best.engine", task="detect") # Load TensorRT engine # Step 3: Real-time video stream processing cap = cv2.VideoCapture("rtsp: / / drone_feed") # UAV RTSP video stream while cap.isOpened(): ret, frame = cap.read() results = model(frame, conf=0.5, iou=0.4536, imgsz=640) # Key parameter settings 3. Performance optimization strategy

[0097] 4. Delay measured data End-to-end latency: 187ms (including 50ms for image transmission, 120ms for inference, and 17ms for post-processing).

[0098] 5. Behavior determination rules: Intelligent identification of road occupation A. Object Tracking and Spatiotemporal Filtering from collections import deque # Initialize target trajectory library target_tracks = { "cart": deque(maxlen=5), # Track queue of small objects such as tables and chairs on the road, snack stalls, temporary stalls on the road, and electric tricycles (5 frames) "goods": deque(maxlen=5), # small vans, small trucks and other medium-sized target trajectory queue "signboard": deque(maxlen=5) # Queue of large target trajectories such as billboards, vans, and medium-sized trucks } # Frame by frame processing for result in results: if result.conf > 0.5: # Calculate the actual coverage area (requires camera calibration parameters) pixel_area = result.width * result.height real_area = pixel_area * (gps_altitude**2) / focal_length**2 # Area conversion formula # Update trajectory library target_tracks[result.class_id].append({ "position": result.xywh[0], "area": ​​real_area, "timestamp": time.time() }) B. Alarm triggering logic ef check_violation(track): # Condition 1: The target exists for 5 consecutive frames if len(track) < 5: return False # Condition 2: Time window ≤ 10 seconds time_window = track[-1]["timestamp"] - track[0]["timestamp"] if time_window > 10: return False # Condition 3: Average area ≥ 2m 2 avg_area = sum(item["area"] for item in track) / 5 return avg_area >= 2.0 # Traverse all target categories for class_name, track in target_tracks.items(): if check_violation(track): send_alert(f"Road occupation warning: {class_name} continuous road occupation area {avg_area:.1f}㎡") C. Area Conversion

[0099] D. Cross-modal verification: multi-source data fusion Figure 3 This is a schematic diagram of multimodal data fusion, which shows the data fusion logic in a hierarchical structure: 1) The drone obtains longitude and latitude (GPS) and attitude angle (IMU) in real time, and the positioning data update frequency is ≥10Hz.

[0100] 2) The coordinates of the electronic fence on the ground side are accurate to the centimeter level and transmitted to the fusion center via LoRa wireless.

[0101] 3) Kalman filter fusion # Algorithm pseudocode def kalman_fusion(uav_gps, ground_fence): # State prediction: drone motion model compensation predicted_pos = predict(uav_gps) # Measurement Update: Geo-Fence Spatial Constraints fused_pos = update(predicted_pos, ground_fence) return fused_pos # Output sub-meter positioning (error ≤ 0.5m) 4) Target trajectory matching

[0102] 5) Alarm decision-making mechanism Matching successful → triggering the road occupation alarm - must also meet the following requirements: Coverage area ≥ 2m 2 + Stay time ≥ 10 seconds Matching failed → Filter false positives - such as a courier truck that stops temporarily.

[0103] E. Kalman filter fusion process 1) State vector: (center target coordinates + acceleration) 2) Prediction step:

[0104] Where F is the state transfer matrix and Q is the process noise.

[0105] 3) Update steps:

[0106] Where H is the measurement matrix, R is the measurement noise, and Z is the K For sensor data.

[0107] F. Geo-fence assisted verification Geofence data: For example: { "type": "Polygon", "coordinates": [[ [116.3912, 39.9075], / / Boundary point of the prohibited swing zone [116.3928, 39.9081], [116.3940, 39.9063] ]] } Validation logic: if target_in_polygon(drone_gps, ground_cam_gps, fence_polygon): confirm_alert() # The target is within the field of view of both the drone and the ground and is in the prohibited area Step 4: Alarm and closed-loop management 1. Set up a hierarchical response mechanism: Level 1 Alarm: The system automatically broadcasts a warning (such as "Please do not occupy the road for business") through the TTS voice module and pushes a screenshot of the violation to the urban management platform.

[0108] Secondary linkage: For persistent violations (repeated within 10 seconds), the nearest patrol car equipped with AR glasses (displaying the target location frame and historical records) will be dispatched to handle the situation on the spot.

[0109] Data archiving: Violation event records are stored as structured data (time, location, image evidence chain), supporting SQL queries and heat map analysis (such as statistics on high-incidence periods of violations).

[0110] 2. Device design A. UAV hardware configuration: Equipped with a 4K gimbal camera (supporting 30x zoom), RTK positioning module, and edge computing unit (computing power ≥ 10TOPS).

[0111] The battery life is ≥40 minutes and supports automatic return to home charging (the distance between wireless charging piles is ≤1km).

[0112] B. Ground Control Platform: A visual interface developed based on PyQt can display drone inspection images, detection frames and alarm information in real time, and support one-click export of violation reports (Excel format).

[0113] Figure 4 The figure shows an intelligent monitoring device for road occupation based on the YOLO algorithm and drone collaboration proposed in this application, which includes: The data acquisition module is used to collect image data of road occupation business scenes using drones along a preset grid path and pre-process the data; Model training module, used to train the YOLO algorithm based on preprocessed data to generate and optimize the target detection model; The detection and analysis module uses the optimized target detection model to perform target detection and time-series behavior analysis on the video stream collected by the drone in real time to determine whether there is any road occupation behavior; The alarm management module is used to trigger graded alarms based on the judgment results, record and process road-blocking business behaviors, and store violation records as structured data, supporting query and analysis to complete the closed-loop management process.

[0114] When the above device is running, the steps of the intelligent monitoring method for road occupation business based on the YOLO algorithm and drone collaboration disclosed in this application are implemented.

[0115] The flowcharts and block diagrams in the accompanying drawings illustrate possible implementations of the apparatus, methods, and computer program products according to various embodiments of the present application, including architecture, functions, and operations. In these figures, each box may represent a module, a program segment, or a portion of a code, which contains one or more executable instructions for implementing a specified logical function. It should be noted that each box in the block diagram and / or flowchart, and the combination of these boxes, can be implemented using a dedicated hardware-based system to implement the specified function or operation, or can be implemented by a combination of dedicated hardware and computer instructions.

[0116] like Figure 5 As shown, an embodiment of the present application further discloses an electronic device, comprising: a processor 310, a communication interface 320, a memory 330 for storing a computer program executable by the processor, and a communication bus 340. The processor 310, the communication interface 320, and the memory 330 communicate with each other via the communication bus 340. The processor 310 executes the executable computer program to implement the steps of the aforementioned method for intelligently monitoring road occupation based on the YOLO algorithm and drone collaboration.

[0117] It is understood that, in addition to the memory and processor, the electronic device may also include an input device (e.g., a keyboard), an output device (e.g., a display), and other communication modules. These input devices, output devices, and other communication modules all communicate with the processor via an I / O interface (i.e., an input / output interface).

[0118] The operation of the present application can be implemented by writing computer program code using one or more programming languages ​​or a combination thereof. The programming languages ​​include but are not limited to the following types: Object-oriented programming languages, such as Java, Smalltalk, C++, etc.; A conventional procedural programming language, such as "C" or a similar programming language.

[0119] The execution methods of the program code include but are not limited to: Executes entirely on the user's computer; Partially executed on the user's computer and partially on a remote computer; Executed as a standalone software package; Executes entirely on the remote computer or server.

[0120] In scenarios involving a remote computer, the remote computer can be connected to the user's computer via any type of network, including but not limited to a local area network (LAN) or a wide area network (WAN). Additionally, the remote computer can be connected to an external computer via an Internet service provider, such as the Internet.

[0121] Furthermore, the present application also discloses a computer-readable storage medium. When the instructions in the computer-readable storage medium are executed by the processor of an electronic device, the electronic device is enabled to execute the various steps of the intelligent monitoring method for road occupation business based on the YOLO algorithm and the collaboration of drones disclosed in the present application.

[0122] In the context of this application, computer-readable storage media refers to tangible media that can store computer program code and related data. Specific examples include, but are not limited to, the following: (1) Portable computer disk: A removable magnetic storage medium such as a floppy disk.

[0123] (2) Hard disk: includes fixed storage devices such as mechanical hard disks and solid-state hard disks.

[0124] (3) Random Access Memory (RAM): Volatile storage medium used for temporary storage of data and program code.

[0125] (4) Read-only memory (ROM): A non-volatile storage medium used to store fixed programs and data.

[0126] (5) Erasable Programmable Read-Only Memory (EPROM) or Flash Memory: A non-volatile storage medium that supports multiple erasing and programming.

[0127] (6) Fiber optic storage device: storage medium based on fiber optic technology.

[0128] (7) Compact Disc Read-Only Memory (CD-ROM): A read-only medium that stores data in the form of an optical disc.

[0129] (8) Optical storage devices: storage media based on optical principles, such as DVDs and Blu-ray discs.

[0130] (9) Magnetic storage devices: storage media based on magnetic principles, such as magnetic tapes and disks.

[0131] (10) Any suitable combination of the above: for example, combining multiple storage media to meet different storage requirements.

[0132] These computer-readable storage media can be used to store the program code and related data described in this application to support the operation of the program and the persistent storage of data.

[0133] In particular, according to embodiments of the present application, the processes described in the flowcharts can be implemented as computer software programs. For example, embodiments of the present application relate to a computer program product comprising a computer program borne on a non-transitory computer-readable medium. The computer program includes program code for executing the intelligent road occupation monitoring method based on the YOLO algorithm and drone collaboration disclosed in this application. When the computer program is executed by a processing device, it can implement the above-mentioned functions defined in the embodiments of the present application.

[0134] Although the above discussion contains several specific implementation details, these details should not be interpreted as limiting the scope of this application. The above description is only a preferred embodiment of the present application and an illustration of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in this application is not limited to the technical solutions formed by the specific combination of the above technical features. At the same time, this application should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concepts.

[0135] Those skilled in the art should also understand that they may modify the technical solutions described in the aforementioned embodiments, or replace some of the technical features therein with equivalents, without departing from the spirit and scope of the technical solutions of the embodiments of the present application. Such modifications or replacements will not cause the essence of the corresponding technical solutions to deviate from the core spirit and scope of the technical solutions of the embodiments of the present application.< / height> < / width>

Claims

1. An intelligent monitoring method for road occupation based on the YOLO algorithm and drone collaboration, characterized by: The method comprises: S1: Use drones to collect image data of road occupation business scenes along a preset grid path and pre-process the data; S2: Train the YOLO algorithm based on the preprocessed data to generate and optimize the target detection model; S3: Use the optimized target detection model to perform target detection and time series behavior analysis on the video stream collected by the drone in real time to determine whether there is any road occupation behavior; S4: Trigger graded alarms based on the judgment results, record and process road-blocking business behaviors, and store violation records as structured data to support query and analysis, completing the closed-loop management process.

2. The method according to claim 1, characterized in that The step S1 described in which the image data of the road-occupying business scene is collected by the drone according to a preset grid path includes: the drone is equipped with an RTK positioning module, and the image data of the road-occupying business scene is collected according to a preset 100m×100m grid path. The drone speed is ≤5m / s, the flight altitude is 30-50m, and the collected image resolution is ≥1920×1080.

3. The method according to claim 1, characterized in that The data is pre-processed in step S1, including: S11: Enhance images in low-light or hazy scenes by using histogram equalization combined with gamma correction to restore contrast; S12: Based on Zhang Zhengyou’s calibration method, the radial distortion and tangential distortion of the drone lens are corrected. The radial distortion correction formula is: Where, is the radial distortion coefficient, is the lateral distortion coordinate, is the longitudinal distortion coordinate, x is the image horizontal coordinate, and y is the image vertical coordinate; The tangential distortion correction formula is: Where, , , is the longitudinal distortion coordinate, , x is the horizontal coordinate of the image, y is the vertical coordinate of the image; S13: The scene data with accurate geometric features is described by the intrinsic parameter matrix. The formula of the intrinsic parameter matrix is: Where, is the focal length, The main point coordinates.

4. The method according to claim 1, wherein The training and optimization of the target detection model in step S2 includes: S21: Collect images of roadside business operations under different lighting conditions, including daytime, nighttime, sunny, and rainy days, and label small, medium, and large objects in the YOLO standard format. S22: Divide the labeled data into training and validation sets, and use Mosaic data augmentation and random HSV perturbation to improve model robustness; S23: YOLOv8s is selected as the base model, with an input size of 640×640 and a CSPDarknet53 backbone network. Nine groups of initial anchor boxes are generated through K-means clustering and assigned to the P3, P4, and P5 detection layers of YOLOv8s to match the scale distribution of small, medium, and large objects, respectively. S24: A dynamic label assignment strategy is used, combined with cosine annealing learning rate scheduling and SGD optimizer training model, with 50-100 epochs and a batch size of 16.

5. The method according to claim 4, characterized in that The three types of targets described in step S21, namely small, medium and large, include: Small targets include vendor carts, pavement tables and chairs, food stalls, temporary pavement stalls, and electric tricycles; Medium-sized targets include road-blocking cargo, small vans, and small trucks; Large targets include illegal billboards, vans, and medium-sized trucks.

6. The method according to claim 4, characterized in that The three detection layers of YOLOv8s described in step S23, where P3 is downsampled 8 times, P4 is downsampled 16 times, and P5 is downsampled 32 times; The dynamic label allocation strategy described in step S24 is high weight when IoU>0.6 and low weight when IoU≤0.6; the initial learning rate of the cosine annealing learning rate scheduling is 0.01, and the cosine annealing is reduced from the initial value to the minimum value of 0.001; the momentum of the SGD optimizer is 0.9, and the weight decay is 5e-4.

7. The method according to claim 1, characterized in that Step S3 includes: S31: Deploy the optimized object detection model to the edge computing unit and achieve low-latency inference through TensorRT acceleration; S32: Target detection is performed on the video stream collected by the drone in real time, and the target category, location, confidence level and pixel area are output; S33: Track the target trajectory based on the DeepSORT or FairMOT algorithm and calculate the actual coverage area. The area conversion formula is: S34: Determine whether there is any business activity occupying the road. The determination method is a spatiotemporal joint determination. The determination rule is: if the target is detected in a fixed area for 5 consecutive frames or the stay time is ≥10 seconds, and the average actual coverage area is ≥2m², then it is determined that there is business activity occupying the road.

8. The method according to claim 1, characterized in that The graded alarm in step S4 includes: Level 1 Alarm: triggers a cloud broadcast to play a warning message and pushes a screenshot of the violation to the urban management platform; if not handled within 15 minutes, it will be upgraded to Level 2 Alarm; Level 2 alert: Dispatch patrol cars equipped with AR glasses overlay the illegal target location frame and historical records in real time to assist law enforcement officers in handling the situation.

9. An intelligent monitoring device for road occupation based on the YOLO algorithm and drone collaboration, characterized by: The device comprises: The data acquisition module is used to collect image data of road occupation business scenes using drones along a preset grid path and pre-process the data; Model training module, used to train the YOLO algorithm based on preprocessed data to generate and optimize the target detection model; The detection and analysis module uses the optimized target detection model to perform target detection and time-series behavior analysis on the video stream collected by the drone in real time to determine whether there is any road occupation behavior; The alarm management module is used to trigger graded alarms based on the judgment results, record and process road-blocking business behaviors, and store violation records as structured data, supporting query and analysis to complete the closed-loop management process.

10. An electronic device, characterized in that: include: memory and processor; Memory: used to store computer programs; Processor: used to execute the computer program to implement the steps of the intelligent monitoring method for road occupation business based on the YOLO algorithm and the collaboration of the drone as described in any one of claims 1-8.

Images