On-road parking peccancy detection method and system based on view angle of unmanned aerial vehicle

Abstract

The invention relates to the technical field of traffic engineering, in particular to a curb parking peccancy detection method and system based on an unmanned aerial vehicle visual angle. The method comprises the following steps: step 1, acquiring an image and transmitting data based on an unmanned aerial vehicle, and performing data preprocessing to obtain image data; step 2, constructing a Hyper-Yolov1.1'backbone network-neck network-detection head 'target detection framework; step 3, based on the vehicle information of the DroneVehile data set, carrying out detection model training and optimization, identifying an image, and outputting information such as the position, size, category and confidence of a vehicle bounding box; step 4, based on parking space corner information of the PSD data set, carrying out detection model training and optimization, identifying an image, and outputting information such as position, size, category and confidence of a parking space corner bounding box; and step 5, constructing a parking space bounding box according to the parking space angular points, and performing violation judgment based on a double-threshold strategy. By adopting the technical scheme, efficient, accurate and low-misjudgment curb parking peccancy detection is realized.

Description

Technical Field

[0001] This invention relates to the field of traffic engineering technology, specifically to a method and system for detecting on-street parking violations based on the perspective of an unmanned aerial vehicle (UAV). Background Technology

[0002] With the continuous increase in the number of cars in my country, illegal parking in urban on-street parking areas has become an increasingly prominent problem, not only affecting traffic efficiency but also easily causing road safety hazards. Therefore, accurate and efficient on-street parking violation detection has become a key measure to regulate parking order and ensure smooth traffic flow.

[0003] Currently, the detection of illegal parking on the street mainly relies on manual patrols, fixed monitoring equipment, and satellite imagery. Manual patrols are costly, inefficient, and prone to omissions due to incomplete coverage. While fixed monitoring equipment can achieve real-time monitoring of some areas, it is limited by installation location and angle, resulting in numerous blind spots and insufficient detection accuracy. In addition, satellite navigation images are greatly affected by the environment, making direct detection difficult to meet the requirements for precision and leading to a high false positive rate.

[0004] Therefore, existing on-street parking violation detection methods have the following shortcomings: First, the detection efficiency of existing on-street parking violation detection methods is limited; second, the detection accuracy is not high; third, the detection precision is insufficient, the false judgment rate is high, and it is difficult to adapt to complex and ever-changing parking environments. Therefore, there is an urgent need to design intelligent, efficient, accurate, and low-false-judgment violation detection methods for on-street parking. Summary of the Invention

[0005] To address the problems of low efficiency, insufficient accuracy, and high false positive rate in existing on-street parking violation detection methods, this invention proposes an on-street parking violation detection method and system based on the perspective of an unmanned aerial vehicle (UAV). This method aims to acquire image information using a visual sensor mounted on a UAV, and by integrating two core technologies—vehicle detection and parking space detection—to accurately identify on-street parked vehicles and their respective parking spaces. Based on a dual-threshold strategy, it intelligently judges violations according to the spatial relationship between the vehicle and the parking space, thereby achieving efficient, accurate, and low-false-positive on-street parking violation detection.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0007] A method for detecting on-street parking violations based on the perspective of an unmanned aerial vehicle (UAV) is carried out according to the following steps:

[0008] Step 1: Acquire images and transmit data using the drone, and perform data preprocessing to obtain image data;

[0009] Step 2: Construct the Hyper-Yolov 1.1 target detection framework consisting of a "backbone network - neck network - detection head";

[0010] Step 3: Based on the vehicle information in the DroneVehicle dataset, train and optimize the detection model, recognize the image, and output information, including the position, size, category, and confidence information of the vehicle bounding box.

[0011] Step 4: Based on the parking space corner information of the PSD dataset, train and optimize the detection model, recognize the image, and output information, including the position, size, category, and confidence information of the parking space corner bounding box.

[0012] Step 5: Construct the parking space bounding box based on the corner points of the parking space, and determine the violation based on the dual threshold strategy. The dual thresholds are the intersection-union ratio of the vehicle bounding box and the parking space bounding box and the longitudinal normalized offset of the vehicle center point.

[0013] Furthermore, step 1 includes the following steps:

[0014] Step 1.1: The UAV flies automatically along the planned route. The imaging equipment on the UAV adjusts the exposure parameters and continuously captures clear original images. Then, the original image data is sent to the ground receiving equipment through the data transmission module.

[0015] Step 1.2: Perform data preprocessing on the received raw image data. Data preprocessing includes image enhancement, data enhancement, and format standardization.

[0016] Furthermore, step 2 includes the following steps:

[0017] Step 2.1: Construct a hybrid aggregation network as the backbone network to extract features. Adjust the number of input feature channels through 1×1 convolution to obtain intermediate features. After multi-path differentiation processing, the features are spliced ​​and aggregated to extract target enhancement features that can be adapted to key corner points of vehicles and parking spaces.

[0018] Step 2.2: Construct a hypergraph-based cross-layer and cross-location representation network as the neck network fusion feature, process the backbone network output features in stages, construct a hypergraph and complete convolution, and fuse to generate a multi-scale feature map for use by the detection head, so as to achieve efficient fusion and optimization of target features;

[0019] Step 2.3: Configure the decoupled detection head to receive the multi-scale feature map output by the neck network. Output the category probability and position size of the vehicle and parking space corner points through independent branches. Combine confidence prediction and non-maximum suppression to obtain accurate detection results of the vehicle and parking space corner points.

[0020] Further, in step 2.2, the construction of the hypergraph and the completion of convolution specifically involves:

[0021] Super Edge Aggregated feature vectors With vertex Update features Represented as:

[0022] (4)

[0023] in, For the super edge Aggregated feature vectors, Represents vertices Does it belong to the super-edge? , For set Number of elements in the middle As vertices The original characteristics; This represents the trainable weight parameter matrix, with dimension 1. Used for vertex features Perform a linear transformation; Represents vertices Update features, Indicates super edge Does it contain vertices? , For set The number of elements in the middle represents the vertex. The number of associated superedges;

[0024] The hypergraph convolution representation of the two-stage hypergraph message passing matrix is:

[0025] (5)

[0026] in, For hypergraph convolution operations, As vertices Features included Represents the vertex-hyperedge incidence matrix. Represents the vertex degree matrix. Represents the hypermarginality matrix. Combinatorial representation normalizes feature information.

[0027] Furthermore, step 3 includes the following steps:

[0028] Step 3.1: Data preparation and preprocessing. Adjust the number and names of vehicle categories in the DroneVehicle dataset, convert the original annotation format, and record the category ID, normalized coordinates of the center point, and normalized width and height of the vehicle target to ensure that the data is compatible with the input requirements of the Hyper-YOLOv1.1 framework.

[0029] Step 3.2, Model Training and Optimization: GELAN-C-Hyper is selected as the backbone network, and hyperparameter configuration is loaded. The total loss function is composed of bounding box regression loss, classification loss, and target existence loss. A vehicle detection model adapted to the UAV perspective is trained.

[0030] Step 3.3, Model Deployment and Detection: Deploy the trained and optimized model weights to the actual scene, receive images collected in real time by the UAV, and output the detected vehicle bounding boxes, category information and confidence scores to achieve high-precision vehicle detection.

[0031] Furthermore, step 4 includes the following steps:

[0032] Step 4.1: Data preparation and preprocessing. Adjust the parking space label attributes of the PSD dataset, add type and valid status labels, and convert the parking space corner coordinates to the bounding box format adapted by Hyper-YOLOv1.1. Record the parking space corner parameters, remove fuzzy samples, and ensure that the data meets the model input requirements.

[0033] Step 4.2, Model Training and Optimization: Using GELAN-M as the backbone network, a hierarchical strategy is adopted to train the model on the parking space corner data in the PSD dataset. In terms of training strategy, the loss weights and data preprocessing process are adjusted, the number of training iterations is increased, and hyperparameter configurations for parking space corner detection are loaded.

[0034] Step 4.3, Model Deployment and Detection: Deploy the trained model weights to the detection system. By receiving the images collected by the UAV in Step 1, output the size, position, and confidence of the parking space corner bounding boxes, providing accurate positioning data for the subsequent construction of parking space bounding boxes.

[0035] Furthermore, step 5 includes the following steps:

[0036] Step 5.1: Parking space key point extraction and line fitting. Extract the center point of the corner bounding box of the parking space as key points and set them into a point set. Use the least squares method to fit the line. By solving the normal equation, obtain the final line parameters of the outer boundary of the parking space.

[0037] Step 5.2: Constructing the rectangular bounding box of the parking space. Based on the effective intersection points of the fitted straight line and the image boundary, and combined with the preset parking space width, draw perpendicular lines to the corner points to determine the precise position and size of the rectangular bounding box of the parking space.

[0038] Step 5.3, Dual Threshold Calculation and Violation Judgment: Calculate the intersection-union ratio (IUR) of the vehicle bounding box and the parking space bounding box, and the longitudinal normalized offset of the vehicle center point; when the IUR is not lower than the threshold value... Furthermore, the longitudinal normalized offset of the vehicle center point does not exceed the threshold. If the time frame is met, it is deemed compliant; otherwise, it is deemed a violation. This represents the threshold value for the intersection-union ratio (IoU) between the vehicle bounding box and the parking space bounding box. This represents the threshold for the longitudinal normalized offset of the vehicle's center point.

[0039] Further, in step 5.1, the extraction of key parking space points and line fitting specifically involves:

[0040] First, the center point of the parking space corner detection box is extracted as the key point; the formula for calculating the normalized coordinates of the center point of the bounding box is:

[0041] (twenty three)

[0042] (twenty four)

[0043] in, The coordinates of the center point of the parking space corner detection frame. and These are the pixel coordinates of the two ends of the bounding box of the parking space corner. , These are the image width and image height, respectively.

[0044] The detected normalized center points are set into a point set. ,in It is the number of key points detected. It is the index of the key point in the image;

[0045] Secondly, the least squares method is used to fit the key points to determine a straight line representing the direction of the parking space; the objective is to minimize the actual observed values. Compared with model predictions The sum of squared residuals between them, and the minimization of residuals, are expressed as:

[0046] (25)

[0047] in, The slope of the fitted line, The intercept of the fitted line, Normalized coordinates for the center point;

[0048] To facilitate the solution, the point set Transform into matrix form and construct an augmented matrix. and observation vector Represented as:

[0049] (26)

[0050] in, For augmented matrices, For observation vectors;

[0051] By solving the normal equation The optimal solution is expressed as:

[0052] (27)

[0053] in, , This is the optimal solution for the linear parameters corresponding to minimizing the residuals. The optimal solution is obtained by finding this optimal solution. The equation of the fitted straight line is obtained.

[0054] Furthermore, in step 5.3, the dual threshold calculation and violation determination specifically involve:

[0055] The intersection-union ratio (IU / UU) formula and constraints for the vehicle bounding box and the parking space bounding box are as follows:

[0056] (29)

[0057] in, For vehicle bounding box, For parking space boundary boxes, This represents the intersection area of ​​the vehicle bounding box and the parking space bounding box. The area is the union of the vehicle bounding box and the parking space bounding box.

[0058] (30)

[0059] in, This is the ratio of the intersection area to the union area of ​​the vehicle bounding box and the parking space bounding box. This is the threshold for the intersection-union ratio (IoU) of the vehicle bounding box and the parking space bounding box. The range is ;

[0060] The formula and constraints for the longitudinal normalized offset of the vehicle center point are as follows:

[0061] (31)

[0062] in, This represents the longitudinal normalized offset of the vehicle's center point. The normalized ordinate of the vehicle bounding box center point. Normalized coordinates of the center point of the parking space boundary box;

[0063] (32)

[0064] in, The threshold representing the longitudinal normalized offset of the vehicle center point. The range is .

[0065] Based on the same inventive concept, the present invention also provides an on-street parking violation detection system based on UAV perspective, used to implement the above-mentioned on-street parking violation detection method based on UAV perspective. The system includes a preprocessing module, a target detection framework module, a vehicle and parking space corner detection module, and a violation judgment module.

[0066] Compared with the prior art, the present invention has the following beneficial technical effects:

[0067] This invention proposes a method for detecting on-street parking violations based on the perspective of unmanned aerial vehicles (UAVs). By employing UAVs for high-altitude overhead data collection and automated flight path planning, intelligent and efficient on-street parking violation detection is achieved, effectively overcoming the drawbacks of limited efficiency, incomplete coverage, and easy omissions associated with manual patrols. The maneuverability and wide coverage of UAVs enable them to quickly and continuously scan large areas, significantly shortening detection time. Embodiments of this invention show that the method improves violation detection efficiency by 25.1%, providing a more efficient and economical solution for on-street parking violation management.

[0068] This invention integrates the advanced Hyper-YOLOv1.1 target detection framework with high-quality, multi-angle images acquired by drones from different angles, achieving higher detection accuracy and effectively overcoming the blind spot problem caused by the limited installation location and angle of fixed monitoring equipment. Trained on a refined dataset, the model can accurately identify vehicles and parking spaces, with an overall detection accuracy 12.3% higher than fixed monitoring, effectively compensating for the blind spot problem of fixed monitoring.

[0069] This invention utilizes high-quality, multi-angle images captured by drones, which are less affected by environmental factors. Combined with indirect, precise detection of parking spaces at corner points and refined analysis using a dual-threshold strategy, it achieves a significantly reduced false positive rate. This effectively overcomes the drawbacks of satellite navigation images, which are greatly affected by environmental factors and lack sufficient refinement in direct detection, leading to errors in judgment. The stable, high-definition images captured by drones, combined with indirect, precise detection and refined analysis using a dual-threshold strategy, more accurately identify the parking status of vehicles, reducing the false positive rate by 26.96% and ensuring the correctness of violation judgments. Attached Figure Description

[0070] Figure 1 This is a schematic diagram of the on-street parking violation detection method based on the perspective of an unmanned aerial vehicle (UAV) according to the present invention.

[0071] Figure 2 This is a schematic diagram of the hybrid aggregation network (MANet) in this invention;

[0072] Figure 3This is a schematic diagram of the HyperC2Net, a cross-layer and cross-location representation network based on a hypergraph, in this invention.

[0073] Figure 4 This is a schematic diagram of the parking space corner point and its coordinates in step 5.1 of the specific implementation method;

[0074] Figure 5 This is a flowchart illustrating the design of the violation detection based on a dual-threshold strategy in this invention.

[0075] Figure 6 This is a schematic diagram showing the location of the roadside parking lanes for three different scenarios in a specific embodiment. Detailed Implementation

[0076] To make the objectives, technical solutions, and advantages of this invention clearer, a detailed description is provided below in conjunction with the accompanying drawings.

[0077] In recent years, drones have been widely used in the transportation sector due to their multifunctionality, flexibility, and wide coverage, especially in traffic monitoring and violation detection. Furthermore, with the continuous advancement of technologies such as high-precision imaging, real-time data transmission, and visual recognition, the reliability of drones in detecting on-street parking violations has been continuously improving. In violation detection, the high-altitude perspective of drones, with its ability to achieve real-time, blind-spot-free coverage of the detection area, has broad application prospects. To address the shortcomings of existing on-street parking violation detection technologies, such as low detection efficiency, low accuracy, insufficient precision, and high false positive rate, this invention proposes an on-street parking violation detection method and system based on the perspective of a drone. Specific Implementation Method 1

[0079] This invention provides a method for detecting on-street parking violations based on the perspective of an unmanned aerial vehicle (UAV), such as... Figure 1 As shown, the procedure is as follows:

[0080] Step 1: Acquire images and transmit data based on the drone, and perform data preprocessing to obtain image data.

[0081] Step 1.1: The UAV flies automatically along the planned route. The imaging equipment on the UAV adjusts the exposure parameters and continuously captures clear original images. Then, the original image data is sent to the ground receiving equipment through the data transmission module.

[0082] During the image acquisition process, the drone first plans a flight path covering the detection area using a pre-programmed procedure. Then, the drone executes an automatic flight mission along the planned path, continuously capturing images using its onboard imaging equipment. The captured raw image data is stored in the drone's onboard storage medium. Simultaneously, the image data is wirelessly transmitted in real-time and stably to ground receiving equipment via the drone's high-speed data transmission module.

[0083] Step 1.2: Perform data preprocessing on the received raw image data. Data preprocessing includes image enhancement, data enhancement, and format standardization.

[0084] After receiving the stored raw image data, the images are preprocessed to transform the original UAV imagery into standardized, high-quality image data suitable for subsequent model analysis. Data preprocessing includes image enhancement, data enhancement, and format standardization.

[0085] Image enhancement focuses on improving the visual quality of an image, highlighting key information about the target, and reducing interference. This includes applying filtering techniques to suppress and eliminate random noise, adjusting contrast and brightness to enhance the visual depth of the image, and blurring or sharpening the image based on its characteristics to smooth the image or enhance the edge and texture features of the target.

[0086] Data augmentation techniques adjust the pose and angle of acquired images to optimize how targets are presented in the image, simplifying data processing in subsequent detection processes. These include geometric transformations such as image rotation and scaling.

[0087] Format standardization converts all input image data into a uniform format and size to meet the model's input requirements and ensure consistent processing. This includes uniform image format conversion, size normalization, and pixel value standardization.

[0088] Applying the aforementioned enhancements and format standardization to the original image simultaneously can provide high-quality image data that is compatible with the model for subsequent detection of vehicle and parking space corners.

[0089] Step 2: Construct the Hyper-Yolov 1.1 target detection framework consisting of a backbone network, a neck network, and a detection head. Hyper-Yolov 1.1 is based on the YOLOv8 framework, using a hybrid aggregation network (MANet) as the backbone network and a hypergraph-based cross-layer and cross-location representation network (HyperC2Net) as the neck network.

[0090] Step 2.1, see Figure 2A hybrid aggregation network (MANet) is constructed as the backbone network to extract features. The number of input feature channels is adjusted by 1×1 convolution to obtain intermediate features. After multi-path differentiation processing, the features are spliced ​​and aggregated to extract target enhancement features that can be adapted to key corner points of vehicles and parking spaces.

[0091] MANet collaboratively integrates three typical convolutional variants. First, it performs channel expansion and preliminary transformation using 1×1 convolutions to transform the input features. The number of channels was adjusted to And obtain intermediate features ,in Indicates the target number of output channels.

[0092] The intermediate features are then subjected to multi-path differentiation processing, and the number of channels is compressed back to its original value. .

[0093] Path pair Then perform a 1×1 convolution to output the features. Recalibrate the semantic weights between channels.

[0094] Path 2 uses depthwise separable convolution (DSConv), first expanding the number of channels to 1×1 convolution. Then, depthwise convolution is used to extract spatial features for each channel individually. Finally, 1×1 convolution is used to integrate the spatial features of different channels, compressing the number of channels back to normal. Output features .

[0095] Path 3 employs a cross-stage local fusion module (C2f) to integrate intermediate features. Divided into , Two parts, Direct transmission preserves low-level features. pass The convolutional neck module, consisting of "3×3 convolution + residual connection", extracts high-order features. .

[0096] Low-level texture features and high-level semantic features are fused through subsequent splicing.

[0097] The structure of MANet can be represented as follows:

[0098] (1)

[0099] in, For input features, , , All are 1×1 convolutions. As an intermediate feature, For depthwise separable convolution, For feature segmentation, The convolution neck is a fusion of 3×3 convolution and residual connections. The target number of output channels, and the output characteristics of each path. The number of channels is , The number of times the convolution neck is executed.

[0100] Finally, these features from different paths are concatenated and aggregated, and then compressed using a 1×1 convolution to unify the total number of channels. The final output is enhanced features. , As shown below:

[0101] (2)

[0102] in, To enhance the features of the final output, To merge convolutions, Output features for each path. This is for splicing operations.

[0103] MANet aggregates information extracted from different paths, effectively extracting features at different scales. This allows for a better understanding of the contextual information of vehicles and parking space corners in the parking environment, resulting in richer and more distinctive feature representations. It also improves the ability to perceive targets from different perspectives and sizes, extracting target features at different scales.

[0104] Step 2.2, see Figure 3 A hypergraph-based cross-layer and cross-location representation network (HyperC2Net) is constructed as the neck network fusion feature. The output features of the backbone network are processed in stages, a hypergraph is constructed and convolution is performed, and multi-scale feature maps are fused to generate a multi-scale feature map for use by the detection head, thereby achieving efficient fusion and optimization of target features.

[0105] HyperC2Net introduces the concept of a "hypergraph" to model the complex relationships between features, breaking the limitations of traditional feature pyramid networks. It can capture the correlation between features at different levels and spatial locations, enabling effective feature fusion. The construction process can be divided into three key steps: hypergraph construction, hypergraph convolution, and feature fusion using the HGC-SCS framework.

[0106] (1) Hypergraph construction

[0107] First, the output features of the backbone network are divided into five discrete feature stages, and corresponding feature map representations are generated, denoted as... Corresponding to different semantic levels ( Low-level texture features, (These are high-order semantic features). Channel-level connections are made between these five basic feature maps to form richer cross-layer visual features.

[0108] SuperMap Typically determined by its vertex set and super edge set Definition: In calculations, the incidence matrix is ​​commonly used. This is represented by the following: First, the visual features synthesized from the connections are deconstructed to form the vertex set of the hypergraph. Secondly, a distance-threshold-based strategy is used to construct hyperedges. For each vertex in the hypergraph... , using distance threshold to build The hyperedges consist of all feature points within a distance threshold. The constructed hyperedge set... It can be represented as:

[0109] (3)

[0110] in, As vertices The super-edge set, For a vertex in the vertex set, Represents vertices With vertex The distance between them Distance threshold For a vertex set; when the vertex distance is less than a threshold At that time, the vertex is considered With vertex They are semantically related.

[0111] (2) Hypergraph Convolution

[0112] HyperC2Net employs convolutional operations based on a hypergraph structure. It applies a spatial hypergraph convolution method with additional residual connections to perform high-order learning of vertex features, facilitating the propagation of high-order messages across the hypergraph structure.

[0113] The hypergraph convolution process first needs to define the relationship between vertices and hyperedges, and then use the calculated degree matrix to assist in feature aggregation. Specifically, for a given vertex... and super-edge Two indicator functions were introduced. and . Represents vertices Does it belong to the super-edge? ,like but Otherwise, it is 0. Indicates super edge Does it contain vertices? ,like but Otherwise, it is 0.

[0114] Within this framework, the feature update of a single vertex can be described as a weighted sum of its original features and the neighborhood features aggregated through the hyperedge. Specifically, the hyperedge... Aggregated feature vectors With vertex Update features It can be represented as:

[0115] (4)

[0116] in, For the super edge Aggregated feature vectors, Represents vertices Does it belong to the super-edge? , For set The number of elements in the middle represents the super-edge. The total number of vertices connected; As vertices The original characteristics, This represents the trainable weight parameter matrix, with dimension 1. Used for vertex features Perform a linear transformation; Represents vertices Update features, Indicates super edge Does it contain vertices? , For set The number of elements in the middle represents the vertex. The number of associated superedges.

[0117] Introducing the vertex-hyperedge incidence matrix ,in The total number of vertices. Let be the total number of superedges, and Satisfy: If vertex Belongs to the super-edge If the value is 1, then it is 0; otherwise, it is 0. This is used to quantify the subordinate relationship between vertices and superedges.

[0118] diagonal matrix This is a vertex degree matrix, where the diagonal elements represent vertices. The total number of superedges; similarly, define a diagonal matrix. This is a hyperedge degree matrix, where the diagonal elements represent hyperedges. The total number of vertices included. Therefore, the hypergraph convolution of the two-stage hypergraph message passing matrix can be expressed as:

[0119] (5)

[0120] in, For hypergraph convolution operations, As vertices Features included Represents the vertex-hyperedge incidence matrix. Represents the vertex degree matrix. Represents the hypermarginality matrix. Combined representation of feature information normalization, This represents the trainable weight parameter matrix. This allows for the aggregation of information about different vertices through their shared hyperedges and the other vertices connected by these hyperedges.

[0121] (3) Fusion features of the HGC-SCS framework

[0122] By combining hypergraph construction and convolution strategies, the HGC-SCS framework is introduced for feature fusion. First, multi-layer feature maps are... By cascading along the channel dimension, a unified fusion feature is formed. Represented as:

[0123] (6)

[0124] in, Indicates a unified and integrated characteristic. This indicates a matrix cascading operation.

[0125] After constructing the hypergraph structure, high-order perceptual features are obtained by using hypergraph convolution. :

[0126] (7)

[0127] in, Represents higher-order perceptual features. For hypergraph convolution operations, To unify and integrate characteristics, This represents the vertex-hyperedge incidence matrix.

[0128] HyperC2Net introduces a fusion function. High-order perceptual features The process of fusing features with the original backbone network to enhance the model's perceptual capabilities is called "semantic scattering." Selective fusion... The core semantic features output by the backbone network Generate the feature map that is ultimately output to the detection head. The fusion process can be represented as:

[0129] (8)

[0130] in, This represents the final detection map that includes vehicle features and parking space corner features. The fusion function includes residual addition and C2f smoothing process. For higher-order perceptual features, This is the feature information map output from the backbone network before fusion.

[0131] HyperC2Net models complex spatial relationships between features using hypergraphs, and is particularly adept at capturing the fine geometric and semantic relationships between vehicles and parking spaces from a drone's top-down perspective. This enhances localization accuracy and robustness in complex scenarios, providing strong support for accurate traffic violation detection.

[0132] Step 2.3: Configure the decoupled detection head to receive the multi-scale feature map output by the neck network. Output the category probability and position size of the vehicle and parking space corner points through independent branches. Combine confidence prediction and non-maximum suppression to obtain accurate detection results of the vehicle and parking space corner points.

[0133] Hyper-YOLOv1.1 adopts the decoupled detection head design of YOLOv8, separating the two core tasks of classification and regression. The decoupled detection head process is divided into input feature reception, decoupling branch, confidence prediction, and multi-scale fusion.

[0134] (1) Input feature reception

[0135] The detection head receives enhanced multi-scale feature maps from the output of the neck network HyperC2Net. For any scale feature map ,in The scale representing the feature map The size is usually ,in and Represents the spatial dimension of the feature map, while Represents the number of feature channels.

[0136] (2) Decoupling branches

[0137] For each scale of feature map, the detection head is equipped with independent classification and regression branches.

[0138] The classification branch assigns the most probable class label to the potential target for each scale of the feature map. The classification branch employs a series of 3×3 convolutional layers, accompanied by BN normalization and SiLU activation functions, to further generate high-level semantic representations.

[0139] (9)

[0140] (10)

[0141] in, For feature scale The corresponding feature map, For input features, For the first Features after convolution To refine the number of convolutional layers, The dimension of the feature map is consistent with the output. BN represents normalization, and SiLU represents the activation function.

[0142] Subsequently, a 1×1 convolutional layer is applied to output the refined features. , The index of the last convolutional layer is used to adjust the feature channels to equal the total number of target classes. Output feature map Original scores :

[0143] (11)

[0144] in, For feature scale The corresponding feature map Refined features after convolution For 1×1 convolution, For feature map The corresponding original scores, i.e. have Each channel has a pixel value at any location in each channel, which represents the original score for that location in that category.

[0145] Finally, the raw score for each prediction is extracted and transformed into a probability distribution with a sum of 1 across all categories using the Softmax activation function. This yields the probability that the predicted point belongs to each category. For any position on the feature map... It belongs to the category probability It can be represented as:

[0146] (12)

[0147] in, Corresponding to feature map Upper position This indicates that it belongs to the category. The probability of; This indicates exponentiation. This represents the total number of target categories.

[0148] The regression branch is responsible for accurately predicting the spatial location and size of the target. For each input feature map, the regression branch applies several 3×3 convolutional layers to extract local geometric features related to the target's location and size. Subsequently, these high-dimensional features are mapped onto the regression parameters of the bounding box through a special 1×1 convolutional layer. The output of this convolutional layer has a fixed number of 4 channels, corresponding to the x-coordinate offset of the target's center point. , center point ordinate offset Scaling factor of target width relative to feature map grid and the scaling factor of the target height relative to the feature map grid .

[0149] and After processing with the Sigmoid activation function to limit it to the range [0, 1], it ensures that the predicted center point is roughly aligned with the center point of the grid cell. and go through The use of exponential functions facilitates better learning of scale changes.

[0150] The original parameters are further combined with the stride of the feature map, for a grid cell on the feature map. The coordinates of its upper left corner on the original image are Predicted bounding box center point coordinates The dimensions can be obtained through the following conversions:

[0151] (13)

[0152] in, The coordinates of the center point of the bounding box. For bounding box dimensions, This represents the size of the receptive field corresponding to the scale feature map. The location of the grid cell. These are the horizontal and vertical offsets of the center point, respectively. These are the scaling factors for the target width and height relative to the feature map grid, respectively.

[0153] (3) Confidence prediction

[0154] Confidence prediction is achieved by receiving feature maps from each scale. The features are processed by a series of convolutional layers to extract information about the existence of the target, and a 1×1 convolutional layer is applied to output a confidence score.

[0155] Applying the Sigmoid activation function to the output transforms it into an easily interpretable probability. For any location on the feature map... Its predicted confidence score It can be represented as:

[0156] (14)

[0157] in, This indicates the probability that the prediction box contains the target. For feature map The corresponding raw scores are represented by the Sigmoid function, which is a binary probabilistic activation function. The confidence score is combined with the class probability output from the classification branch to filter out the final detection results.

[0158] (4) Multi-scale fusion

[0159] Multi-scale fusion merges predicted bounding boxes at different scales, removes redundancy through non-maximum suppression (NMS), and outputs the final detection result. This step mainly includes two core processes: first, merging and initially screening predicted bounding boxes at different scales; and second, eliminating redundant predicted bounding boxes through non-maximum suppression technology, ultimately outputting high-quality detection results.

[0160] The predicted bounding boxes output by detection heads at different scales are collected. For feature maps at different scales, each grid cell contains bounding box parameters. Category probability and confidence score The prediction, where the class probability is... and confidence score The final score can be obtained by merging. :

[0161] (15)

[0162] in, Indicates the location Predicted as category The final score, For class probabilities, This represents the confidence score.

[0163] After obtaining a set of candidate bounding boxes, for the problem that the same target may be detected repeatedly by multiple predicted boxes, non-maximum suppression (NMS) is used to remove redundancy by iteratively selecting the predicted box with the highest score.

[0164] The NMS process can be described as follows: All candidate boxes are ranked according to their final scores. Sort the boxes from highest to lowest score, select the highest-scoring predicted box as one of the current retained detection results, and remove it from the candidate box list.

[0165] Specifically, the highest-scoring prediction box is first calculated. With all remaining candidate boxes The Intersection over Union (IoU) is a commonly used metric to measure the degree of overlap between two bounding boxes. It is defined as the ratio of the area of ​​their intersection to the area of ​​their union.

[0166] (16)

[0167] Where IoU represents the intersection-union ratio, Indicates the highest score prediction box. This represents all remaining candidate boxes. It is the intersection area of ​​the two bounding boxes. It is the area of ​​the union of the two bounding boxes.

[0168] If candidate box IoU and If the IoU exceeds the preset threshold, it is considered... The detection is related to Same goal, and because Their scores were higher, therefore Those deemed redundant will be removed.

[0169] Repeat the above process: select the highest-scoring predicted box from the remaining candidate boxes, add it to the final detection result list, remove it, and use it to suppress other overlapping predicted boxes in the list, until all candidate boxes have been processed.

[0170] The final output set of bounding boxes has been fully integrated and redundant removed, and can accurately reflect the target location and category in the image. It is the most accurate and representative target bounding box, which serves as the final detection result.

[0171] Step 3: Based on the vehicle information in the DroneVehicle dataset, train and optimize the detection model, recognize the image, and output information, including the position, size, category, and confidence information of the vehicle bounding box.

[0172] Step 3.1: Data preparation and preprocessing. Adjust the number and names of vehicle categories in the DroneVehicle dataset, convert the original annotation format, and record the category ID, normalized coordinates of the center point, and normalized width and height of the vehicle target to ensure that the data is compatible with the input requirements of the Hyper-YOLOv1.1 framework.

[0173] The DroneVehicle dataset is a large-scale, high-quality RGB-IR cross-modal vehicle detection dataset for drones. It contains 28,439 pairs of high-resolution images, covering diverse traffic scenes such as urban roads and parking lots. The dataset provides directional bounding box annotations for 953,087 vehicle instances, accurately describing vehicle orientation, and is subdivided into seven vehicle categories. Some data also includes occlusion and illumination-assisted labels.

[0174] Applying the DroneVehicle dataset to the Hyper-YOLOv1.1 framework involves data preparation and preprocessing, adjusting the number of classes and their names, and converting the annotation format to one compatible with Hyper-YOLOv1.1. This includes recording the normalized coordinates of the target center points in the image. And the normalized width of the target. and height This can be determined by the coordinates of the target's center point, its width and height, and the width of the image. and height To obtain, that is:

[0175] (17)

[0176] in, Image width, Image height, The actual coordinates of the target center point. For normalized coordinates, To normalize the width and height, These are the width and height of the vehicle target, respectively.

[0177] Step 3.2, Model Training and Optimization: GELAN-C-Hyper is selected as the backbone network, and hyperparameter configuration is loaded. The total loss function is composed of bounding box regression loss, classification loss, and target existence loss. A vehicle detection model adapted to the UAV perspective is trained.

[0178] During the model training and optimization phases, GELAN-C-Hyper was selected as the backbone network for Hyper-YOLOv1.1. The training strategy included loading hyperparameter configurations such as learning rate, optimizer, and data augmentation. This configuration was optimized for training from scratch and is suitable for low-resource environments. The initial learning rate was 0.005 to 0.02, employing a cosine decay strategy, ultimately reducing the learning rate to 5% to 15% of the initial value. With training rounds The change takes the form of:

[0179] (18)

[0180] in, Number of training rounds The learning rate Minimum learning rate, maximum learning rate This represents the total number of training rounds. In addition, a learning rate warm-up phase is included to ensure the stability of the training process.

[0181] The optimizer uses SGD with a momentum factor of 0.8 to 0.95 and a weight decay of 0.00001 to 0.001 to accelerate convergence and prevent overfitting. For data augmentation, simulated lighting is achieved by adjusting HSV hue, saturation, and brightness. Random rotation angles are limited to ±2° to ±15°, translation ratios are 5% to 20% of the image size, scaling is set to 0.5 to 1.5 times, and Mosaic enhancement is enabled with a probability of 30% to 80%.

[0182] The loss function is set by the bounding box regression loss. Classification loss and target existence loss The composition, and the total loss, are expressed as:

[0183] (19)

[0184] in, Total loss; For bounding box regression loss; For classifying losses, The target existence loss indicates whether a grid cell contains the target center; These are the weight coefficients for bounding box regression loss, classification loss, and target existence loss, respectively.

[0185] In the Hyper-YOLOv1.1 framework, complete intersection and joint loss are used. As the bounding box regression loss, it is expressed as:

[0186] (20)

[0187] in, For bounding box regression loss, Indicates cascading loss. The intersection-union ratio (IUU) of the predicted bounding box and the ground truth bounding box. and These are the center points of the predicted bounding box and the ground truth bounding box, respectively. For Euclidean distance, The length of the diagonal of the smallest closed frame that encloses the two boxes. This is the proportional control coefficient. To ensure consistent aspect ratio.

[0188] Classification loss Using cross-entropy loss, it can be expressed as:

[0189] (twenty one)

[0190] in, The classification loss is (i.e., the cross-entropy loss in formula (21)). The target predicted by the model belongs to the category The probability of; For the true category in the category The probability of the above is expressed using one-hot encoding, meaning there is only one such encoding. One is 1, the rest are 0.

[0191] In terms of loss function weight allocation, the proportions of bounding box regression, classification, and object presence loss should prioritize the stability of object localization, with object presence loss having the highest proportion, followed by bounding box classification loss, and finally bounding box regression loss. The training strategy includes 3 to 5 rounds of learning rate warm-up, with initial momentum set to 0.7 to 0.9.

[0192] Step 3.3, Model Deployment and Detection: Deploy the trained and optimized model weights to the actual scene, receive images collected in real time by the UAV, and output the detected vehicle bounding boxes, category information and confidence scores to achieve high-precision vehicle detection.

[0193] After training, the optimized model weights are obtained and used to deploy the model to real-world application scenarios. The model receives RGB images collected in real time by the drone and outputs the bounding boxes, category information, and confidence scores of the detected vehicles.

[0194] Step 4: Based on the parking space corner information of the PSD dataset, train and optimize the detection model, recognize the image, and output information, including the position, size, category, and confidence information of the parking space corner bounding box.

[0195] Step 4.1: Data Preparation and Preprocessing. Adjust the parking space label attributes in the PSD dataset, add type and valid status labels, and convert the parking space corner coordinates to the bounding box format adapted by Hyper-YOLOv1.1. Record the parking space corner parameters, remove blurry samples, and ensure that the data meets the model input requirements.

[0196] The Parking-slot-dataset (PSD) is a high-quality resource for on-street parking space detection, containing 12,165 high-resolution images of 1920×1080 resolution, covering diverse scenes including indoor and outdoor spaces, underground parking lots, and urban roads. The data encompasses various lighting and weather conditions and includes interference such as glare and water stains. The dataset annotates key corner points of over 560,000 parking space instances and provides parking space visibility and occupancy status information.

[0197] The PSD dataset was applied to the Hyper-YOLOv1.1 framework. During data preprocessing, the dataset was classified into training and validation sets, with a fixed random seed set to ensure reproducibility of the partitioning results. The annotation format was converted to a format compatible with the object detection framework, recording the category identifiers of parking space targets in the image and the normalized coordinates of the parking space corner points. The normalization formula is:

[0198] (twenty two)

[0199] in, These are the actual coordinates of the corner point of the parking space. These are the image width and image height, respectively. Normalized coordinates for the corner points of the parking spaces.

[0200] The generated rectangle is normalized to its center point. Expanding to the left and right from the center Expand upwards and downwards After testing, the normalized width was finally set. The value ranges from 0.085 to 0.095, with an altitude of [missing information]. The value ranges from 0.02 to 0.04.

[0201] Step 4.2, Model Training and Optimization: Using GELAN-M as the backbone network, a hierarchical strategy is employed to train the model on the parking space corner data in the PSD dataset. The training strategy involves adjusting the loss weights and data preprocessing, increasing the number of training iterations, and loading hyperparameter configurations specifically for parking space corner detection.

[0202] During the model training and optimization phases, GELAN-M was used as the backbone network of the Hyper-Yolov 1.1 framework. A hierarchical strategy was employed, and the model was trained based on parking space corner data from the PSD dataset. The training strategy, which included loading hyperparameter configurations such as learning rate, optimizer, and data augmentation, was largely consistent with the training steps for vehicle detection, but differences existed in the details of model optimization.

[0203] To address the characteristics of the PSD dataset, the loss function weights were first adjusted by increasing the weight ratio between the parking space corner regression loss and the target existence loss, thereby emphasizing the accuracy of spatial localization. Secondly, in the data preprocessing stage, sharpening and contrast enhancement were added, and the probability of random noise injection was increased to improve the model's robustness under complex lighting and occlusion conditions. Finally, to fully learn subtle features such as parking space corners, the number of training iterations was increased compared to vehicle detection.

[0204] Step 4.3, Model Deployment and Detection: Deploy the trained model weights to the detection system. By receiving the images collected by the UAV in Step 1, output the size, position, and confidence of the parking space corner bounding boxes, providing accurate positioning data for the subsequent construction of parking space bounding boxes.

[0205] Step 5: Construct the parking space bounding box based on the corner points of the parking space, and determine the violation based on the dual threshold strategy. The dual thresholds are the intersection-union ratio of the vehicle bounding box and the parking space bounding box and the longitudinal normalized offset of the vehicle center point.

[0206] Step 5.1: Extraction of key points and line fitting of parking spaces. The center points of the corner bounding boxes of parking spaces are extracted as key points and set into a point set. The least squares method is used to fit the line. By solving the normal equation and removing outliers, the final line parameters of the outer boundary of the parking space are obtained.

[0207] The obtained vehicle detection bounding box and parking space corner detection bounding box need to be converted into a parking space bounding box first.

[0208] See Figure 4 First, the center points of these parking space corner detection boxes are extracted as key points. The formula for calculating the normalized coordinates of the center points of the bounding boxes is:

[0209] (twenty three)

[0210] (twenty four)

[0211] in, The coordinates of the center point of the parking space corner detection frame. and These are the pixel coordinates of the two ends of the bounding box of the parking space corner. , These are the image width and image height, respectively.

[0212] The detected normalized center points are set into a point set. ,in It is the number of key points detected. It is the index of the key point in the image.

[0213] Secondly, the least squares method is used to fit the key points to determine a straight line representing the parking space direction. The objective is to minimize the actual observed values. Compared with model predictions The sum of squared residuals between them, and the minimization of residuals, are expressed as:

[0214] (25)

[0215] in, The slope of the fitted line, The intercept of the fitted line, The coordinates are the normalized coordinates of the center point.

[0216] To facilitate the solution, the point set Transform into matrix form and construct an augmented matrix. and observation vector Represented as:

[0217] (26)

[0218] in, For augmented matrices, This is the observation vector.

[0219] By solving the normal equation The optimal solution is expressed as:

[0220] (27)

[0221] in, , This is the optimal solution for the linear parameters corresponding to minimizing the residuals. The optimal solution is obtained by finding this optimal solution. The equation of the fitted line can then be obtained.

[0222] To improve the accuracy and robustness of the fitting, outlier removal can be introduced. For the straight line obtained from the initial fitting... Calculate the perpendicular distance from each point to the line using the following formula:

[0223] (28)

[0224] in, For point set The point in the middle, For point The perpendicular distance to the fitted line. The slope of the fitted line, This is the intercept of the fitted line.

[0225] Then, the median and standard deviation of all residuals are calculated. Points with residuals greater than a dynamic threshold are removed. Using the set of corner points remaining after outlier removal, least squares fitting is re-performed to form the filtered point set. ,in .in This indicates the number of corner points in the filtered point set. For point set The points in the middle. Repeat the above least squares fitting and outlier removal process to obtain the final, more accurate line parameters. and .

[0226] Step 5.2: Construct the rectangular bounding box of the parking space. Based on the effective intersection points of the fitted straight line and the image boundary, and combined with the preset parking space width, draw perpendicular lines to the corner points to determine the precise position and size of the rectangular bounding box of the parking space.

[0227] The specific boundaries of the parking space are determined by calculating the intersection points of the fitted straight line and the image boundary, taking into account the image width. and image height Calculate the intersection points of the line with the left, right, top, and bottom boundaries. Select two points from these intersection points that fall within the image area as the endpoints of the parking space's outer boundary.

[0228] Projecting the corner points of the parking space onto the fitted straight line, for the very few corner points that are not detected or lost during processing, corrections can be made by combining the preset longitudinal length of the parking space. Then, combining the preset lateral width of the parking space, a perpendicular line is drawn from each projection point to the parking side, thereby obtaining the accurate position and size of the rectangular bounding box of the parking space.

[0229] Step 5.3, see below Figure 5 Dual threshold calculation and violation determination: Calculate the intersection-union ratio (IUU) of the vehicle bounding box and the parking space bounding box, and the longitudinal normalized offset of the vehicle center point; when the IUUU value is not lower than the threshold... Furthermore, the longitudinal normalized offset of the vehicle center point does not exceed the threshold. If the time frame is met, it is deemed compliant; otherwise, it is deemed a violation. The threshold representing the intersection-union ratio (IoU) between the vehicle bounding box and the parking space bounding box. The threshold representing the longitudinal normalized offset of the vehicle's center point.

[0230] This strategy avoids misjudgment through single-frame spatial relationship analysis and does not rely on time-series tracking data.

[0231] To ensure that the main area where the vehicle is parked is within a legal parking space, the detected vehicle bounding box is calculated. With the bounding boxes of all identified valid parking spaces in the image Crossover ratio between value, Defined as the ratio of the intersection area to the union area of ​​the vehicle bounding box and the parking space bounding box:

[0232] (29)

[0233] in, For vehicle bounding box, For parking space boundary boxes, This represents the intersection area of ​​the vehicle bounding box and the parking space bounding box. The area is the union of the vehicle bounding box and the parking space bounding box. Value threshold is The constraints are:

[0234] (30)

[0235] in, This is the ratio of the intersection area to the union area of ​​the vehicle bounding box and the parking space bounding box. This is the threshold for the intersection-union ratio (IoU) of the vehicle bounding box and the parking space bounding box. The range is .

[0236] To further refine the assessment, even if the vehicle's IoU value is high, it is still necessary to evaluate whether the vehicle's center position is properly aligned with the parking space center. This can be determined by calculating the longitudinal normalized offset of the vehicle's center point relative to the parking space center point.

[0237] First, from the center of the vehicle boundary frame and the center of the parking space boundary frame Calculate the longitudinal normalized offset of the vehicle's center point, which is the offset in the direction of the outer edge of the parking space:

[0238] (31)

[0239] in, This represents the longitudinal normalized offset of the vehicle's center point. The normalized ordinate of the vehicle bounding box center point. These are the normalized coordinates of the center point of the parking space bounding box. The threshold for the longitudinal normalized offset of the vehicle center point is... The constraints are:

[0240] (32)

[0241] in, The threshold representing the longitudinal normalized offset of the vehicle center point. The range is This ensures that even if the overall overlap of the vehicles is high, their center point will not deviate significantly longitudinally, meaning that the vehicles' forward and backward movement within the parking space does not exceed the permissible range.

[0242] A vehicle is considered properly parked only if it meets both conditions (30) and (32). That is, the vehicle's IoU value with the parking space boundary must be greater than or equal to... And its normalized vertical offset is less than or equal to Otherwise, it will be judged as illegal parking. Specific Implementation Method Two

[0244] Based on the same inventive concept, the present invention also provides an on-street parking violation detection system based on UAV perspective, used to implement the above-mentioned on-street parking violation detection method based on UAV perspective. The system includes a preprocessing module, a target detection framework module, a vehicle and parking space corner detection module, and a violation judgment module.

[0245] Example

[0246] (I) Basic information of the embodiments and three evaluation schemes

[0247] The on-street parking lanes along Huaihe Road, Huashan Road, and Haihe Road in Nangang District, Harbin City, were selected as a case study. A schematic diagram of the parking lane locations is shown below. Figure 6 As shown in Table 1, to verify the effectiveness of this invention in improving the accuracy and robustness of on-street parking violation detection, test input schemes for two time periods, off-peak (daytime) and peak (nighttime / complex lighting), were designed.

[0248] Table 1 Test Input Scheme

[0249]

[0250] Note: The training and prior data sources are the DroneVehicle and PSD datasets, with 90% used as the training set and 10% as the validation set. The initial learning rate was 0.01, employing a cosine decay strategy, eventually reducing the learning rate to 10% of the initial value. The optimizer selected was SGD with a momentum factor of 0.9 and a weight decay of 0.0005 to accelerate convergence and prevent overfitting. For data augmentation, simulated lighting was achieved by adjusting HSV hue, saturation, and brightness, with random rotation angles limited to [specific values ​​to be filled in]. The translation ratio was 10% of the image size, the scaling range was set to 0.5~1.5 times, and Mosaic enhancement was enabled with a 50% probability. The gains for bounding box regression, classification, and object presence loss were 0.05, 0.25, and 0.7, respectively.

[0251] Set normalized width ,high The evaluation criteria and regions are uniformly ignored. In the table, "Annotation Item (Quantity) = 2" refers to the vehicle boundary box and parking space key points, respectively.

[0252] (1) Traditional fixed monitoring / background modeling solution

[0253] Scheme 1 uses background subtraction combined with morphological post-processing and connected component analysis to detect stationary targets, and determines parking space occupancy based on region overlap. The background update rate and threshold are set as fixed parameters, and it lacks the ability to fit and correct for aerial viewpoint geometric constraints. The algorithm parameters remain fixed and consistent with the aforementioned method description.

[0254] (2) Single threshold matching scheme

[0255] Scheme 2 uses a vehicle detection model to locate vehicles. Parking spaces are given as a rectangle prior, and only the IoU threshold is used for occupancy determination, without introducing the constraint of the vehicle center longitudinal offset Δy. The confidence threshold and NMS strategy are the same as those in Scheme 3 during the detection phase to ensure fair comparison.

[0256] (3) The dual threshold discrimination scheme proposed in this paper

[0257] Scheme 3 employs Hyper-YOLOv1.1 to implement vehicle detection and parking space corner detection respectively. Robust fitting is performed on the parking space corners in each frame to generate structured parking space boundary rectangles. Based on the spatial relationship between vehicles and parking spaces, a dual threshold is used for violation judgment: IoU ≥ 0.90, and the longitudinal offset of the vehicle relative to the center of the parking space is [value missing]. (Normalized by the width of the parking space frame) ≤0.10.

[0258] Schemes 1 and 2 maintain the same resolution, frame rate, ignored region, confidence threshold, NMS, and IoU as Scheme 3 to ensure comparisons are conducted under identical conditions. Furthermore, the end-to-end maximum processing latency target is set to ≤55 ms / frame. The upper limit of the threshold is set to 0.10.

[0259] All three schemes were tested with the inputs set as shown in Table 1. Schemes 1 and 2 performed detection and occupancy determination directly on the same frame set; Scheme 3 performed keypoint fitting, parking space boundary generation, and dual threshold discrimination on the same frame set. Evaluations were conducted under uniform input conditions in both off-peak and peak periods. The evaluation warm-up consisted of 200 frames, and the effective evaluation frame count was 1800 frames.

[0260] (II) Evaluation Indicators

[0261] To verify the effectiveness of this method, a unified original video source was used to compare the proposed dual-threshold discrimination detection scheme with two control schemes. All three schemes shared the same data and evaluation criteria. Evaluation metrics included overall performance (mAP@0.5), precision, recall, the harmonic mean of precision and recall (F1), false positive rate, false negative rate, average localization error, center longitudinal offset compliance rate, and average processing latency.

[0262] Evaluations were conducted on two scenarios, and the statistical results of the evaluation indicators were continuously recorded and finally output. Each scenario was repeated 10 times. The number of labeled items in the evaluation set was fixed at 2 (vehicle bounding box, parking space corner point, and occupancy / violation as derived labels). The average value of each indicator of the three methods was calculated.

[0263] (III) Evaluation Results

[0264] The evaluation results are shown in Table 2.

[0265] Table 2 Comparison of Evaluation Indicators for the Detection Effects of the Three Schemes

[0266]

[0267] The F1 accuracy of the proposed method (Scheme 3) is 95.6% during the day and 92.6% at night, representing improvements of 22.08% and 42.46% compared to Scheme 1, and 8.51% and 13.34% compared to Scheme 2, respectively. Regarding false alarm rate, the proposed method reduces the false alarm rate by 56.96% and 59.52% during the day and night compared to Scheme 1, and by 35.85% and 34.62% compared to Scheme 2, respectively. Regarding false negative rate, the proposed method reduces the false negative rate by 80.48% and 79.90% during the day and night compared to Scheme 1, and by 64.75% and 62.09% compared to Scheme 2, respectively. In terms of geometric positioning, the average positioning error of the proposed method reduces by 67.35% and 68.09% during the day and night compared to Scheme 1, and by 49.21% and 48.28% compared to Scheme 2, respectively.

[0268] In summary, this method effectively suppresses misjudgments of "high IoU but excessive longitudinal offset" by using structured parking space boundaries and dual threshold discrimination, achieving a center longitudinal offset compliance rate of 98.1% (daytime) and 96.4% (nighttime). It also significantly improves detection accuracy and stability while maintaining near real-time processing (approximately 52–55 ms / frame).

[0269] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for detecting illegal on-street parking based on the perspective of a UAV, characterized in that it is Follow these steps: Step 1: Acquire images and transmit data using the drone, and perform data preprocessing to obtain image data; Step 2: Construct the Hyper-Yolov 1.1 "backbone network - neck network - detection head" target detection framework; Step 3: Based on the vehicle information in the DroneVehicle dataset, train and optimize the detection model, recognize the image, and output information, including the position, size, category, and confidence information of the vehicle bounding box. Step 4: Based on the parking space corner information of the PSD dataset, train and optimize the detection model, recognize the image, and output information, including the position, size, category, and confidence information of the parking space corner bounding box. Step 5: Construct the parking space bounding box based on the corner points of the parking space, and determine the violation based on the dual threshold strategy. The dual thresholds are the intersection-union ratio of the vehicle bounding box and the parking space bounding box and the longitudinal normalized offset of the vehicle center point.

2. The drone-view-based on-street parking violation detection method of claim 1, wherein, Step 1 includes the following steps: Step 1.1: The UAV flies automatically along the planned route. The imaging equipment on the UAV adjusts the exposure parameters and continuously captures clear original images. Then, the original image data is sent to the ground receiving equipment through the data transmission module. Step 1.2: Perform data preprocessing on the received raw image data. Data preprocessing includes image enhancement, data enhancement, and format standardization.

3. The method for detecting on-street parking violations based on an unmanned aerial vehicle (UAV) perspective according to claim 1, characterized in that, Step 2 includes the following steps: Step 2.1: Construct a hybrid aggregation network as the backbone network to extract features. Adjust the number of input feature channels through 1×1 convolution to obtain intermediate features. After multi-path differentiation processing, the features are spliced ​​and aggregated to extract target enhancement features that can be adapted to key corner points of vehicles and parking spaces. Step 2.2: Construct a hypergraph-based cross-layer and cross-location representation network as the neck network fusion feature, process the backbone network output features in stages, construct a hypergraph and complete convolution, and fuse to generate a multi-scale feature map for use by the detection head, so as to achieve efficient fusion and optimization of target features; Step 2.3: Configure the decoupled detection head to receive the multi-scale feature map output by the neck network. Output the category probability and position size of the vehicle and parking space corner points through independent branches. Combine confidence prediction and non-maximum suppression to obtain accurate detection results of the vehicle and parking space corner points.

4. The method for detecting on-street parking violations based on an unmanned aerial vehicle (UAV) perspective according to claim 3, characterized in that, In step 2.2, the construction of the hypergraph and the completion of convolution specifically involve: Super Edge Aggregated feature vectors With vertex Update features Represented as: (4) in, For the super edge Aggregated feature vectors, Represents vertices Does it belong to the super-edge? , For set Number of elements in the middle As vertices The original characteristics; This represents the trainable weight parameter matrix, with dimension 1. Used for vertex features Perform a linear transformation; Represents vertices Update features, Indicates super edge Does it contain vertices? , For set The number of elements in the middle represents the vertex. The number of associated superedges; The hypergraph convolution representation of the two-stage hypergraph message passing matrix is: (5) in, For hypergraph convolution operations, As vertices Features included Represents the vertex-hyperedge incidence matrix. Represents the vertex degree matrix. Represents the hypermarginality matrix. Combinatorial representation normalizes feature information.

5. The method for detecting on-street parking violations based on an unmanned aerial vehicle (UAV) perspective according to claim 1, characterized in that, Step 3 includes the following steps: Step 3.1: Data preparation and preprocessing. Adjust the number and names of vehicle categories in the DroneVehicle dataset, convert the original annotation format, and record the category ID, normalized coordinates of the center point, and normalized width and height of the vehicle target to ensure that the data is compatible with the input requirements of the Hyper-YOLOv1.1 framework. Step 3.2, Model Training and Optimization: GELAN-C-Hyper is selected as the backbone network, and hyperparameter configuration is loaded. The total loss function is composed of bounding box regression loss, classification loss, and target existence loss. A vehicle detection model adapted to the UAV perspective is trained. Step 3.3, Model Deployment and Detection: Deploy the trained and optimized model weights to the actual scene, receive images collected in real time by the UAV, and output the detected vehicle bounding boxes, category information and confidence scores to achieve high-precision vehicle detection.

6. The method for detecting on-street parking violations based on the perspective of an unmanned aerial vehicle (UAV) as described in claim 1, characterized in that, Step 4 includes the following steps: Step 4.1: Data preparation and preprocessing. Adjust the parking space label attributes of the PSD dataset, add type and valid status labels, and convert the parking space corner coordinates to the bounding box format adapted by Hyper-YOLOv1.

1. Record the parking space corner parameters, remove fuzzy samples, and ensure that the data meets the model input requirements. Step 4.2, Model Training and Optimization: Using GELAN-M as the backbone network, a hierarchical strategy is adopted to train the model on the parking space corner data in the PSD dataset. In terms of training strategy, the loss weights and data preprocessing process are adjusted, the number of training iterations is increased, and hyperparameter configurations for parking space corner detection are loaded. Step 4.3, Model Deployment and Detection: Deploy the trained model weights to the detection system. By receiving the images collected by the UAV in Step 1, output the size, position, and confidence of the parking space corner bounding boxes, providing accurate positioning data for the subsequent construction of parking space bounding boxes.

7. The method for detecting on-street parking violations based on an unmanned aerial vehicle (UAV) perspective according to claim 1, characterized in that, Step 5 includes the following steps: Step 5.1: Parking space key point extraction and line fitting. Extract the center point of the corner bounding box of the parking space as key points and set them into a point set. Use the least squares method to fit the line. By solving the normal equation, obtain the final line parameters of the outer boundary of the parking space. Step 5.2: Constructing the rectangular bounding box of the parking space. Based on the effective intersection points of the fitted straight line and the image boundary, and combined with the preset parking space width, draw perpendicular lines to the corner points to determine the precise position and size of the rectangular bounding box of the parking space. Step 5.3, Dual Threshold Calculation and Violation Judgment: Calculate the intersection-union ratio (IUR) of the vehicle bounding box and the parking space bounding box, and the longitudinal normalized offset of the vehicle center point; when the IUR is not lower than the threshold value... Furthermore, the longitudinal normalized offset of the vehicle center point does not exceed the threshold. If the time frame is met, it is deemed compliant; otherwise, it is deemed a violation. This represents the threshold value for the intersection-union ratio (IoU) between the vehicle bounding box and the parking space bounding box. This represents the threshold for the longitudinal normalized offset of the vehicle's center point.

8. The method for detecting illegal parking on the road based on the perspective of an unmanned aerial vehicle (UAV) as described in claim 7, characterized in that, In step 5.1, the extraction of key parking space points and line fitting specifically involves: First, the center point of the parking space corner detection box is extracted as the key point; the formula for calculating the normalized coordinates of the center point of the bounding box is: (23) (24) in, The coordinates of the center point of the parking space corner detection frame. and These are the pixel coordinates of the two ends of the bounding box of the parking space corner. , These are the image width and image height, respectively. The detected normalized center points are set into a point set. ,in It is the number of key points detected. It is the index of the key point in the image; Secondly, the least squares method is used to fit the key points to determine a straight line representing the direction of the parking space; the objective is to minimize the actual observed values. Compared with model predictions The sum of squared residuals between them, and the minimization of residuals, are expressed as: (25) in, The slope of the fitted line, The intercept of the fitted line, Normalized coordinates for the center point; To facilitate the solution, the point set Transform into matrix form and construct an augmented matrix. and observation vector Represented as: (26) in, For augmented matrices, For observation vectors; By solving the normal equation The optimal solution is expressed as: (27) in, , This is the optimal solution for the linear parameters corresponding to minimizing the residuals. The optimal solution is obtained by finding this optimal solution. The equation of the fitted straight line is obtained.

9. The method for detecting illegal parking on the road based on the perspective of an unmanned aerial vehicle (UAV) as described in claim 7, characterized in that, In step 5.3, the dual threshold calculation and violation determination specifically involve: The intersection-union ratio (IU / UU) formula and constraints for the vehicle bounding box and the parking space bounding box are as follows: (29) in, For vehicle bounding box, For parking space boundary boxes, This represents the intersection area of ​​the vehicle bounding box and the parking space bounding box. The area is the union of the vehicle bounding box and the parking space bounding box. (30) in, This is the ratio of the intersection area to the union area of ​​the vehicle bounding box and the parking space bounding box. This is the threshold for the intersection-union ratio (IoU) of the vehicle bounding box and the parking space bounding box. The range is ; The formula and constraints for the longitudinal normalized offset of the vehicle center point are as follows: (31) in, This represents the longitudinal normalized offset of the vehicle's center point. The normalized ordinate of the vehicle bounding box center point. Normalized coordinates of the center point of the parking space boundary box; (32) in, The threshold representing the longitudinal normalized offset of the vehicle center point. The range is .

10. A roadside parking violation detection system based on the perspective of an unmanned aerial vehicle (UAV), characterized in that, The system is used to implement the on-street parking violation detection method based on the perspective of an unmanned aerial vehicle (UAV) as described in any one of claims 1 to 9. The system includes a preprocessing module, a target detection framework module, a vehicle and parking space corner detection module, and a violation discrimination module.

Images