Intelligent crowd gathering detection system and method under view angle of unmanned aerial vehicle

Abstract

The invention discloses an intelligent crowd gathering detection method under the view angle of an unmanned aerial vehicle, and the method comprises the following steps: image collection: carrying out the patrolling shooting of a target region through the unmanned aerial vehicle, and obtaining an aerial image; pedestrian detection: preprocessing the acquired image, and inputting the preprocessed image into a special YOLOv8-AFPN detection model for pedestrian detection; wherein the special YOLOv8-AFPN detection model is constructed by introducing an AFPN into the YOLOv8 and carrying out multi-stage training on the AFPN and the YOLOv8; the AFPN is used for replacing a PAN-FPN structure adopted by the YOLOv8 in a Neck part; and aggregation analysis: introducing a density-based aggregation judgment algorithm on the basis of completing pedestrian detection by the YOLOv8-AFPN detection model, and identifying an aggregation behavior by analyzing detected pedestrian spatial distribution. The invention further discloses an intelligent crowd gathering detection system under the view angle of the unmanned aerial vehicle. According to the invention, high maneuverability and wide-angle visual field of the unmanned aerial vehicle are utilized to carry out patrol flight on the urban open area, and a special deep learning model is used to realize real-time and accurate identification and early warning of the personnel gathering phenomenon.

Description

Technical Field

[0001] This invention relates to the field of public safety technology, and in particular to an intelligent system and method for detecting crowd gatherings from the perspective of an unmanned aerial vehicle (UAV). Background Technology

[0002] Effective monitoring of crowds gathering in public places is crucial for maintaining public safety, preventing stampedes, and implementing emergency management. However, existing monitoring methods have significant technical shortcomings when facing this task: (1) Large blind spots in fixed monitoring: The field of view of the deployed fixed ground cameras is blocked by buildings, green belts, etc., forming a large number of blind spots. Once people leave the preset monitoring range or enter the visual blind spot, it will cause a monitoring gap; (2) Limitations of manual detection: Manually monitoring videos or patrolling the site is not only inefficient, but also highly dependent on personal experience in determining "gathering," lacking unified and quantifiable standards. Under fatigue, it is easy to miss or falsely report, and cannot provide objective and reliable early warning information; (3) Delayed response: From the discovery of the anomaly and the reporting at each level to the dispatch of personnel to handle the situation on-site, the entire process takes a long time. This lag means that the management often only intervenes after the gathering incident has escalated or caused consequences, missing the best opportunity to manage the situation. (4) Lack of a global perspective: Ordinary monitoring methods cannot quickly obtain key information such as "which area has the most people" and "where are people moving to", making it difficult to grasp the population distribution density and aggregation trend in different locations within the area.

[0003] Although drone technology has become widespread, it is currently mostly used in fields such as aerial photography and surveying. There is a lack of end-to-end solutions specifically for detecting crowd gatherings in complex urban scenarios, especially in terms of real-time analysis, automatic navigation, and accurate identification.

[0004] Based on this, the present invention proposes an intelligent detection system and method for crowd gathering from the perspective of unmanned aerial vehicles (UAVs) to solve the problems mentioned above. Summary of the Invention

[0005] The purpose of this invention is to provide an intelligent detection system and method for crowd gathering from the perspective of unmanned aerial vehicles (UAVs). By utilizing the high mobility and wide field of view of UAVs, the system can patrol open urban areas and, with the help of a dedicated deep learning model, achieve real-time and accurate identification and early warning of crowd gathering phenomena.

[0006] To address the aforementioned technical problems, this invention provides an intelligent method for detecting crowd gatherings from the perspective of an unmanned aerial vehicle (UAV), comprising the following steps: Image acquisition: Aerial images containing pedestrians are obtained by using drones to conduct patrol flights over the target area; Pedestrian detection: The acquired images are preprocessed and then input into a dedicated YOLOv8-AFPN detection model for pedestrian detection; The dedicated YOLOv8-AFPN detection model is constructed by introducing AFPN into YOLOv8 and training it through multiple stages; the AFPN is used to replace the PAN-FPN structure used in the Neck part of YOLOv8. Clustering Analysis: Based on the pedestrian detection completed by the YOLOv8-AFPN detection model, a density-based clustering judgment algorithm is introduced to identify clustering behavior by analyzing the spatial distribution of detected pedestrians.

[0007] Furthermore, the feature fusion of the AFPN includes the following steps: In Backbone's bottom-up feature extraction process, two adjacent shallow features are fused, and higher-level deep features are gradually incorporated into the fusion process. In the final stage, the top-level features are added to the feature fusion process, and the fused features are further learned using residual units.

[0008] Furthermore, the AFPN integrates an ASFF, which is used to assign appropriate spatial weights to features at different levels during the feature fusion process of the AFPN; the formula for the ASFF is: ; in, ; This represents the target layer, i.e., the layer at which the desired feature map is located; Representing features from different levels, n=1,2,3; Representative from the first Layer conversion to the first The feature vector of the layer at position (i,j); the transformation includes upsampling or downsampling and channel adjustment; , and These correspond to the spatial weights of the features in layers 1, 2, and 3 at position (i,j), respectively.

[0009] Furthermore, the multi-stage training includes the following steps: Phase 1: Random number seed 1, no pre-trained model used, training set: validation set = 4:1, strong data augmentation used; Second stage: random number seed 2, using the model from the first stage as the pre-trained model, training set:validation set = 4:1, using weak data augmentation and low learning rate; Phase 3: Random number seed 3, using the model from Phase 2 as the pre-trained model, training set: validation set = 1:3, data augmentation disabled, low learning rate.

[0010] Furthermore, the training and validation set data originate from the UAV-Person dataset, and the construction steps of the UAV-Person dataset include: The process involves using drones to capture images, then sampling the captured video frames to obtain the necessary data. The collected data is labeled to ensure that clear targets in each image are accurately marked; After the annotation was completed, the dataset UAV-Person was formed by combining it with publicly available drone datasets online.

[0011] Furthermore, the density-based clustering judgment algorithm employs a region density algorithm, the execution flow of which includes the following steps: The detected image is divided into an N×M uniform grid, with each grid cell being S×S pixels in size; Count the number of pedestrians in each grid cell and calculate the local density of each grid cell, where density = number of pedestrians in the grid cell / grid area; A density threshold is set. If the density of a certain grid exceeds the threshold, the grid is marked as a "potential clustering area". Adjacent "potential clustering areas" are merged to form a clustered region.

[0012] Furthermore, the density-based clustering decision algorithm employs the DBSCAN density clustering method, and the operation process of the DBSCAN density clustering method includes the following steps: Extract the center coordinates of the bounding boxes of all pedestrians from the YOLO detection results to construct a set of pedestrian location points; Set the neighborhood radius eps and the minimum number of samples required to form the core point; Perform DBSCAN clustering on the point set to obtain multiple clusters; Iterate through each cluster. If the number of points in the cluster is greater than or equal to the minimum number of samples, it is determined to be a cluster. Draw all the bounding boxes of the cluster on the image, and represent the clustered region with a convex hull or minimum bounding rectangle. Draw the bounding box of the clustered region and the density heatmap on the image.

[0013] Furthermore, the preprocessing steps include: The acquired images were adjusted to 640×640 and then normalized and augmented.

[0014] Furthermore, it also includes the following steps: Report generation: After the cluster analysis is completed, a risk assessment report is generated based on the analyzed data.

[0015] This invention also proposes an intelligent crowd gathering detection system from the perspective of unmanned aerial vehicles (UAVs) for running the aforementioned intelligent crowd gathering detection method from the perspective of unmanned aerial vehicles (UAVs), including an airborne detection module, a data transmission module, a cloud management platform, and a model optimization module; The airborne detection module is deployed on the UAV's airborne edge computing device, integrating an image acquisition unit, a preprocessing unit, a YOLOv8-AFPN detection unit, and a clustering analysis unit. The image acquisition unit is used to scan and capture aerial images of the target area. The preprocessing unit adjusts the acquired images to 640×640 pixels and performs normalization. The YOLOv8-AFPN detection unit loads a dedicated YOLOv8-AFPN detection model for pedestrian detection. The clustering analysis unit analyzes the spatial distribution of pedestrians using a region density algorithm or DBSCAN density clustering method to identify clustering behavior. The data transmission module establishes a two-way data interaction between the airborne detection module and the cloud management platform using a low-latency image transmission link. This is used to transmit the original aerial images, pedestrian detection results, cluster area markers, density heatmaps, and preliminary risk information output by the airborne detection module back to the cloud management platform in real time, as well as to receive drone patrol instructions and model update packages pushed by the model optimization module from the cloud management platform. The cloud management platform includes a data storage unit, a situation visualization unit, a report generation unit, and an alarm push unit. The data storage unit stores all image data, detection logs, and aggregation analysis results transmitted back by the data transmission module, and also stores the UAV-Person dataset. The situation visualization unit displays information such as aggregation areas, density heatmaps, and pedestrian distribution in an intuitive manner. The report generation unit automatically generates risk assessment reports based on aggregation analysis data. The alarm push unit pushes alarm information to management personnel terminals when medium- or high-risk aggregations are identified. The model optimization module is used to continuously train the dedicated YOLOv8-AFPN detection model based on cloud data, and to remotely iterate and upgrade the model via OTA.

[0016] Compared with the prior art, the present invention has at least the following beneficial effects: (1) The intelligent crowd gathering detection method under the perspective of UAV provided by the present invention solves the problem of blind spots in fixed monitoring by collecting images through UAV patrol flights. It can flexibly cover every corner of open areas such as squares and transportation hubs, making up for the lack of global perspective in the existing technology. At the same time, by replacing the PAN-FPN structure of YOLOv8 with AFPN to form a dedicated detection model and using the dedicated detection model for pedestrian detection, the feature loss problem of small targets and dense pedestrians under the perspective of UAV can be effectively alleviated, and the detection accuracy is greatly improved. Furthermore, the density-based clustering judgment algorithm is used to quantitatively judge the clustering behavior, replacing manual experience judgment, and avoiding false alarms and missed reports caused by subjectivity. (2) The intelligent detection system for crowd gathering from the perspective of UAV provided by the present invention integrates the entire process of “collection-preprocessing-detection-analysis” in its airborne detection module. Combined with the two-way data interaction of the low-latency image transmission link, it realizes “calculation while flying and transmission immediately after detection”, which solves the problem of delayed response in the existing technology and ensures the timeliness of detection and alarm. At the same time, the cloud management platform intuitively displays information such as crowd density heat map and gathering area marking through the situation visualization unit. Combined with the data storage unit, it realizes full data retention, which helps managers to quickly grasp the distribution and gathering trend of the entire area of ​​the crowd, provides a scientific basis for resource allocation, and avoids the limitations of the “fragmented” monitoring of traditional manual patrol. Attached Figure Description

[0017] Figure 1 This is a flowchart of the intelligent crowd gathering detection method from the perspective of an unmanned aerial vehicle (UAV) in Embodiment 1 of the present invention; Figure 2 This is a structural diagram of the Backbone in Embodiment 1 of the present invention; Figure 3 This is a flowchart of the AFPN processing in Embodiment 1 of the present invention; Figure 4 This is a flowchart of the ASFF processing in Embodiment 1 of the present invention; Figure 5 This is a structural block diagram of the intelligent crowd gathering detection system from the perspective of an unmanned aerial vehicle (UAV) in Embodiment 2 of the present invention. Detailed Implementation

[0018] The intelligent crowd gathering detection system and method from the perspective of an unmanned aerial vehicle (UAV) according to the present invention will be described in more detail below with reference to the schematic diagrams, which illustrate preferred embodiments of the present invention. It should be understood that those skilled in the art can modify the present invention described herein while still achieving the advantageous effects of the present invention. Therefore, the following description should be understood as being of general knowledge to those skilled in the art and is not intended to limit the present invention.

[0019] Furthermore, based on the teachings of this specification, those skilled in the art can form new technical solutions through cross-combination of different implementation methods without creating technical contradictions. Such variations should all be considered to fall within the protection scope of this patent.

[0020] The invention is described more specifically by way of example in the following paragraphs with reference to the accompanying drawings. The advantages and features of the invention will become clearer from the following description. It should be noted that the drawings are in a very simplified form and use non-precise proportions, and are only used to facilitate and clarify the illustration of the embodiments of the invention.

[0021] Example 1 like Figure 1 As shown in the figure, this invention proposes an intelligent method for detecting crowd gatherings from the perspective of an unmanned aerial vehicle (UAV), comprising the following steps: S1. Image Acquisition: Use drones to conduct patrol and photography of the target area to obtain aerial images containing pedestrians; In step S1, specifically, a DJI industrial-grade drone (such as the Mavic 3 Enterprise) is used for fixed-point cruise shooting at an altitude of 50-80 meters, with an image resolution of 4000×3000 pixels, providing sufficient detail for small target detection.

[0022] S2. Pedestrian Detection: The acquired images are preprocessed (image size adjusted to 640×640, normalized, and data augmented) and then input into a dedicated YOLOv8-AFPN detection model for pedestrian detection. The dedicated YOLOv8-AFPN detection model is constructed by introducing AFPN into YOLOv8 and training it through multiple stages. AFPN replaces the PAN-FPN structure used in the Neck section of YOLOv8. In step S2, specifically, AFPN is a progressive feature pyramid network that adopts the idea of ​​progressive feature fusion. Instead of fusing features from all levels at once, it works like a snowball, starting with adjacent low-level features and gradually incorporating deeper, more abstract features. That is, in the bottom-up feature extraction process of the backbone, two adjacent shallow features are fused first, and then higher-level, deeper features are progressively included in the fusion process. In the final stage, the top-level features are added to the feature fusion process, and residual units are used for further learning after fusion. This progressive fusion process avoids large semantic gaps between non-adjacent levels and reduces information loss during multi-layer transmission.

[0023] Figure 2 This is a structural diagram of the Backbone. Figure 2In the YOLOv8 model, the Backbone has 10 layers and is the main network. As the number of convolutional layers increases, feature maps of different receptive fields of each feature pyramid are extracted.

[0024] Convolutional layers (Conv) are used to extract feature maps using convolutional kernels. Different layers use different numbers of kernels. For example, layer 0 uses 64 3x3 kernels with a stride of 2, while layer 1 uses 128 3x3 kernels. The resulting feature maps will also be different in size.

[0025] C2f (Channel to Feature) is a module for feature fusion, designed to generate richer feature information.

[0026] SPPF (Spatial Pyramid Pooling Fast) is a spatial pyramid pooling technique that aims to perform pooling operations of different sizes on feature maps and then integrate the results to obtain a fixed-size output.

[0027] from Figure 2 As can be seen, YOLOv8 only constructs a feature pyramid structure on the three feature layers P3, P4, and P5. This is because P3, P4, and P5 have the greatest impact on the detection task. With limited computing resources, this allows YOLOv8 to achieve the optimal balance between speed and accuracy.

[0028] Figure 3 This is the processing flowchart of AFPN. It extracts the last layer features from each feature layer of the backbone, generating a set of features at different scales, denoted as {C2, C3, C4, C5}. The black arrows represent convolution, and the gray arrows represent Adaptive Spatial Feature Fusion (ASFF). First, the shallow features C2 and C3 are input into the feature pyramid network, then C4 is added, and finally C5 is added.

[0029] Feature fusion between C2 and C3 reduces their semantic gap. Since C3 and C4 are adjacent hierarchical features, it reduces the semantic gap between C2 and C4, and so on.

[0030] Because YOLOv8 only constructs feature pyramid structures at the three feature layers P3, P4, and P5, we assume these features at different scales are {C3, C4, C5}. The AFPN processing flow is as follows: The first stage involves fusing shallow features. Shallow features C2 and C3 are input into the AFPN. C2 is downsampled and then subjected to adaptive spatial fusion (ASFF) with C3. ASFF assigns spatial weights to the downsampled C2 and C3, and the sum is weighted, resulting in F1. C3 is upsampled and then subjected to ASFF with C2. ASFF assigns spatial weights to C2 and the upsampled C3, and the sum is weighted, resulting in F2. The convolutions of F1 and F2 produce the initial versions P2 and P3.

[0031] The second stage involves incorporating mid-layer features. C4 is input into the AFPN, followed by ASFF (Automatic ASFF) between P2, P3, and C4. First, C4 undergoes ASFF with the downsampled P2 and P3. ASFF then assigns spatial weights to the upsampled C4 and P2 and P3, performing a weighted sum, referred to as F3, which generates the initial version of P4 after convolution. Similarly, P3 undergoes ASFF with the upsampled C4 and downsampled P2, resulting in a new P3; P2 undergoes ASFF with the upsampled C4 and upsampled P3, generating a new P2. The third stage: Integrating top-level features, C5 is input into AFPN. C5 will be combined with downsampled P2, P3, and P4 to perform ASFF, and finally convolution to produce P5. This process is repeated to generate the final P2, P3, P4, and P5.

[0032] Since YOLOv8 only inputs {C3, C4, C5} into the feature pyramid network, it ultimately only generates the output {P3, P4, P5}.

[0033] Furthermore, the AFPN integrates ASFF. ASFF is used to assign appropriate spatial weights to features at different levels during the feature fusion process of the AFPN, in order to resolve "information conflicts" (e.g., different objects at the same location) that may occur during multi-level feature fusion.

[0034] Because low-level features (such as C2) are often high-resolution and rich in detail (edges, colors, textures), but have weak semantic meaning and contain a lot of noise, while high-level features (such as C5) are often low-resolution and have strong semantic meaning, but lose detail. During feature fusion, for the same spatial location (i, j), features from different levels may provide contradictory information. Therefore, ASFF assigns appropriate spatial weights to features at different levels. Taking the fusion of three levels of features as an example... Figure 3 As shown.

[0035] Specifically, the formula for ASFF is: ; in, ; This represents the target layer, i.e., the layer at which the desired feature map is located; Representing features from different levels, n=1,2,3; Representative from the first Layer conversion to the first The feature vector of the layer at position (i,j); the transformation includes upsampling or downsampling and channel adjustment; , and These correspond to the spatial weights of the features in layers 1, 2, and 3 at position (i,j), respectively.

[0036] In step S2, the multi-stage training includes the following steps: Phase 1: Random number seed 1, no pre-trained model used, training set: validation set = 4:1, strong data augmentation used; Second stage: random number seed 2, using the model from the first stage as the pre-trained model, training set:validation set = 4:1, using weak data augmentation and low learning rate; Phase 3: Random number seed 3, using the model from Phase 2 as the pre-trained model, training set: validation set = 1:3, data augmentation disabled, low learning rate.

[0037] In the above implementation process, the training set and validation set are divided from the dataset UAV-Person. That is, the training set and validation set data come from the dataset UAV-Person. Therefore, before performing multi-stage training, the dataset UAV-Person needs to be constructed.

[0038] Specifically, the construction process of the UAV-Person dataset is as follows: First, a DJI industrial-grade drone (such as the Mavic 3 Enterprise) was used for fixed-point cruise filming at an altitude of 50-80 meters and an image resolution of 4000×3000 pixels to provide sufficient detail for small target detection. Then, tools such as ffmpeg were used to sample frames from the filmed video to collect the necessary data. Next, X-Anylabeling was used to label the collected pedestrian data, ensuring that clear targets in each image were accurately labeled. It is important to note that the labeling criteria were based on clarity and independence, and targets with low clarity or recognizability were not labeled. After labeling, the UAV-Person dataset was created using publicly available drone datasets. Since YOLO training can be augmented through hyperparameter settings, data augmentation was not performed during the preprocessing stage.

[0039] S3. Clustering Analysis: Based on the pedestrian detection completed by the YOLOv8-AFPN detection model, a density-based clustering judgment algorithm is introduced to identify clustering behavior by analyzing the spatial distribution of detected pedestrians.

[0040] In step S3, the density-based clustering judgment algorithm adopts the regional density algorithm or the DBSCAN density clustering method.

[0041] The operation process of the region density algorithm includes the following steps: (1) Divide the detection image into a uniform grid of N×M, with each grid cell being S×S pixels in size; (2) Count the number of pedestrians in each grid cell and calculate the local density of each grid cell, where density = number of pedestrians in the grid cell / grid area; (3) Set a density threshold. If the density of a certain grid exceeds the threshold, mark the grid as a "potential clustering area". Merge adjacent "potential clustering areas" to form a clustering area.

[0042] The DBSCAN density clustering method includes the following steps: (1) Extract the center coordinates of the bounding boxes of all pedestrians from the YOLO detection results and construct a set of pedestrian location points; (2) Set the neighborhood radius eps and the minimum number of samples required to form the core point; (3) Perform DBSCAN clustering on the point set to obtain multiple clusters; (4) Traverse each cluster. If the number of points in the cluster is greater than or equal to the minimum number of samples, it is determined to be an aggregation. Draw all the bounding boxes of the cluster on the image, and use the convex hull or minimum bounding rectangle to represent the aggregation region. Draw the bounding box of the aggregation region and the density heatmap on the image.

[0043] S4. Report Generation: After the cluster analysis is completed, a risk assessment report is generated based on the analyzed data.

[0044] In summary, the intelligent crowd gathering detection method from the perspective of drones provided in this embodiment solves the problem of blind spots in fixed monitoring by collecting images through drone patrols. It can flexibly cover every corner of open areas such as squares and transportation hubs, making up for the lack of a global perspective in existing technologies. At the same time, by replacing the PAN-FPN structure of YOLOv8 with AFPN to form a dedicated detection model and using this dedicated detection model for pedestrian detection, the feature loss problem of small targets and dense pedestrians under the perspective of drones can be effectively alleviated, and the detection accuracy can be greatly improved. Furthermore, a density-based clustering judgment algorithm is used to quantitatively judge clustering behavior, replacing manual experience judgment, and avoiding false alarms and missed detections caused by subjectivity.

[0045] Example 2 like Figure 4 As shown, this embodiment proposes an intelligent crowd gathering detection system from the perspective of an unmanned aerial vehicle (UAV), which is used to run the intelligent crowd gathering detection method from the perspective of an UAV as described in Embodiment 1. The system includes an airborne detection module, a data transmission module, a cloud management platform, and a model optimization module.

[0046] Specifically, the airborne detection module is deployed on the UAV's airborne edge computing device, integrating an image acquisition unit, a preprocessing unit, a YOLOv8-AFPN detection unit, and a clustering analysis unit. The image acquisition unit is used to patrol and photograph the target area and acquire aerial images. The preprocessing unit is used to adjust the acquired images to 640×640 pixels and perform normalization. The YOLOv8-AFPN detection unit is used to load a dedicated YOLOv8-AFPN detection model for pedestrian detection. The clustering analysis unit is used to analyze the spatial distribution of pedestrians and identify clustering behavior through a region density algorithm or DBSCAN density clustering method.

[0047] The data transmission module establishes a two-way data interaction between the airborne detection module and the cloud management platform using a low-latency image transmission link. This is used to transmit the original aerial images, pedestrian detection results, cluster area markers, density heatmaps, and preliminary risk information output by the airborne detection module back to the cloud management platform in real time, as well as to receive drone patrol instructions and model update packages pushed by the model optimization module from the cloud management platform.

[0048] The cloud management platform includes a data storage unit, a situation visualization unit, a report generation unit, and an alarm push unit. The data storage unit stores all image data, detection logs, and aggregation analysis results transmitted back by the data transmission module, and also stores the UAV-Person dataset. The situation visualization unit displays information such as aggregation areas, density heatmaps, and pedestrian distribution in an intuitive way. The report generation unit automatically generates risk assessment reports based on aggregation analysis data. The alarm push unit pushes alarm information to management personnel terminals when medium- or high-risk aggregations are identified.

[0049] The model optimization module is used to continuously train the dedicated YOLOv8-AFPN detection model based on cloud data, and to remotely iterate and upgrade the model via OTA.

[0050] In the above embodiments, the image acquisition unit of the airborne detection module covers blind spots that fixed monitoring cannot reach through drone patrols. The preprocessing unit, YOLOv8-AFPN detection unit, and aggregation analysis unit complete the entire process of "acquisition-preprocessing-detection-analysis" efficiently at the edge. The data transmission module receives the airborne output through a low-latency image transmission link and transmits aerial images, detection results, aggregation markers, and other data back in real time. The two work together to form a closed loop of "aerial acquisition-edge computing-real-time data transmission," which not only solves the problem of limited field of view of fixed monitoring but also avoids response delays caused by data transmission lag, achieving dynamic full-area coverage and near real-time analysis of the target area.

[0051] The data transmission module synchronizes the onboard aggregation analysis results (such as aggregation areas, number of people, and density) to the cloud management platform in real time. The cloud-based situation visualization unit, combined with historical data from the storage unit, intuitively presents a heat map of the entire population distribution and aggregation hotspots, helping managers quickly grasp the overall situation. At the same time, the cloud-based report generation unit automatically generates risk assessment reports based on the onboard analysis data, and the alarm push unit pushes notifications in real time when medium- to high-risk aggregations are identified. The three work together to seamlessly connect "edge analysis - cloud judgment - personnel response," avoiding the problems of "information fragmentation" and "delayed decision-making" in traditional manual inspections, and providing data support for rapid evacuation and disposal.

[0052] The cloud management platform's storage unit continuously accumulates detection data transmitted from the airborne system, providing rich training samples for the model optimization module. Based on this data, the model optimization module continuously trains the YOLOv8-AFPN model, and then pushes the optimized model to the airborne detection module via the OTA (Over-The-Air) function of the data transmission module, completing edge-end model iteration. These three elements form a positive cycle of "data accumulation - model optimization - edge application," enabling the system to adapt to complex scenarios such as different weather conditions, lighting, and occlusion, avoiding the performance degradation problem of traditional models after deployment, and ensuring long-term detection accuracy and stability.

[0053] In summary, the intelligent crowd gathering detection system from the perspective of drones provided in this embodiment integrates a complete "collection-preprocessing-detection-analysis" process unit in its airborne detection module. Coupled with bidirectional data interaction via a low-latency image transmission link, it achieves "calculation while flying and transmission immediately upon detection," solving the problem of delayed response in existing technologies and ensuring the timeliness of detection and alarms. At the same time, the cloud management platform intuitively displays information such as crowd density heat maps and gathering area markers through the situation visualization unit, and achieves full data retention in combination with the data storage unit. This helps managers quickly grasp the distribution and gathering trends of the entire population, providing a scientific basis for resource allocation and avoiding the limitations of "fragmented" monitoring by traditional manual patrols.

[0054] In summary, compared with the prior art, the present invention has at least the following advantages: (1) The intelligent crowd gathering detection method under the perspective of UAV provided by the present invention solves the problem of blind spots in fixed monitoring by collecting images through UAV patrol flights. It can flexibly cover every corner of open areas such as squares and transportation hubs, making up for the lack of global perspective in the existing technology. At the same time, by replacing the PAN-FPN structure of YOLOv8 with AFPN to form a dedicated detection model and using the dedicated detection model for pedestrian detection, the feature loss problem of small targets and dense pedestrians under the perspective of UAV can be effectively alleviated, and the detection accuracy is greatly improved. Furthermore, the density-based clustering judgment algorithm is used to quantitatively judge the clustering behavior, replacing manual experience judgment, and avoiding false alarms and missed reports caused by subjectivity. (2) The intelligent detection system for crowd gathering from the perspective of UAV provided by the present invention integrates the entire process of “collection-preprocessing-detection-analysis” in its airborne detection module. Combined with the two-way data interaction of the low-latency image transmission link, it realizes “calculation while flying and transmission immediately after detection”, which solves the problem of delayed response in the existing technology and ensures the timeliness of detection and alarm. At the same time, the cloud management platform intuitively displays information such as crowd density heat map and gathering area marking through the situation visualization unit. Combined with the data storage unit, it realizes full data retention, which helps managers to quickly grasp the distribution and gathering trend of the entire area of ​​the crowd, provides a scientific basis for resource allocation, and avoids the limitations of the “fragmented” monitoring of traditional manual patrol.

[0055] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A method for intelligent detection of crowd gathering from the perspective of an unmanned aerial vehicle (UAV), characterized in that, Includes the following steps: Image acquisition: Aerial images containing pedestrians are obtained by using drones to conduct patrol flights over the target area; Pedestrian detection: The acquired images are preprocessed and then input into a dedicated YOLOv8-AFPN detection model for pedestrian detection; The dedicated YOLOv8-AFPN detection model is constructed by introducing AFPN into YOLOv8 and training it through multiple stages; the AFPN is used to replace the PAN-FPN structure used in the Neck part of YOLOv8. Clustering Analysis: Based on the pedestrian detection completed by the YOLOv8-AFPN detection model, a density-based clustering judgment algorithm is introduced to identify clustering behavior by analyzing the spatial distribution of detected pedestrians.

2. The intelligent crowd gathering detection method from the perspective of an unmanned aerial vehicle (UAV) as described in claim 1, characterized in that, The feature fusion of the AFPN includes the following steps: In Backbone's bottom-up feature extraction process, two adjacent shallow features are fused, and higher-level deep features are gradually incorporated into the fusion process. In the final stage, the top-level features are added to the feature fusion process, and the fused features are further learned using residual units.

3. The intelligent crowd gathering detection system and method from the perspective of an unmanned aerial vehicle (UAV) as described in claim 2, characterized in that, The AFPN integrates an ASFF, which is used to assign appropriate spatial weights to features at different levels during the feature fusion process of the AFPN; the formula for the ASFF is: ； in, ; This represents the target layer, i.e., the layer at which the desired feature map is located; Representing features from different levels, n=1,2,3; Representative from the first Layer conversion to the first The feature vector of the layer at position (i, j); the transformation includes upsampling or downsampling and channel adjustment; , and These correspond to the spatial weights of the features in layers 1, 2, and 3 at position (i, j), respectively.

4. The intelligent crowd gathering detection method from the perspective of an unmanned aerial vehicle (UAV) as described in claim 1, characterized in that, The multi-stage training includes the following steps: Phase 1: Random number seed 1, no pre-trained model used, training set: validation set = 4:1, strong data augmentation used; Second stage: random number seed 2, using the model from the first stage as the pre-trained model, training set:validation set = 4:1, using weak data augmentation and low learning rate; Phase 3: Random number seed 3, using the model from Phase 2 as the pre-trained model, training set:validation set = 1:3, data augmentation disabled, low learning rate.

5. The intelligent crowd gathering detection method from the perspective of an unmanned aerial vehicle (UAV) as described in claim 4, characterized in that, The training and validation sets are derived from the UAV-Person dataset, which is constructed using the following steps: The process involves using drones to capture images, then sampling the captured video frames to obtain the necessary data. The collected data is labeled to ensure that clear targets in each image are accurately marked; After the annotation was completed, the dataset UAV-Person was formed by combining it with publicly available drone datasets online.

6. The intelligent crowd gathering detection method from the perspective of an unmanned aerial vehicle (UAV) as described in claim 1, characterized in that, The density-based clustering decision algorithm employs a region density algorithm, and the operation flow of the region density algorithm includes the following steps: The detected image is divided into an N×M uniform grid, with each grid cell being S×S pixels in size; Count the number of pedestrians in each grid cell and calculate the local density of each grid cell, where density = number of pedestrians in the grid cell / grid area; A density threshold is set. If the density of a certain grid exceeds the threshold, the grid is marked as a "potential clustering area". Adjacent "potential clustering areas" are merged to form a clustered region.

7. The intelligent crowd gathering detection system and method from the perspective of an unmanned aerial vehicle (UAV) as described in claim 1, characterized in that, The density-based clustering decision algorithm employs the DBSCAN density clustering method, and the operation process of the DBSCAN density clustering method includes the following steps: Extract the center coordinates of the bounding boxes of all pedestrians from the YOLO detection results to construct a set of pedestrian location points; Set the neighborhood radius eps and the minimum number of samples required to form the core point; Perform DBSCAN clustering on the point set to obtain multiple clusters; Traverse each cluster. If the number of points in the cluster is greater than or equal to the minimum number of samples, it is determined to be a cluster. Draw all the bounding boxes of the cluster on the image, and represent the clustered region with a convex hull or minimum bounding rectangle. Draw the bounding box of the clustered region and the density heatmap on the image.

8. The intelligent crowd gathering detection system and method from the perspective of an unmanned aerial vehicle (UAV) as described in claim 1, characterized in that, The preprocessing steps include: The acquired images were adjusted to 640×640 and then normalized and augmented.

9. The intelligent crowd gathering detection method from the perspective of an unmanned aerial vehicle (UAV) as described in claim 1, characterized in that, It also includes the following steps: Report generation: After the cluster analysis is completed, a risk assessment report is generated based on the analyzed data.

10. A crowd gathering intelligent detection system from an unmanned aerial vehicle (UAV) perspective, used to run the crowd gathering intelligent detection method from an UAV perspective as described in any one of claims 1-9, characterized in that, It includes an airborne detection module, a data transmission module, a cloud management platform, and a model optimization module; The airborne detection module is deployed on the UAV's airborne edge computing device and integrates an image acquisition unit, a preprocessing unit, a YOLOv8-AFPN detection unit, and an aggregation analysis unit. The image acquisition unit is used to scan and capture aerial images of the target area. The preprocessing unit is used to adjust the acquired images to 640×640 pixels and perform normalization. The YOLOv8-AFPN detection unit is used to load a dedicated YOLOv8-AFPN detection model for pedestrian detection; the clustering analysis unit is used to analyze the spatial distribution of pedestrians and identify clustering behavior through a region density algorithm or DBSCAN density clustering method. The data transmission module establishes a two-way data interaction between the airborne detection module and the cloud management platform using a low-latency image transmission link. This is used to transmit the original aerial images, pedestrian detection results, cluster area markers, density heatmaps, and preliminary risk information output by the airborne detection module back to the cloud management platform in real time, as well as to receive drone patrol instructions and model update packages pushed by the model optimization module from the cloud management platform. The cloud management platform includes a data storage unit, a situation visualization unit, a report generation unit, and an alarm push unit. The data storage unit stores all image data, detection logs, and aggregation analysis results transmitted back by the data transmission module, and also stores the UAV-Person dataset. The situation visualization unit displays information such as aggregation areas, density heatmaps, and pedestrian distribution in an intuitive manner. The report generation unit automatically generates risk assessment reports based on aggregation analysis data. The alarm push unit pushes alarm information to management personnel terminals when medium- or high-risk aggregations are identified. The model optimization module is used to continuously train the dedicated YOLOv8-AFPN detection model based on cloud data, and to remotely iterate and upgrade the model via OTA.

Images