Unmanned aerial vehicle patrol method for public security management

Abstract

The invention belongs to the technical field of public security management, and particularly relates to an unmanned aerial vehicle patrol method for public security management, which comprises the following steps: step 1, unmanned aerial vehicle hardware configuration, step 2, patrol path planning, and step 3, computer vision technology: through video data acquired by a camera, performing real-time analysis on pictures by using the computer vision technology, and obtaining a patrol path of the unmanned aerial vehicle; identifying an abnormal behavior; step 4, a machine learning model; step 5, an automatic alarm system; and step 6, data storage and analysis. According to the method, abnormal behaviors or events in large-scale data can be detected and predicted in real time, the self-adaptive learning capability is achieved, an alarm is triggered immediately when the abnormal behaviors or events occur, the real-time performance is particularly important in the aspect of preventing and dealing with emergencies, a security administrator can be helped to make a response rapidly, and occurrence or expansion of the abnormal behaviors is reduced.

Description

Technical Field

[0001] The present invention belongs to the technical field of public security management, and in particular relates to a drone patrol method for public security management. Background Art

[0002] With the rapid development of drone technology, drone technology related to security monitoring also needs to be updated. In the current public security management, it mainly relies on the full coverage of surveillance cameras, combined with human patrols, to achieve timely police response to abnormal behaviors such as suspicious gatherings, but this traditional public security management method is not efficient enough and still has many limitations, such as human resource consumption, lack of efficiency and loss of field of view. Therefore, in order to solve the above problems, a drone combining computer vision and machine learning technology is proposed, which can not only broaden the application scenarios of public security management, but also significantly improve the efficiency and accuracy of task execution. Summary of the invention

[0003] The purpose of the present invention is to provide a drone patrol method for public security management, so as to solve the problems existing in the background technology.

[0004] In order to achieve the above technical objectives, the technical solution adopted by the present invention is as follows:

[0005] A drone patrol method for public security management comprises the following steps:

[0006] Step 1. UAV hardware configuration: The drone is equipped with a high-definition camera, infrared camera, panoramic camera, GPS module and obstacle avoidance sensor to ensure that it can conduct autonomous patrol and real-time monitoring in various environments;

[0007] Step 2. Patrol route planning: The patrol route of the drone is pre-set through GPS and Geographic Information System (GIS). The drone can not only patrol according to the preset route, but also dynamically adjust the route according to real-time needs;

[0008] Step 3. Computer vision technology: Use computer vision technology to analyze the video data collected by the camera in real time and identify abnormal behavior;

[0009] Step 4. Machine learning model: Train the deep learning model to identify and classify various types of abnormal behaviors. The model is optimized through a large amount of training data to improve the accuracy of recognition and the speed of response.

[0010] Step 5. Automatic alarm system: When the drone detects abnormal behavior, the system will automatically trigger an alarm and send the relevant video and location information to the backend management center to promptly notify relevant personnel for processing;

[0011] Step 6. Data storage and analysis: The data collected during the patrol process is stored in the cloud for subsequent analysis and mining to support decision-making. This data can also be used to further optimize the machine learning model and improve the intelligence level of the system.

[0012] A population-based meta-heuristic algorithm is used to implement path planning for drone swarms:

[0013] Step 1.1: UAV flight area representation, using (x, y, z) as the three-dimensional coordinates of the path waypoints, the flight space of the drone group is represented as follows:

[0014]

[0015] Step 1.2: Obstacle model, model the obstacles, (x k1 ,y k1 , z k1 ) is the radius R k1 The coordinates of the circular obstacle in the three-dimensional environment, then Obstacles can be represented as:

[0016]

[0017] The coordinates (x k1 ,y k1 , z k1 ) is calculated as follows:

[0018]

[0019]

[0020]

[0021] Step 1.3: Modeling of the objective function. The objective function includes determining the path length, considering environmental constraints, and avoiding collisions with obstacles and other drones in the group. Therefore, the objective function can be expressed as:

[0022] F=F pl +F oc +F mc

[0023] where F pl is the cost associated with the path length, F oc is the cost of the drone colliding with an obstacle, F mc is the collision cost between drones;

[0024] Step 1.4: Improved sine-cosine algorithm, based on chaotic candidate solution initialization; introduce nonlinear decreasing step size, improve position update strategy; add convergence factor to the search mechanism to speed up the convergence of the search.

[0025] In step 3, the image is collected by the camera, and then the image behavior is detected in real time to identify abnormal behavior; its framework mainly consists of two parts: the main model and the auxiliary SSL model, and the main part includes a dual-stream architecture involving spatial stream and temporal stream;

[0026] Step 3.1: Main model (an effective two-stream gating network): This main network consists of three modules: spatial block, temporal block and merging block; the spatial RGB module receives as input the consecutive frames, which are cropped to extract the region of interest (ROI), and the temporal block receives as input the optical flow of the frame sequence, which is calculated using the GunnarFarneback algorithm;

[0027] The ROI calculation in step 3 includes:

[0028] Step 3.11: Given the normalized and denoised optical flow frame s i , whose amplitude can be calculated as Where S i，j Represents: the jth component of the i-th frame;

[0029] Step 3.12: The sum of the amplitudes of each frame generates a 224×224 motion intensity map, on which the average is calculated;

[0030] Step 3.13: Use it as a threshold for additional noise filtering;

[0031] Step 3.2: Auxiliary model (VICReg of joint video streaming architecture).

[0032] The auxiliary model consists of using the basic idea of ​​VICReg to use a loss function with different terms. The variance regularization term v(Z) is calculated along the batch dimension as the standard deviation of the embedding, and the covariance regularization term c(Z) encourages the network to decorrelate the dimensions of the embedding. The invariance term s(Z, Z) is minimized by ′ ) to reduce the difference between the two data modes, s(Z, Z ′ ) is the mean squared Euclidean distance between each pair of embedding vectors, and the overall loss function L(Z, Z′) is the weighted average of these terms:

[0033] L(Z, Z′)=λs(Z, Z′)+μ[v(Z)+v(Z′)]+ν[c(Z)+c(Z′)]

[0034] Among them, λ, μ and ν are hyperparameters, specifically, λ=μ=25 and v=1.

[0035] The present invention can detect and predict abnormal behaviors or events in large-scale data in real time, and has adaptive learning capabilities, and can immediately trigger alarms when encountering abnormal behaviors or events. This real-time performance is particularly important in preventing and responding to emergencies, and can help security administrators respond quickly and reduce the occurrence or expansion of abnormal behaviors. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The present invention is further illustrated by means of the non-limiting examples given in the accompanying drawings.

[0037] Figure 1 A schematic diagram of the patrol route planning process of the present invention;

[0038] Figure 2 It is a schematic diagram of the computer vision technology recognition process of the present invention. DETAILED DESCRIPTION

[0039] In order to enable those skilled in the art to better understand the present invention, the technical solution of the present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0040] Example 1: Figure 1-2 As shown, the steps of a drone patrol method for public security management of the present invention are as follows:

[0041] 1. UAV hardware configuration: The UAV is equipped with a high-definition camera, an infrared camera, a panoramic camera, a GPS module and an obstacle avoidance sensor to ensure that it can conduct autonomous patrols and real-time monitoring in various environments.

[0042] 2. Patrol route planning: The patrol route of the drone can be pre-set through GPS and Geographic Information System (GIS). The drone can not only patrol according to the preset route, but also dynamically adjust the route according to real-time needs.

[0043] 3. Computer vision technology: Video data collected by the camera is analyzed in real time using computer vision technology to identify abnormal behavior.

[0044] 4. Machine Learning Model: Train deep learning models to identify and classify various types of abnormal behaviors. The models are optimized through a large amount of training data to improve the accuracy of recognition and the speed of response.

[0045] 5. Automatic alarm system: When the drone detects abnormal behavior, the system will automatically trigger an alarm and send relevant video and location information to the background management center, and promptly notify relevant personnel for processing.

[0046] 6. Data storage and analysis: The data collected during the patrol process is stored in the cloud for subsequent analysis and mining to support decision-making. This data can also be used to further optimize the machine learning model and improve the intelligence level of the system.

[0047] Step 2:

[0048] In this step, the patrol route of the drone is preset through GPS and GIS, as well as high-precision maps of known patrol areas. Based on the online path planning algorithm, the drone can dynamically adjust the path according to real-time requirements.

[0049] For path planning in this step, we will use an improved population-based metaheuristic algorithm, the sine-cosine algorithm (SCA), which we call the iSCA algorithm here. Through the iSCA path planning algorithm, we will implement path planning for the drone swarm, avoid collisions between drones and environmental obstacles, and determine the most effective patrol path in the area.

[0050] Specific steps:

[0051] 1. UAV flight area representation: In the path planning of UAV swarm, the goal is to find an optimal and feasible path for the UAV from the starting position to the target position under the constraints of complex environment. Throughout the study, we use (x, y, z) as the three-dimensional coordinates of the path waypoints. The flight space of the UAV swarm is represented as follows:

[0052]

[0053] 2. Obstacle model: Nowadays, various sensing technologies such as infrared, lidar, GPS, etc. can be used to obtain accurate and up-to-date terrain maps and obstacle locations. In this step, it is assumed that the spatial boundaries and locations of obstacles are known in advance. We model the obstacles. If (x k1 ,y k1 , z k1 ) is the radius R k1 In a three-dimensional environment The coordinates of the circular obstacle, then The obstacle can be represented as follows

[0054]

[0055] The coordinates (x k1 ,y k1 , z k1 ) is calculated as follows:

[0056]

[0057]

[0058]

[0059] 3. Modeling of the objective function: In path planning, the objective function includes determining the path length, considering environmental constraints, and avoiding collisions with obstacles and other drones in the group. Our objective function aims to minimize the impact of obstacles while

[0060] Minimize the overall path length. Therefore, the objective function can be expressed as:

[0061] F=F pl +F oc +F mc (6)

[0062] where F pl is the cost associated with the path length, F oc is the cost of the drone colliding with an obstacle, F mc is the collision cost between drones. The goal is to minimize the objective function F. The next section describes F pl 、F oc and F mc Mathematical formula of .

[0063] 3.1 Costs associated with path length: The expected flight path of a mission is shorter because shorter paths consume less fuel and are less likely to encounter unforeseen threats. To evaluate the costs associated with path length, we

[0064] The following path length ratios (PLRs) are used:

[0065]

[0066] Where D is the total number of path points in the path, (x j ,y j , z j ) are the coordinates of the jth path point, (x1, y1, z1) and (x D ,y D , z D ) are the coordinates of the starting and ending waypoints of the path, respectively. Here, the denominator represents the length of the shortest path between the starting and ending waypoints, and the numerator represents the length of the flight path. Therefore, F pl Always greater than or equal to 1, the smaller the F pl Values ​​higher correspond to shorter flight path lengths.

[0067] 3.2 Obstacle Cost: In order for the drone to fly safely, the planned path must avoid all obstacles. Even if a point in the solution passes through an obstacle, it may incur a high cost. i It's a dronei The planning path, and (x j ,y j , z j ), j = 1, 2, ..., D is the path i Medium th The coordinates of the waypoints, then each waypoint (j=1, 2, ..., D) should be checked against all obstacles to see if they are waypoints that fall into obstacles. To do this, the distance between the waypoint and the center of the obstacle is considered. If the distance between the waypoint and the center of the obstacle is greater than the radius of the obstacle, the waypoint is considered not to have fallen into the obstacle. In this case, a negligible cost is given to the objective function as the obstacle cost. On the contrary, when the distance between them is less than the radius of the obstacle, a high cost is assigned as a penalty. Therefore, the cost of obstacle avoidance is defined as follows:

[0068]

[0069] Where α∈[0,1] is the control parameter, k is the total number of obstacles, yes The radius of the obstacle, dist(j, k1) represents the radius of the obstacle on the path i Medium th Waypoints and The distance between obstacle centers is defined as follows:

[0070]

[0071] where (x(j), y(j), z(j)) represents j th The coordinates of the path points, (x0(k1), y0(k1), z0(k1)) are The coordinates of the obstacle center.

[0072] 3.3 Cost of drone member collision: When planning a path for a drone swarm, collision avoidance must be considered. All drones should maintain a reasonable distance from each other. The probability of collision between drones increases with the density of the drone swarm. Therefore, when generating a drone swarm path, it is extremely important to ensure that the drones are not too close to each other. If the path i It's a drone i The planned path, path o is the planned path of any other UAV, then path i Each waypoint must be associated with the path o ) is checked at each waypoint. To do this, it is necessary to consider the safety distance (sd) between paths. The associated cost of a collision between drones can be written as follows:

[0073]

[0074] Where D and Path i and path o The number of path points in . α∈[0,1] is the control parameter, sd is the number of path points in .

[0075] Distance between machines, dist(j, j1) represents the path i and path o The distance between is defined as follows:

[0076]

[0077] Where (x j ,y j , z j )and Path i and path o In the above model, the cost of the UAV member collision is mainly determined by the path i and other paths o Because the safety distance (sd) must be maintained, the cost will increase when the distance between paths is less than or equal to sd, and the cost decreases rapidly as the distance between paths increases.

[0078] 4. Improved Sine-Cosine Algorithm (iSCA): In order to avoid local optimality and search for global optimality more accurately and quickly, this study makes the following improvements to SCA: initialization of candidate solutions based on chaos; introduction of nonlinear decreasing step size to improve the position update strategy; and addition of a convergence factor to the search mechanism to speed up the convergence of the search.

[0079] 4.1 Chaos-based population initialization: In any evolutionary algorithm, population initialization plays a very important role in the convergence speed and quality of the final solution. Generally speaking, random initialization is the most commonly used method to generate the initial population when there is no information about the solution. SCA uses uniformly distributed random solutions to initialize the population of candidate solutions. According to, when the distribution is more uniform, the population maintains rich diversity, thereby increasing the chances of faster convergence and better solution quality. Therefore, chaos-based initialization helps to maintain better diversity among potential drone swarm paths. Compared with random distributions over 10,000 times, logistic maps have the advantage of more uniform distribution. In this work, in order to enrich the diversity of the initial population, logistic maps, which are one of the simplest and most widely used chaotic maps, are used.

[0080] y j+1 =μ×yj ×(1-y j ), j = 0, 1, 2, ... (12)

[0081] where y j for j th Chaotic variable. μ is the bifurcation coefficient. If μ∈[3.57, 4], a chaotic state occurs. When μ=4, y0∈(0, 1), the system generates a uniform chaotic signal, which will be used to initialize the candidate solution. The steps to implement initialization based on logical mapping are as follows:

[0082] 1. First, set y0∈(0, 1) and use the following formula to generate chaotic variables of D (overall dimension):

[0083] y j+1 =μ×y j ×(1-y j ), j=0, 1, 2,..., D(13)

[0084] where y j Indicates j th variable.

[0085] 2. Repeat the previous step for i=0, 1, 2, ..., N (population size), and generate initial chaotic variables for each candidate solution i.

[0086] 3. Initialize the candidate solution as follows: x i，j =x min，j +y i，j ×(x max，j -x min，j ), i=1, 2,..., N, j=1, 2,..., D(14)

[0087] 4. The i-th candidate solution using logisticmap is:

[0088]

[0089] 4.2 Improved Position Update Mechanism: In SCA, the control parameter r1 controls exploration in early iterations and exploitation in later iterations. This transition parameter can be further modified in order to balance exploration and exploitation during the search process. The control parameter r1 in SCA is a linear function that decreases linearly from 0. Due to its linearity, it sometimes produces abrupt changes when jumping from one iteration to the next. In some cases, the mutation may result in skipping good solutions; therefore, valuable information about the quality of the search area may be lost. In this study, a modified formula for r1 is proposed to make it decrease exponentially from 0 to avoid all these problems.

[0090]

[0091] Where t and MAX iteration are the current number of iterations and the maximum number of iterations respectively.

[0092] 4.3 Convergence Factor: A convergence factor CF is introduced into the search mechanism. This convergence factor CF helps iSCA to converge quickly while balancing exploration and exploitation. CF is defined as follows:

[0093]

[0094] In (Formula 17), the convergence factor (CF) is inversely proportional to the number of iterations. Its smaller value corresponds to less dependence on the current position, and its higher value plays a greater role in determining the new position. Initially, when CF is large, the search process is clearly guided by the current position. In later iterations, when CF is small, it plays a smaller role and the new position depends more on the global optimal solution. As mentioned above, the nonlinear decreasing step size (Formula 16) helps to balance the exploration and exploitation of the search process well, while the convergence factor (Formula 17) helps to converge quickly. Therefore, the following proposed search equation (18) is used in iSCA to merge the two techniques and cancel the absolute value term, thereby obtaining better UAV swarm path planning performance in terms of solution quality, accuracy and convergence speed.

[0095]

[0096] Algorithm advantages:

[0097] 1. Path quality: By introducing chaos-based initialization methods and other improvements, iSCA can provide better path planning results.

[0098] 2. Accuracy: iSCA significantly improves the accuracy of drone swarm routing, which means iSCA is more stable and reliable in complex situations.

[0099] 3. Operational efficiency: iSCA is more efficient in generating the optimal path for drone swarms and has a faster running time than SCA. This makes iSCA more advantageous in scenarios that require fast calculations.

[0100] Step 3:

[0101] In this step, the camera collects images and then detects the behaviors in the images in real time to identify abnormal criminal behaviors.

[0102] For the detection of violent behavior in this step, we will adopt an innovative joint stream embedding architecture JOSENet. This architecture integrates an efficient multimodal video streaming network with a novel self-supervised learning paradigm tailored for video traffic. Through the JOSENet architecture, we will monitor violent behavior in real time to ensure that violent behavior in the drone patrol area can be captured in real time.

[0103] Framework Description: The proposed JOSENet framework for violence detection basically consists of two parts, the main model and the auxiliary SSL model. The main part includes a two-stream architecture involving spatial and temporal streams. This choice generally guarantees significant performance for video detection tasks. JOSENet involves a two-stream flow gating network (FGN) that performs violent action detection using labeled data from both spatial and temporal streams. To reduce both memory and computational costs, we use a lightweight setting for the model in terms of the number of frames per video. The JOSENet framework benefits from an auxiliary model to avoid any performance loss. This auxiliary network implements a novel SSL approach that receives unlabeled data as input. The weights of the auxiliary model are optimized by minimizing the VICReg loss and are subsequently used for pre-training of the main supervised model. After the pre-training phase, a fine-tuning strategy is used to refine and tailor the pre-trained weights to meet the specific requirements of the main model. The auxiliary SSL network allows JOSENet to achieve the best trade-off between performance and resources used. In the following, we will introduce the two models in the proposed framework in detail.

[0104] The main model (an effective two-stream gating network): The main target model is based on a two-stream FGN. The motivation for choosing a two-stream architecture is the benefits brought by multi-modal video architectures and the excellent performance of FGN on the RWF-2000 dataset. The main network consists of three modules: spatial block, temporal block, and merging block.

[0105] Spatial Block. The spatial RGB module receives as input consecutive frames that are cropped to extract regions of interest (ROI). The purpose of ROI is to reduce the amount of input video data so that the network only focuses on areas with greater motion intensity. The calculation of ROI includes normalization and subtracting the mean of each optical flow frame to achieve denoising. Given a normalized and denoised optical flow frame s i , whose amplitude can be calculated as Where S i，jRepresents the jth component of the i-th frame. The sum of the amplitudes of each frame produces a 224×224 motion intensity map, over which the average is calculated and used as a threshold for additional noise filtering (i.e., if it is less than this threshold, the motion intensity map value is zeroed). In order to obtain the ROI center based on the motion intensity map, the probability density function along the two dimensions x, y of the motion intensity map is used. 10 different candidate points are selected as the center of the ROI and randomly extracted from this probability density function. The final value of the center (cx, cy) is obtained by the average of these 10 points for better robustness. Based on (cx, cy), an ROI of size 112×112 is extracted from the RGB frame, thereby applying cubic interpolation to reconstruct N frames of size 224×224. After processing, the output of the RGB block passes through the ReLU activation function. The input dimension of the resulting RGB block is 3×N×224×224, where the first dimension represents the RGB channels of the video clip.

[0106] Temporal block. The temporal block receives the optical flow of the frame sequence as input. The optical flow is calculated using the GunnarFarneback algorithm. For each RGB segment (F0, F1, ..., F s ), from each pair (F i-1 , F i ) to compute an optical flow frame. The resulting flow block size is 2×N×224×224. A sigmoid activation function at the end of this branch scales the output of the RGB embedding.

[0107] Merge block. The last FGN module defines the fusion strategy for managing the RGB and flow streams. The outputs of the RGB block and the flow block are multiplied and processed through temporal max pooling. This is a self-learned pooling strategy that uses the flow block as a gate to decide which information in the RGB block should be kept or discarded. Finally, the fully connected layer generates the output for the input.

[0108] Computational efficiency of the main model: Since violence detection is a real-time application, it is necessary to reduce the computational cost as much as possible, thereby finding more efficient ways to produce inferences and deploy models with lower memory usage and frame rate during inference. In order to keep the cost as low as possible, it is necessary to first process the number of frames N in the input video clip, and at the same time, the network should be able to use the correct window size T tf (Time Footprint) to learn the correct features. FGN uses N = 64 frames, N fps = 12.8 frames per second (FPS). To speed up inference, we use N = 16, which is a common value for the most commonly used action recognition architectures such as R(2+1)D, C3D, I3D, and P3D.

[0109] It is well known that in action recognition, very short windows can perfectly identify most activities, and using very high frame rates generally improves classification performance. However, action recognition methods do not always achieve the best performance at higher frame rates, but each method achieves the best results at different frame rates. Therefore, we set N fps We consider N as a hyperparameter and, for practical reasons, aim to reduce it. Thus, we find that fps = 7.5 is the best trade-off between computational cost and performance, resulting in This option can be considered to apply to a broad category of violent actions.

[0110] Efficient implementation. To reduce the segment length N, we modify the 2×2×2 max pooling layer of the merging block to 1×2×2 (i.e., the temporal dimension is reduced by 1), so that the output dimension of this block remains unchanged. In addition, to halve the memory requirements while speeding up the operation, we use mixed precision. To reduce internal covariance shift, we apply a 3D batch norm layer after each activation function in all blocks in the architecture, except the fully connected (FC) layers. Finally, to avoid overfitting, spatial culling with p=0.2 is applied after each batch normalization layer of the first two 3D convolution blocks in both RGB and optical flow blocks.

[0111] Auxiliary Model (VICReg for Joint Video Stream Architecture): Self-Supervised Pre-Training. The original FGN does not involve any pre-training. While in some cases pre-training may not improve the performance of classification metrics, it has been shown to improve the robustness of the model. In addition, using data from multiple sources helps to better generalize the two-stream architecture model. In this work, we study how self-supervised pre-training differs from fine-tuning methods in: 1) compensating for the performance loss caused by the resource limitations of the main network; 2) handling unlabeled data typical of real-world surveillance video applications; and 3) improving generalization performance by avoiding bias towards source labels in the source task. We employ four different SSL techniques: Odd-One-Out (O3D), ArrowofTime (AoT), Space-TimeCubicPuzzle (STCP), and VICReg. Clearly, all of the above SSL techniques work well for our FGN. Novel VICReg for JOSENet. In our framework, we focus on variance-variance-covariance regularization (VICReg), an SSL method for joint embedding architectures that preserves the information content of the embeddings while not requiring large amounts of memory, contrast samples, or memory banks like contrastive methods. To the best of our knowledge, VICReg is applied to video streams for the first time. The VICReg solution proposed for JOSENet relies on augmenting the joint information of RGB and flow batches. Let us consider an RGB batch I and an optical flow batch I′, both associated with the same input segment. Two augmented views of these batches X and X′ can be generated by using random transformations t and t′ sampled from the distribution T. The augmented batches are fed to two different encoders f θ and f′ θ′ The two branches do not have the same architecture and do not share the same weights. The outputs of these branches are fed as input into two Siamese merging blocks m γ , which have the same structure as the merging block, but without the temporal max pooling. In practice, the removal of this pooling seems to be beneficial due to the increase in the dimensionality of the expander input. The output representations Y and Y′ are fed as input to the two Siamese expanders h φ Thus, we generate a batch embedding Z=h of n vectors of dimension d. φ (Y) and Z′=h φ(Y′). By utilizing the Siamese pooling block in the self-supervised phase, we are able to generate a representation of the input data by leveraging a large part of the FGN architecture. As a result of this approach, we are able to obtain highly informative and useful feature representations of the input data that can subsequently be used for the main task. This particular solution is not feasible for other SSL techniques and can only be designed by using a method that can handle multi-modality, such as VICReg. In fact, among several configurations tested, we will show that this is the best setting for pre-training our two-stream FGN. We exploit the basic idea of ​​VICReg to use a loss function with three different terms. The variance regularization term v(Z) is calculated along the batch dimension as the standard deviation of the embeddings and aims to prevent complete collapse. The covariance regularization term c(Z) encourages the network to decorrelate the dimensions of the embeddings and thus avoid encoding similar information. This decorrelation of the embedding layer also leads to decorrelation of the representation layer. Our method requires reducing the difference between the two data modalities by minimizing the invariance term s(Z, Z′), which is the mean squared Euclidean distance between each pair of embedding vectors. The overall loss function L(Z, Z′) is the weighted average of these terms:

[0112] L(Z, Z′)=λs(Z, Z′)+μ[v(Z)+v(Z′)]+v[c(Z)+c(Z′)](19)

[0113] Among them, λ, μ, and ν are hyperparameters (specifically, λ = μ = 25 and v = 1 work best in most cases). Since VICReg is an information maximization method, it does not need to use the techniques commonly used in the comparison methods. In addition, although VICReg is proposed for Siamese networks, one of its biggest advantages is that the two branches do not share the same parameters, architecture, and more importantly, do not share the same input mode.

[0114] Algorithm advantages:

[0115] 1. Real-time monitoring and response: Using JOSENet's efficient algorithms, the system can detect abnormal behaviors or events in large-scale data in real time and trigger alarms immediately. This real-time nature is particularly important in preventing and responding to emergencies, helping security administrators respond quickly and reduce the occurrence or expansion of criminal behavior. Compared with traditional algorithms, JOSENet can extract valuable information from massive data faster and provide real-time alarms and decision support.

[0116] 2. Accurate prediction and prevention: JOSENet has powerful prediction capabilities by learning historical data and behavior patterns. It can predict the probability of crime in a certain area within a specific time period, helping security personnel to deploy forces in advance and implement effective preventive measures. This accurate prediction capability significantly improves the efficiency of public security management, which is difficult to achieve with traditional algorithms.

[0117] 3. Adaptive learning capability: JOSENet has the ability to adaptively learn and dynamically adjust models and strategies based on new security situations and data. The system can be continuously optimized and improved without frequent manual intervention and reconfiguration. Compared with traditional fixed parameter algorithms, JOSENet is more capable of responding to emergencies and abnormal data and maintaining an efficient working state.

[0118] Step 4:

[0119] Train the deep learning model to classify and identify different types of targets. Train the model based on big data to ensure that it can effectively identify suspicious persons. The machine learning model used in this step will be consistent with the JOSENet model in step 3, and the SSL model (VICreg) in JOSENet will be used for self-supervisory training. In each suspicious person identification, the model itself will be optimized to make the final result more accurate and the identification of suspicious persons more precise.

[0120] Step 5:

[0121] After identifying abnormal violent behavior, the abnormal person is identified. After the system identifies abnormal behavior and confirms the abnormal person, the alarm mechanism is immediately triggered. The alarm information includes but is not limited to: event type, abnormal behavior description, abnormal person's identity information or characteristics, time and place of the event, etc. The system automatically sends the alarm information to relevant security agencies, such as the police station or security monitoring center. At the same time, the on-duty personnel can also be informed through voice alarm, SMS notification or email and other methods. After the alarm is triggered, the system continues to monitor the development of the situation and take further measures according to the preset safety protocol, such as closing the site entrance, issuing evacuation instructions, etc.

[0122] Step 6:

[0123] During patrols, the system continuously collects various data, including video, audio, sensor information, abnormal behavior detection records, and identified personnel information. These data are uploaded in real time and securely stored in a cloud database to ensure data integrity and security. Data storage uses high availability and redundant backup strategies to prevent data loss or damage and comply with legal requirements for privacy and data protection.

[0124] Data uploaded to the cloud is preprocessed before storage, including noise removal, format conversion, and data compression, to improve storage efficiency and the accuracy of subsequent analysis. Preprocessing also generates metadata, such as timestamps, geolocation tags, and event tags, to facilitate subsequent retrieval and analysis. Data stored in the cloud is regularly extracted for analysis, using techniques such as machine learning, data mining, and statistical analysis to discover hidden patterns, trends, and anomalies in the data.

[0125] The system analyzes patrol data to identify high-risk areas, common abnormal behavior patterns and time patterns, etc., to provide data support for the formulation of more effective security strategies. The analysis results will automatically generate reports to provide security managers with intuitive visual data displays, such as heat maps, trend maps and statistical summaries. These reports are used to support security decisions, such as adjusting patrol routes, optimizing resource allocation and formulating new safety rules and regulations, to improve the efficiency and effectiveness of patrols.

[0126] The specific embodiment of step 2:

[0127] 1. Environmental modeling and flight area definition: By building a mathematical model of the flight area, the flight space of the drone swarm is determined. This includes defining the three-dimensional coordinate range of the flight area and considering possible obstacles.

[0128] 2. Construction of the objective function: The objective function consists of three main parts: path length cost, obstacle avoidance cost, and collision cost between drones. The goal of path planning is to minimize this objective function and find a short and safe path.

[0129] 3. Improved Sine-Cosine Algorithm (iSCA) initialization: Chaotic mapping is used to initialize the algorithm to ensure a good distribution of the population and increase the diversity of the initial solutions.

[0130] 4. Path optimization and search: The improved sine-cosine algorithm (iSCA) is used for path optimization. The algorithm updates the path of the drone through iterations, gradually approaching the optimal solution. In each iteration, the algorithm calculates the cost of the current path according to the objective function and searches for a better path by adjusting the position of the drone.

[0131] 5. Convergence judgment and optimized path output: The convergence of the algorithm is accelerated by setting the convergence factor. When the algorithm meets the set convergence conditions or reaches the maximum number of iterations, the iteration stops. Finally, the optimal drone group path is output to ensure that the path can meet the mission requirements and avoid obstacles and collisions.

[0132] 6. Path verification and adjustment: Verify the output optimal path to ensure that it is feasible in the actual environment. If problems are found, return to the optimization step and readjust the path.

[0133] The specific embodiment of step 3 wherein:

[0134] 1. Model training: To train the main model of the JOSENet framework for self-supervised learning, we will use the RWF-2000 dataset (a dataset specifically for abnormal behavior detection) and the HMDB51 dataset (a human action dataset based on movie clips, containing movie clips and online videos of actions such as jumping, kissing, and laughing). After training, the model will be able to accurately identify abnormal behaviors and body movements.

[0135] 2. Data preprocessing: The input video is segmented into RGB frames and optical flow frames to capture spatial (such as the position and shape of objects) and temporal information (such as the continuity of actions, speed changes, etc.). The motion intensity map is used to determine key regions (ROIs) that may contain potential abnormal behaviors, thereby focusing computing resources on the areas where abnormalities are most likely to occur.

[0136] 3. Feature extraction: Extract features of RGB frames and optical flow frames respectively. The features of RGB frames reflect objects and scenes in the video (such as people's appearance, vehicles, environment, etc.), while the features of optical flow frames reflect motion information in the video (such as people's movement trajectory, direction and speed of object movement, etc.). By extracting these features, the system captures clues of various abnormal behaviors that may appear in the video.

[0137] 4. Model fusion: By fusing RGB features and optical flow features and applying a temporal max pooling strategy to optimize the final feature vectors, these optimized feature vectors are classified through a fully connected layer to identify specific abnormal behavior types.

[0138] 5. Computation optimization: Reduce the number of frames, lower the frame rate, and use mixed precision calculations to reduce computing costs. Ensure that the system can detect abnormal behavior in real time and respond.

[0139] 6. Self-supervised learning: Use the VICReg self-supervised learning model to pre-train unlabeled data to improve the performance of the model. Due to the huge amount of surveillance video data and insufficient labeled information, self-supervised learning uses unlabeled data to improve model performance and make it better generalized to actual abnormal behavior detection tasks.

[0140] 7. Result output: The JOSENet architecture performs binary classification based on the input RGB fragments and optical flow, where 1 is detected if abnormal behavior is detected, and 0 is otherwise. The final classification result output includes the identification and location of the type of abnormal behavior detected. The system will generate an alarm signal and mark the specific location and type of suspicious behavior in real time on the monitoring interface for security personnel to take timely action.

[0141] The above embodiments are merely illustrative of the principles and effects of the present invention, and are not intended to limit the present invention. Anyone familiar with the technology may modify or change the above embodiments without violating the spirit and scope of the present invention. Therefore, all equivalent modifications or changes made by those with ordinary knowledge in the technical field without departing from the spirit and technical ideas disclosed by the present invention should still be covered by the claims of the present invention.

Claims

1. A drone patrol method for public security management, characterized in that: The following steps are involved: Step 1. UAV hardware configuration: The drone is equipped with a high-definition camera, infrared camera, panoramic camera, GPS module and obstacle avoidance sensor to ensure that it can conduct autonomous patrols and real-time monitoring in various environments; Step 2. Patrol route planning: Through GPS and Geographic Information System (GIS), the patrol route of the drone is pre-set. The drone can not only patrol according to the preset route, but also dynamically adjust the route according to real-time needs; Step 3. Computer vision technology: Use computer vision technology to analyze the video data collected by the camera in real time and identify abnormal behavior; Step 4. Machine learning model: Train the deep learning model to identify and classify various types of abnormal behaviors. The model is optimized through a large amount of training data to improve the accuracy of recognition and the speed of response. Step 5. Automatic alarm system: When the drone detects abnormal behavior, the system will automatically trigger an alarm and send the relevant video and location information to the backend management center to promptly notify relevant personnel for processing; Step 6. Data storage and analysis: The data collected during the patrol process is stored in the cloud for subsequent analysis and mining to support decision-making. This data can also be used to further optimize the machine learning model and improve the intelligence level of the system.

2. The method for patrolling by unmanned aerial vehicles for public security management according to claim 1, characterized in that: A population-based meta-heuristic algorithm is used to implement path planning for drone swarms: Step 1.1: UAV flight area representation, taking (x, y, z) as the three-dimensional coordinates of the path waypoints, the flight space of the drone group is represented as follows: Step 1.2: Obstacle model, model the obstacles, (x k1 ,y k1 , z k1 ) is the radius R k1 The coordinates of the circular obstacle in the three-dimensional environment, then Obstacles can be represented as: The coordinates (x k1 ,y k1 , z k1 ) is calculated as follows: Step 1.3: Modeling of the objective function. The objective function includes determining the path length, considering environmental constraints, and avoiding collisions with obstacles and other drones in the group. Therefore, the objective function can be expressed as: F=F pl +F oc +F mc where F pl is the cost associated with the path length, F oc is the cost of the drone colliding with an obstacle, F mc is the collision cost between drones; Step 1.4: Improved sine-cosine algorithm, based on chaotic candidate solution initialization; introduce nonlinear decreasing step size, improve position update strategy; add convergence factor to the search mechanism to speed up the convergence of the search.

3. The method for patrolling by unmanned aerial vehicles for public security management according to claim 2, characterized in that: The modeling steps of the objective function in step 1.3 are as follows: Step 1.31: Path Cost Related Costs Use the following Path Length Ratio (PLR) to evaluate the costs associated with path length: Where D is the total number of path points in the path, (x j ,y j , z j ) are the coordinates of the jth path point, (x1, y1, z1) and (x D ,y D , z D ) are the coordinates of the starting and ending points of the path respectively; Step 1.32: Obstacle cost. If the distance between the path point and the obstacle center is greater than the radius of the obstacle, the path point is considered not to have fallen into the obstacle. In this case, a negligible cost is given to the objective function as the obstacle cost. When the distance between them is less than the radius of the obstacle, a high cost is assigned as a penalty. The cost definition formula for obstacle avoidance is: Where α∈[0,1] is the control parameter, k is the total number of obstacles, yes The radius of the obstacle, dist(j, k1) represents the radius of the obstacle on the path i Medium th Waypoints and The distance between the centers of obstacles, dist(j, k1) is defined as follows: where (x(j), y(j), z(j)) represents j th The coordinates of the path points, (x0(k1), y0(k1), z0(k1)) are Coordinates of the obstacle center; Step 1.33: The cost of a collision between drone members needs to take into account the safe distance (sd) between the drone group paths. Therefore, the relevant cost of a collision between drones is as follows: Where D and Path i and path o The number of path points in the path, α∈[0,1] is the control parameter, sd is the distance between drones, and dist(j,j1) represents the path i and path o The distance between dist(j, j1) is defined as follows: Where (x j ,y j , z j )and Path i and path o waypoints.

4. The method for patrolling by unmanned aerial vehicles for public security management according to claim 2, characterized in that: The steps of improving the sine-cosine algorithm in step 1.4 are as follows: Step 1.41: Chaos-based population initialization: y j+1 μ×y j ×(1-y j ), j 0, 1, 2,... where y j for j th Chaotic variable, μ is the bifurcation coefficient; Step 1.42: Improved position update mechanism, a modified formula for r1 is proposed to make it decrease exponentially from 0: Where t and MAX iteration are the current number of iterations and the maximum number of iterations respectively; Step 1.43: Convergence factor, introduce the convergence factor CF into the search mechanism:

5. The method for patrolling by unmanned aerial vehicles for public security management according to claim 4, characterized in that: To implement initialization based on logical mapping in step 1.41, the steps are as follows: Step 1.411: Set y0∈(0, 1) and use the above formula to generate chaotic variables of D (overall dimension); Step 1.412: Repeat the previous step for i=0, 1, 2, ..., N (population size), and generate initial chaotic variables for each candidate solution i; Step 1.413: Initialize candidate solution: x i,j =x min,j +y i，j ×(x max，j -x min,j ), i=1,2,...,N, j=1,2,...,D; Step 1.414: Use the i-th candidate solution of logisticmap:

6. The method for patrolling by unmanned aerial vehicles for public security management according to claim 1, characterized in that: In step 3, the image is collected by the camera, and then the image behavior is detected in real time to identify abnormal behavior; its framework mainly consists of two parts: the main model and the auxiliary SSL model, and the main part includes a dual-stream architecture involving spatial stream and temporal stream; Step 3.1: Main model (an effective two-stream gating network): The main network consists of three modules: spatial block, temporal block and merging block; the spatial RGB module receives as input the continuous frames, which are cropped to extract the region of interest (ROI), and the temporal block receives as input the optical flow of the frame sequence, which is calculated using the GunnarFarneback algorithm; Step 3.2: Auxiliary model (VICReg of joint video streaming architecture).

7. The method for patrolling by unmanned aerial vehicles for public security management according to claim 6, characterized in that: The ROI calculation in step 3 includes: Step 3.11: Given the normalized and denoised optical flow frame s i , whose amplitude can be calculated as Where S i，j Represents: the jth component of the i-th frame; Step 3.12: The sum of the amplitudes of each frame generates a 224×224 motion intensity map, on which the average is calculated; Step 3.13: Use this as a threshold for additional noise filtering.

8. The method for patrolling by unmanned aerial vehicles for public security management according to claim 6, characterized in that: The auxiliary model consists of using the basic idea of ​​VICReg to use a loss function with different terms. The variance regularization term v(Z) is calculated along the batch dimension as the standard deviation of the embedding. The covariance regularization term c(Z) encourages the network to decorrelate the dimensions of the embeddings. The difference between the two data modes is reduced by minimizing the invariance term s(Z, Z′), which is the mean squared Euclidean distance between each pair of embedding vectors. The overall loss function L(Z, Z′) is a weighted average of these terms: L(Z, Z′)=λs(Z, Z′)+μ[v(Z)+v(Z′)]+v[c(Z)+c(Z′)] Among them, λ, μ and v are hyperparameters, specifically, λ=μ=25 and v=1.

Images