Campus patrol unmanned aerial vehicle sensing system and method based on ZYNQ
By adopting the ARM+FPGA architecture of the ZYNQ platform in the campus patrol system, combining the convolutional neural network hardware accelerator and a specially designed pedestrian detection model, the problems of blind spots and weak video analysis capabilities are solved, and efficient pedestrian detection and intrusion monitoring are achieved. , improving the intelligence level of campus patrol system.
Patent Information
- Application Number
- CN202411042737.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-31
- Publication Date
- 2024-12-03
AI Technical Summary
The existing campus patrol system has problems such as surveillance blind spots, difficulty in passing during peak traffic, and weak surveillance video analysis capabilities. In addition, deep learning algorithms have large amounts of computing and long inference time on embedded devices, so it is impossible to achieve such as abnormal activity analysis and intrusion detection. and other functions.
Using a ZYNQ-based drone perception system, combined with ARM+FPGA platform, parallel computing of deep learning algorithms is realized. Pedestrian detection, trajectory tracking and intrusion detection are performed through convolutional neural network hardware accelerator and a specially designed pedestrian detection model VarKDnet, and pedestrian detection, trajectory tracking and intrusion detection are carried out. The Deep Sort algorithm and the Kalman filter perform target correlation and monitoring area judgment.
It realizes campus patrol monitoring without dead corners, reduces system costs and power consumption, shortens development cycle, enriches the functions of campus patrol perception system, realizes functions such as pedestrian detection, trajectory tracking, intrusion detection and aggregation analysis, and improves monitoring intelligence level of transformation.
Smart Images

Figure CN119068358A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of pattern recognition technology, and specifically relates to a ZYNQ-based campus patrol drone perception system and method. Background Art
[0002] With the rapid development of artificial intelligence (AI) technology, edge AI has made great progress as an emerging technology. Its development can effectively solve the problems of massive data processing, real-time response, data security, etc. faced by AI in industry applications. As the most densely populated area, campus security issues have long been a concern, and the demand for intelligence is particularly prominent. In actual campus management, the long-standing campus patrol task has gradually evolved from security guards on duty to commuter cars, patrol cars, and now surveillance throughout the campus. Despite the huge expenditure of manpower and financial resources, there are still phenomena such as blind spots in monitoring, difficulty for patrol cars to pass during peak hours, and dazzling and difficult to analyze multi-angle monitoring.
[0003] As a new technology in the era of artificial intelligence, drone technology has the advantages of active attack, high-altitude operation, wide field of view, and fast speed. However, the high price of drone-mounted computing platforms that can complete edge analysis of drone aerial videos limits the application of drones in campus perception and monitoring tasks to a certain extent. The mainstream drone-mounted platforms mainly include NVIDIA's Jet-son series and Intel's NUC series, as well as CPU+GPU solutions proposed by other manufacturers. Dedicated AI computing platforms have superior performance and low power consumption, but they have the characteristics of long development cycle and high cost. As a general-purpose image processor, GPU has the characteristics of high power consumption, which is not friendly to resource-scarce drone platforms. With the widespread application of embedded systems in various industries, the market demand for high-performance, low-power, and programmable solutions continues to increase. Xilinx launched ZYNQ SoC to integrate ARM processors and FPGAs in a single chip, realizing the combination of processor flexibility and FPGA's high parallel computing capabilities. This heterogeneous computing architecture provides significant performance improvements and greater flexibility, and is suitable for fields such as autonomous driving, industrial automation, and 5G communications. By integrating processing and programmable logic, Zynq SoC can meet real-time computing needs, reduce system power consumption and cost, and shorten development cycles.
[0004] The prosperity of deep learning technology has promoted the level of intelligence in all walks of life. Some researchers have introduced deep learning algorithms into video analysis to improve campus management efficiency, such as vehicle detection, license plate recognition, face recognition access control, etc., which has promoted the level of intelligence in campus to a certain extent. However, most of the current deep learning algorithms are general target monitoring algorithms, which are not customized according to actual needs. It is impossible to achieve analysis functions such as abnormal activity analysis, crowd stay, and intrusion detection. The monitoring capabilities of existing video surveillance analysis systems are relatively weak. Algorithms with better performance usually also mean large amounts of calculations and parameters, and the longer it takes to perform inference on embedded devices. In order to improve practicality and real-time performance, it is necessary to develop more lightweight deep learning algorithms. Summary of the invention
[0005] In order to solve the problems raised in the above background technology, the present invention proposes to use drones to replace manual campus patrols, use deep learning algorithms to replace manual video analysis, and realize parallel computing of deep learning algorithms at the edge through the ARM+FPGA ZYNQ platform.
[0006] To achieve the above object, the present invention adopts the following technical solutions:
[0007] The present invention provides a ZYNQ-based campus patrol drone perception system, including: a ZYNQ airborne platform, a power module, a flight control platform and a remote control module;
[0008] The ZYNQ airborne platform includes a PL end and a PS end. The PL end includes a data acquisition module, a convolutional neural network hardware accelerator, a data transmission module and a data storage module. The data acquisition module, the convolutional neural network hardware accelerator, the data transmission module and the data storage module are all connected to the PS end. The power module is connected to the ZYNQ airborne platform and the flight control platform. The ZYNQ airborne platform communicates with the flight control platform through a serial port, and the PS end is connected to the remote control module through the data transmission module.
[0009] Further, the PS end includes a video processing unit, and the video processing unit includes a pedestrian detection module, a pedestrian trajectory tracking module, an intrusion detection module and a pedestrian gathering analysis module;
[0010] The video processing unit is used to analyze and process the video stream collected by the data collection module;
[0011] The pedestrian detection module: detects the confidence and position coordinates of pedestrians from the video stream through the pedestrian detection model VarKDnet, and draws a pedestrian detection box on the image;
[0012] The pedestrian trajectory tracking module: uses the Deep Sort algorithm to implement pedestrian tracking, obtains the pedestrian detection frame, predicts the current position through the Kalman filter, obtains the predicted frame, and then calculates the matching degree between the tracking targets of the previous frame and the current frame through similarity calculation, performs data association through the Hungarian algorithm, and sets a separate ID for each tracking target;
[0013] The intrusion detection module: sets the monitoring area, uses the DeepSort algorithm to track the target according to the detection frame and confidence of the pedestrian, draws the monitoring area and the detection frame in each frame, records the motion data at the same time, determines whether an intrusion occurs by detecting whether the target center point is within the monitoring area, and records the intrusion event;
[0014] The pedestrian gathering analysis module reads previously saved motion data and video data, identifies stationary points by calculating the motion trajectory of each target, and considers that a target is stationary when it remains stationary within a fixed distance for more than a set time, i.e., it is a stationary point. The motion trajectory is drawn using lines of different colors, and color gradient is achieved by decreasing color values. For stationary points, a heat map is generated by drawing circles layer by layer. The size and color of the circles are adjusted according to the stationary time, and the darker the color, the longer the stay time.
[0015] Furthermore, the pedestrian detection model VarKDnet ensures that the algorithm performance does not decrease significantly while achieving faster reasoning speed, specifically:
[0016] 1) It has a three-branch structure, including a main branch composed of variable kernel convolution, feature extraction branches of different scales, and an identity mapping branch that maintains network performance; when the kernel size of the main branch is 3, the kernel size of the feature extraction branch is 1x1, which is mainly used to pre-extract more local information; when the kernel size of the main branch is greater than 3x3, the feature extraction branch is set to DW Conv3x3; in the process of channel number expansion, the convolution kernel size of the trunk branch is dynamically adjusted;
[0017] 2) Through the structural reparameterization technology, the multi-branch training is converted into a single-channel structure. First, the batch normalization layer and the convolution layer are fused. The convolution expression without bias is shown in formula (1), the calculation method of the batch normalization layer is shown in formula (2), and the result of batch normalization after the convolution layer is shown in formula (3), which is regarded as a convolution calculation with bias. Through mathematical equivalence, the convolution layer and the batch normalization layer are fused; secondly, the 3x3 convolution branch and the identity mapping branch are converted into a special 7X7 structure; finally, according to the additivity of convolution, the three branches are fused into a single-channel structure;
[0018] Conx(x)=Wx(1)
[0019]
[0020] Among them, x is the input data; W is the convolution kernel weight; μ is the mean; σ is the standard deviation; γ is the scaling factor; β is the offset parameter;
[0021] 3) The PRelu activation function is used to more effectively fit the pedestrian posture with symmetrical characteristics, and softmax classification and position regression are performed on multiple feature maps at the same time to obtain the pedestrian confidence and its position coordinates.
[0022] Furthermore, the convolutional neural network hardware accelerator is used to implement parallel computing of the human detection algorithm VarKDnet, specifically:
[0023] 1) Accelerate convolution calculation through pipeline operation. When performing the first row of convolution calculation, in the first clock cycle, execute X(n,0)*W00; in the second clock cycle, execute X(n,1)*W01 and add the calculation result of X(n,0)*W00. In addition, the first-level pipeline will also calculate X(n,1)*W00, which is the first calculation value of the next convolution window; in the third clock cycle, the third-level pipeline executes X(n,2)*W02+X(n,1)*W01+X(n,0)*W00, the first-level pipeline executes X(n,2)*W00, and the second-level pipeline executes X(n,1)*W00+X(n,2)*W01; and so on to accelerate subsequent convolution calculations;
[0024] 2) Using the weight sharing mechanism of convolutional neural networks, the same convolution kernel is used to convolve an image. Therefore, convolution kernel reuse is designed, and the same convolution kernel is used to perform convolution operations on multiple windows of an image at the same time to achieve parallel computing;
[0025] 3) A row buffer mechanism is used to further improve the degree of parallelism. The format of the loaded image data is 512*512. Since the input is an RGB image, 3 channels are required, so the row buffer size is set to 3×512.
[0026] Furthermore, the ZYNQ airborne platform adopts FPGA+ARM architecture.
[0027] The present invention also provides a ZYNQ-based campus patrol drone perception method, comprising the following steps:
[0028] Step 1: deploy the trained pedestrian detection model VarKDnet and Deep Sort algorithm to the ZYNQ airborne platform, connect the ZYNQ airborne platform to the flight control platform through the serial port, start the power supply, power the ZYNQ airborne platform and the flight control platform, and take off the drone to perform the campus patrol mission;
[0029] Step 2: During the flight, the PL side of the ZYNQ airborne platform collects the video stream of the crowd in the campus in real time through the camera, and transmits the video stream to the PS side for further analysis and processing;
[0030] Step 3: After receiving the video stream, the PS end scales and normalizes the video stream to 512*512*3, loads the pedestrian detection model VarKDNet to perform pedestrian detection, and outputs the pedestrian's confidence and its position coordinates;
[0031] Step 4: The PS uses the Deepsort algorithm to track the video stream after pedestrian detection, estimates it through Kalman filtering, associates the pedestrian in the previous frame with the pedestrian in the current frame, performs matching correction through the Hungarian algorithm, and sets a separate ID for each tracking target;
[0032] Step 5: After obtaining the results of pedestrian detection and trajectory tracking, intrusion detection and aggregation analysis are implemented by combining the built-in functions of OpenCV;
[0033] Step 6: The ZYNQ airborne platform transmits the processed video to the remote monitoring center through data transmission via the PS terminal for further analysis and decision making.
[0034] Compared with the prior art, the present invention has the following advantages:
[0035] (1) The campus patrol drone perception system proposed in the present invention uses drones as monitoring actuators to take the initiative. Compared with traditional surveillance videos, it has no blind spots and will not affect the passage of pedestrians during peak hours compared with patrol cars.
[0036] (2) The present invention proposes to use FPGA+ARM as the UAV edge computing platform and utilize FPGA to design an AI hardware accelerator to realize parallel processing of deep learning algorithms. Compared with a dedicated AI edge computing platform, this architecture has the advantages of low cost and short development cycle. Compared with a general CPU+GPU solution, it has the advantage of low power consumption.
[0037] (3) The present invention designs a lightweight convolutional neural network algorithm for embedded platforms with limited edge resources, which reduces the resource requirements and speeds up the reasoning speed.
[0038] (4) The present invention enriches the functions of the campus patrol perception system, realizes functions such as pedestrian detection, trajectory tracking, dangerous area intrusion detection, and aggregation analysis, and is a new generation of campus monitoring intelligent perception system. BRIEF DESCRIPTION OF THE DRAWINGS
[0039] Figure 1 This is a schematic diagram of the ZYNQ-based campus patrol drone perception system;
[0040] Figure 2 This is the network architecture diagram of the pedestrian detection model VarKDnet;
[0041] Figure 3 Flowchart for accelerating convolution computation for pipeline operation;
[0042] Figure 4 Flowchart for convolution kernel reuse. DETAILED DESCRIPTION
[0043] In order to further illustrate the technical solution of the present invention, the present invention is further described below through embodiments.
[0044] like Figure 1 As shown, a ZYNQ-based campus patrol drone perception system of this embodiment includes: a ZYNQ airborne platform, a power module, a flight control platform and a remote control module;
[0045] The ZYNQ airborne platform includes a PL end and a PS end. The PL end includes a data acquisition module, a convolutional neural network hardware accelerator, a data transmission module and a data storage module. The data acquisition module, the convolutional neural network hardware accelerator, the data transmission module and the data storage module are all connected to the PS end. The power module is connected to the ZYNQ airborne platform and the flight control platform. The ZYNQ airborne platform communicates with the flight control platform through a serial port, and the PS end is connected to the remote control module through the data transmission module.
[0046] The PS end includes a video processing unit, which includes a pedestrian detection module, a pedestrian trajectory tracking module, an intrusion detection module and a pedestrian gathering analysis module;
[0047] Video processing unit: analyzes and processes the video stream collected by the data acquisition module;
[0048] Pedestrian detection module: The pedestrian detection model VarKDnet is used to detect the confidence and position coordinates of pedestrians from the video stream, and a pedestrian detection box is drawn on the image;
[0049] The pedestrian detection model VarKDnet (such as Figure 2 ), specifically:
[0050] 1) It has a three-branch structure, including a main branch composed of variable kernel convolution, feature extraction branches of different scales, and an identity mapping branch that maintains network performance; when the kernel size of the main branch is 3, the kernel size of the feature extraction branch is 1x1, which is mainly used to pre-extract more local information; when the kernel size of the main branch is greater than 3x3, the feature extraction branch is set to DW Conv3x3; in the process of channel number expansion, the convolution kernel size of the trunk branch is dynamically adjusted;
[0051] 2) Through the structural reparameterization technology, the multi-branch training is converted into a single-channel structure. First, the batch normalization layer and the convolution layer are fused. The convolution expression without bias is shown in formula (1), the calculation method of the batch normalization layer is shown in formula (2), and the result of batch normalization after the convolution layer is shown in formula (3), which is regarded as a convolution calculation with bias. Through mathematical equivalence, the convolution layer and the batch normalization layer are fused; secondly, the 3x3 convolution branch and the identity mapping branch are converted into a special 7X7 structure; finally, according to the additivity of convolution, the three branches are fused into a single-channel structure;
[0052] Conx(x)=Wx(1)
[0053]
[0054] Among them, x is the input data; W is the convolution kernel weight; μ is the mean; σ is the standard deviation; γ is the scaling factor; β is the offset parameter;
[0055] 3) Using the PRelu activation function, softmax classification and position regression are performed simultaneously on multiple feature maps to obtain pedestrian confidence and its position coordinates.
[0056] Pedestrian trajectory tracking module: The Deep Sort algorithm is used to implement pedestrian tracking. After obtaining the pedestrian detection frame, the current position is predicted through the Kalman filter to obtain the predicted frame. The matching degree between the tracking target in the previous frame and the current frame is obtained through similarity calculation. The data is associated through the Hungarian algorithm, and a separate ID is set for each tracking target.
[0057] Intrusion detection module: Set the monitoring area, use the Deep Sort algorithm to track the target according to the detection frame and confidence of the pedestrian, draw the monitoring area and detection frame in each frame, record the motion data at the same time, determine whether an intrusion occurs by detecting whether the target center point is within the monitoring area, and record the intrusion event;
[0058] Pedestrian gathering analysis module: reads previously saved motion data and video data, identifies stationary points by calculating the motion trajectory of each target. When a target remains stationary within a fixed distance for more than a set time, it is considered to be in a stationary state, that is, a stationary point. Use lines of different colors to draw the motion trajectory, and achieve color gradient by decreasing color values. For stationary points, a heat map is generated by drawing circles layer by layer. The size and color of the circles are adjusted according to the stationary time. The darker the color, the longer the stay time.
[0059] The convolutional neural network hardware accelerator is used to implement parallel computing of the human detection algorithm VarKDnet, specifically:
[0060] 1) Accelerate convolution calculations through pipeline operations (such as Figure 3 As shown in the figure, when performing the first row of convolution calculation, in the first clock cycle, X(n,0)*W00 is executed; in the second clock cycle, X(n,1)*W01 is executed, and the calculation result of X(n,0)*W00 is added. In addition, the first-level pipeline will also calculate X(n,1)*W00, which is the first calculation value of the next convolution window; in the third clock cycle, the third-level pipeline executes X(n,2)*W02+X(n,1)*W01+X(n,0)*W00, the first-level pipeline executes X(n,2)*W00, and the second-level pipeline executes X(n,1)*W00+X(n,2)*W01; and so on to accelerate subsequent convolution calculations;
[0061] 2) Using the weight sharing mechanism of convolutional neural networks, design convolution kernel reuse (such as Figure 4 As shown in the figure, the same convolution kernel performs convolution operations on multiple windows of an image at the same time to achieve parallel computing;
[0062] 3) A row buffer mechanism is used to further improve the degree of parallelism. The format of the loaded image data is 512*512, and the row buffer size is set to 3×512.
[0063] The ZYNQ airborne platform of this embodiment adopts FPGA+ARM architecture, the data transmission module adopts 4G network transmission, and the camera adopts 0V5640 camera.
[0064] A campus patrol drone perception method based on ZYNQ in this embodiment includes the following steps:
[0065] Step 1: deploy the trained pedestrian detection model VarKDnet and Deep Sort algorithm to the ZYNQ airborne platform, connect the ZYNQ airborne platform to the flight control platform through the serial port, start the power supply, power the ZYNQ airborne platform and the flight control platform, and take off the drone to perform the campus patrol mission;
[0066] Step 2: During the flight, the PL side of the ZYNQ airborne platform collects the video stream of the crowd in the campus in real time through the camera, and transmits the video stream to the PS side for further analysis and processing;
[0067] Step 3: After receiving the video stream, the PS end scales and normalizes the video stream to 512*512*3, loads the pedestrian detection model VarKDNet to perform pedestrian detection, and outputs the pedestrian's confidence and its position coordinates;
[0068] Step 4: The PS uses the Deepsort algorithm to track the video stream after pedestrian detection, estimates it through Kalman filtering, associates the pedestrian in the previous frame with the pedestrian in the current frame, performs matching correction through the Hungarian algorithm, and sets a separate ID for each tracking target;
[0069] Step 5: After obtaining the results of pedestrian detection and trajectory tracking, intrusion detection and aggregation analysis are implemented by combining the built-in functions of OpenCV;
[0070] Step 6: The ZYNQ airborne platform transmits the processed video to the remote monitoring center through data transmission via the PS terminal for further analysis and decision making.
Claims
1. A campus patrol drone perception system based on ZYNQ, characterized in that: include: ZYNQ airborne platform, power module, flight control platform and remote control module; The ZYNQ airborne platform includes a PL end and a PS end. The PL end includes a data acquisition module, a convolutional neural network hardware accelerator, a data transmission module and a data storage module. The data acquisition module, the convolutional neural network hardware accelerator, the data transmission module and the data storage module are all connected to the PS end. The power module is connected to the ZYNQ airborne platform and the flight control platform. The ZYNQ airborne platform communicates with the flight control platform through a serial port, and the PS end is connected to the remote control module through the data transmission module.
2. According to the ZYNQ-based campus patrol drone perception system of claim 1, it is characterized in that: The PS end includes a video processing unit, which includes a pedestrian detection module, a pedestrian trajectory tracking module, an intrusion detection module and a pedestrian gathering analysis module; The video processing unit is used to analyze and process the video stream collected by the data collection module; The pedestrian detection module: detects the confidence and position coordinates of pedestrians from the video stream through the pedestrian detection model VarKDnet, and draws a pedestrian detection box on the image; The pedestrian trajectory tracking module: uses the Deep Sort algorithm to implement pedestrian tracking, obtains the pedestrian detection frame, predicts the current position through the Kalman filter, obtains the predicted frame, and then calculates the matching degree between the tracking targets of the previous frame and the current frame through similarity calculation, performs data association through the Hungarian algorithm, and sets a separate ID for each tracking target; The intrusion detection module: sets the monitoring area, uses the Deep Sort algorithm to track the target according to the detection frame and confidence of the pedestrian, draws the monitoring area and the detection frame in each frame, records the motion data at the same time, determines whether an intrusion occurs by detecting whether the target center point is within the monitoring area, and records the intrusion event; The pedestrian gathering analysis module reads previously saved motion data and video data, identifies stationary points by calculating the motion trajectory of each target, and considers that a target is stationary when it remains stationary within a fixed distance for more than a set time, i.e., it is a stationary point. The motion trajectory is drawn using lines of different colors, and color gradient is achieved by decreasing color values. For stationary points, a heat map is generated by drawing circles layer by layer. The size and color of the circles are adjusted according to the stationary time, and the darker the color, the longer the stay time.
3. According to claim 2, a ZYNQ-based campus patrol drone perception system is characterized in that: The pedestrian detection model VarKDnet is specifically: 1) It has a three-branch structure, including a main branch composed of variable kernel convolution, feature extraction branches of different scales, and an identity mapping branch that maintains network performance; when the kernel size of the main branch is 3, the kernel size of the feature extraction branch is 1x1, which is mainly used to pre-extract more local information; when the kernel size of the main branch is greater than 3x3, the feature extraction branch is set to DW Conv3x3; in the process of channel number expansion, the convolution kernel size of the trunk branch is dynamically adjusted; 2) Through the structural reparameterization technology, the multi-branch training is converted into a single-channel structure. First, the batch normalization layer and the convolution layer are fused. The convolution expression without bias is shown in formula (1), the calculation method of the batch normalization layer is shown in formula (2), and the result of batch normalization after the convolution layer is shown in formula (3), which is regarded as a convolution calculation with bias. Through mathematical equivalence, the convolution layer and the batch normalization layer are fused; secondly, the 3x3 convolution branch and the identity mapping branch are converted into a special 7X7 structure; finally, according to the additivity of convolution, the three branches are fused into a single-channel structure; Conx(x)=Wx(1) Among them, x is the input data; W is the convolution kernel weight; μ is the mean; σ is the standard deviation; γ is the scaling factor; β is the offset parameter; 3) Using the PRelu activation function, softmax classification and position regression are performed simultaneously on multiple feature maps to obtain the pedestrian confidence and its position coordinates.
4. According to the ZYNQ-based campus patrol drone perception system of claim 1, it is characterized in that: The convolutional neural network hardware accelerator is used to implement parallel computing of the human detection algorithm VarKDnet, specifically: 1) Accelerate convolution calculation through pipeline operation. When performing the first row of convolution calculation, in the first clock cycle, execute X(n,0)*W00; in the second clock cycle, execute X(n,1)*W01 and add the calculation result of X(n,0)*W00. In addition, the first-level pipeline will also calculate X(n,1)*W00, which is the first calculation value of the next convolution window; in the third clock cycle, the third-level pipeline executes X(n,2)*W02+X(n,1)*W01+X(n,0)*W00, the first-level pipeline executes X(n,2)*W00, and the second-level pipeline executes X(n,1)*W00+X(n,2)*W01; and so on to accelerate subsequent convolution calculations; 2) Using the weight sharing mechanism of convolutional neural networks, we design convolution kernel reuse, where the same convolution kernel simultaneously performs convolution operations on multiple windows of an image to achieve parallel computing; 3) A row buffer mechanism is used to further improve the degree of parallelism. The format of the loaded image data is 512*512, and the row buffer size is set to 3×512.
5. A ZYNQ-based campus patrol drone perception system according to any one of claims 1-4, characterized in that: The ZYNQ airborne platform adopts FPGA+ARM architecture.
6. A campus patrol drone perception method based on ZYNQ, characterized in that: The following steps are involved: Step 1: deploy the trained pedestrian detection model VarKDnet and Deep Sort algorithm to the ZYNQ airborne platform, connect the ZYNQ airborne platform to the flight control platform through the serial port, start the power supply, power the ZYNQ airborne platform and the flight control platform, and take off the drone to perform the campus patrol mission; Step 2: During the flight, the PL side of the ZYNQ airborne platform collects the video stream of the crowd in the campus in real time through the camera, and transmits the video stream to the PS side for further analysis and processing; Step 3: After receiving the video stream, the PS end scales and normalizes the video stream to 512*512*3, loads the pedestrian detection model VarKDNet to perform pedestrian detection, and outputs the pedestrian's confidence and its position coordinates; Step 4: The PS uses the Deepsort algorithm to track the video stream after pedestrian detection, estimates it through Kalman filtering, associates the pedestrian in the previous frame with the pedestrian in the current frame, performs matching correction through the Hungarian algorithm, and sets a separate ID for each tracking target; Step 5: After obtaining the results of pedestrian detection and trajectory tracking, intrusion detection and aggregation analysis are implemented by combining the built-in functions of OpenCV; Step 6: The ZYNQ airborne platform transmits the processed video to the remote monitoring center through data transmission via the PS terminal for further analysis and decision making.
