Security video monitoring management method and system for smart campus

Abstract

The invention discloses a security and protection video monitoring management method and system for a smart campus, and relates to the technical field of smart security and protection, and the method comprises the steps: deploying a multi-mode sensing device, carrying out the data collection and preprocessing, generating a data set, generating a feature set through the data set, calculating an abnormal correlation score based on the feature set, generating an abnormal feature subset, and carrying out the recognition of the abnormal feature subset. Using the abnormal feature subset to output a fusion feature vector through a GAT model, and calculating a risk value based on the fusion feature vector; calculating a risk field change rate based on the risk value, calculating a risk probability by using the change rate, updating the risk value through a graph convolutional network, and performing equipment priority division based on the risk probability and the updated risk value; z-score anomaly detection is generated based on the risk probability, and an anomaly region is obtained based on a detection result; and performing hierarchical intervention on the detected abnormal region and the divided device priority. The real-time performance and the self-adaptive capability of the security and protection system in the aspects of risk assessment and intelligent operation and maintenance are improved.

Description

Technical Field

[0001] This invention relates to the field of intelligent security technology, and in particular to a security video surveillance management method and system for smart campuses. Background Technology

[0002] With the continuous advancement of intelligent technologies, the construction of smart campuses has gradually incorporated technologies such as artificial intelligence, big data analytics, and the Internet of Things. This trend has not only improved campus management efficiency but also significantly enhanced the level of intelligent campus security management. Traditional campus security systems mainly rely on static monitoring equipment and simple alarm mechanisms, typically consisting of cameras, access control systems, and alarms. However, with the expansion of campus scale and the diversification of security risks, the limitations of traditional monitoring methods have become increasingly apparent, especially in terms of real-time detection, analysis, and response speed to abnormal events. To overcome this deficiency, intelligent security systems have emerged, employing multimodal sensing technologies (such as video surveillance, sound detection, and environmental sensing). While existing intelligent monitoring technologies have made significant progress, several technical bottlenecks remain. First, most existing security monitoring systems rely solely on a single data source, such as video surveillance or sensor data, making it difficult to comprehensively and accurately assess various risks in the campus environment. Second, existing risk assessment and response mechanisms are mostly based on traditional rules or simple statistical methods, lacking real-time adaptability to complex and dynamic environments. This weakens the system's ability to identify and predict anomalies, failing to meet the high security requirements of smart campuses. Furthermore, many existing systems are inefficient when dealing with large-scale data, unable to process and analyze massive amounts of data from multiple sensors in a timely manner, leading to system lag and resource waste. Summary of the Invention

[0003] In view of the aforementioned existing problems, the present invention is proposed.

[0004] Therefore, this invention provides a security video surveillance management method and system for smart campuses, which solves the problems of insufficient multimodal data fusion, inaccurate risk assessment, and lack of dynamic optimization of equipment scheduling and intervention strategies in the prior art.

[0005] To solve the above-mentioned technical problems, the present invention provides the following technical solution: In a first aspect, the present invention provides a security video surveillance management method for smart campuses, comprising: deploying multimodal sensing devices and performing data acquisition and preprocessing to generate a dataset; generating a feature set using the dataset; calculating anomaly correlation scores based on the feature set to generate anomaly feature subsets; outputting a fused feature vector using the anomaly feature subsets through a GAT model; and calculating a risk value based on the fused feature vectors. The risk field change rate is calculated based on the risk value, the risk probability is calculated using the change rate, the risk value is updated through a graph convolutional network, and the device priority is determined based on the risk probability and the updated risk value. Z-score anomaly detection is generated based on risk probability, and the abnormal region is obtained based on the detection result; Implement tiered interventions for detected abnormal areas and prioritized equipment, calculate the predicted effects based on the tiered interventions, and adjust the intervention strategies based on the predicted effects; Visual reports are generated based on the prediction results and anomalies, and operation and maintenance optimizations are performed.

[0006] As a preferred embodiment of the security video surveillance management method for smart campuses described in this invention, the step of deploying multimodal sensing devices and performing data acquisition and preprocessing to generate a dataset refers to deploying fixed high-definition cameras, microphones, environmental sensors, PIR sensors, DFR0076 flame sensors, and RFID access control card readers in key areas of the campus, acquiring data through multimodal sensing devices, performing data preprocessing and labeling, and generating a labeled dataset based on the processed data.

[0007] As a preferred embodiment of the security video surveillance management method for smart campuses described in this invention, the following steps are taken: generating a feature set using a dataset; calculating anomaly correlation scores based on the feature set to generate an anomaly feature subset; outputting a fused feature vector using the anomaly feature subset through a GAT model; and calculating a risk value index based on the fused feature vector using the dataset. The YOLOv8 object detection model is trained using the Ultralytics framework. After training, the model outputs the center coordinates (x, y) and area S of each detected object, as well as the corresponding category. Based on the flame image, the Haar cascade classifier is trained using OpenCV. After training, the trained cascade model file is output. Extracting numerical features from multimodal data , ,..., each Corresponding to a quantifiable risk indicator, a structured feature set is generated from i numerical features extracted from multimodal data. For each feature Calculate eigenvalue frequencies Calculate real-time features Deviation from normal pattern Calculate features and The absolute value of the correlation coefficient between them For each feature Calculate the entropy value and entropy weight And calculate the abnormal correlation score. ; Construct a spatial graph of the entity region and calculate attention weights. By fusing multimodal anomaly features through the GAT model, node i is generated. Layer characteristics Combine spatial neighborhood relationships to output fused features The risk field is generated using Transformer, and the node risk value R(x,y,t) at time t is calculated.

[0008] As a preferred embodiment of the security video surveillance management method for smart campuses described in this invention, the steps include: calculating the risk field change rate based on the risk value, calculating the risk probability using the change rate through logistic regression, updating the risk value through a graph convolutional network, and prioritizing devices based on the risk probability and the updated risk value. This involves using the detection results of the YOLOv8 model combined with the flame detection results of a Haar cascade classifier to generate fused features. Calculate at position and time Risk increment at time Based on the risk value R(x,y,t), partial differential equations (PDEs) are used to calculate the rate of increase / decrease of the risk value over time. The risk probability is calculated using logistic regression combined with the Sigmoid function. Updated features are generated through a two-layer graph convolutional network. ; The feature vector updated by the two-layer graph convolutional network is input into the linear decoder to obtain the risk prediction value at the next time step. Equipment priority scoring is calculated based on risk probability and risk prediction value. It generates resource allocation schemes by sending control commands via the MQTT protocol.

[0009] As a preferred embodiment of the security video surveillance management method for smart campuses described in this invention, wherein: the Z-score anomaly detection is generated based on risk probability, and the anomaly region is obtained based on the detection result, and the feature mean is calculated based on the risk probability. and variance Z-score detection is used to generate outliers. ,like Preset threshold This is marked as an abnormal region A.

[0010] As a preferred embodiment of the security video surveillance management method for smart campuses described in this invention, the step of implementing tiered intervention for detected abnormal areas and the assigned device priorities, calculating the prediction effect based on the tiered intervention, and adjusting the intervention strategy based on the prediction effect refers to triggering multi-level intervention according to abnormal area A and the assigned device priorities. and In case of emergency, an emergency alert is pushed to the security platform via MQTT, the sound and light alarm is activated on site, and the camera zooms in to track, records, and uploads the encrypted video. like and The system plays a broadcast warning, "Please note that an anomaly has occurred in the vicinity. Please remain vigilant," reminding personnel to pay attention to safety. It also increases the brightness of lights to improve visibility on site, raises the sampling rate of the PIR / flame sensor to 1Hz, and enters "standby" mode. like or Preset threshold The camera reduces its bit rate and resolution, uses intelligent encoding, and pauses data transmission in non-critical areas to free up network resources. Collect risk probability after multi-level intervention Calculate the prediction effect ,like <Preset threshold B, adjust intervention strategy.

[0011] As a preferred embodiment of the security video surveillance management method for smart campuses described in this invention, the step of generating a visual report based on prediction results and anomalies, and performing operation and maintenance optimization, refers to displaying the risk field on a GIS map, highlighting abnormal areas, generating a visual report, and identifying anomalies. Preset threshold This triggers a web interface alarm and highlights the area of ​​the camera data stream that is being transmitted first, with weekly statistical feature averages. and variance And combined with the prediction effect Refresh the preset threshold Q and the abnormal threshold. Monitor camera bandwidth and ESP32 heartbeat. If bandwidth is congested or the device is offline, generate a device health report and trigger a maintenance notification.

[0012] Secondly, the present invention provides a security video surveillance management system for smart campuses, including a data processing and feature extraction module for generating a structured dataset based on collected multimodal data through feature extraction methods; The anomaly detection and risk assessment module is used to calculate anomaly correlation scores and feature fusion based on graph neural networks, generate a subset of anomaly features, and calculate the risk value of each area of ​​the campus using these features. The tiered intervention and equipment priority classification module is used to calculate the risk probability and the updated risk value, and to classify equipment priorities. The anomaly detection and prediction effect adjustment module is used to use the Z-score anomaly detection method to calculate the feature mean and variance based on the risk probability, and further identify anomaly areas. The visualization and report generation module is used to generate visual reports on the risk prediction results and abnormal area information.

[0013] Thirdly, the present invention provides a computer device, including a memory and a processor, wherein the memory stores a computer program, wherein: when the computer program is executed by the processor, it implements any step of the security video surveillance management method for smart campuses as described in the first aspect of the present invention.

[0014] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, it implements any step of the security video surveillance management method for smart campuses as described in the first aspect of the present invention.

[0015] The beneficial effects of this invention are as follows: By constructing a multimodal feature set and combining the detection results of YOLOv8 and Haar cascade classifiers, deep fusion of multi-source information is achieved, effectively overcoming the problem of insufficient monitoring accuracy from a single data source in existing technologies; by introducing anomaly correlation scores and entropy weight calculation mechanisms, key features that significantly impact risk assessment can be dynamically identified, improving the accuracy and robustness of anomaly detection; by using the GAT model and Transformer to generate a risk field and combining it with PDE to achieve dynamic updates of risk values, overcoming the shortcomings of existing technologies in capturing spatiotemporal dependencies and risk evolution patterns; at the resource scheduling level, by calculating device priorities based on risk probability and updated risk values, differentiated configuration of monitoring resources is achieved; and by using hierarchical intervention and visualization output, the system's proactive early warning and rapid response capabilities are enhanced, improving the intelligence level of the smart campus security system. Attached Figure Description

[0016] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1This is a flowchart of a security video surveillance management method for smart campuses, as shown in Example 1.

[0018] Figure 2 This is a schematic diagram of a security video surveillance management system for a smart campus, as shown in Example 1.

[0019] Figure 3 This is a flowchart of the data processing and risk assessment process in Example 1. Detailed Implementation

[0020] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0021] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0022] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0023] Example 1, referring to Figures 1 to 3 This is the first embodiment of the present invention, which provides a security video surveillance management method for smart campuses, including the following steps: S1. Deploy multimodal sensing devices, collect and preprocess data to generate datasets, use the datasets to generate feature sets, calculate anomaly correlation scores based on feature sets to generate anomaly feature subsets, use the anomaly feature subsets to output fused feature vectors through the GAT model, and calculate risk values ​​based on the fused feature vectors. Specifically, deploying multimodal sensing devices and performing data acquisition and preprocessing to generate datasets involves deploying fixed high-definition cameras, microphones, environmental sensors, PIR sensors (M5StickC HAT, detection range 5m), DFR0076 flame sensors (760-1100nm, 60° probe angle) and RFID access control card readers in key areas of the campus (classrooms, canteens, corridors, and school gates). Data is collected through multimodal sensing devices, and data preprocessing and labeling are performed, specifically including: The video was downsampled to 5fps (FFmpeg), pixel values ​​were normalized to [0,1], and the LabelImg tool was used to annotate bounding boxes, scene, coverage class, and flame image. The area of ​​the detection box was then calculated. The size of the target in the image is represented by [T, 128], the audio spectral features are extracted using STFT (window 25ms, step size 10ms), the sensor data is normalized using Z-score (mean 0, standard deviation 1), the access control data is converted into a time series ([T, 1]), the historical data is vectorized using Word2Vec (128-dimensional, Gensim, trained for 100 epochs), and a labeled dataset is generated based on the processed data. .

[0024] The data acquisition includes real-time video streams (720p, 30fps, latency <200ms) acquired by cameras via the RTSP protocol, ambient audio (WAV format) acquired by microphones, and temperature, humidity, light intensity, PIR events (GPIO high level 2s), DFR0076 voltage (0-5V), and access control card swipe records (timestamp, personnel ID) acquired by sensors. Historical data is extracted from the campus security database to record events (fights, fires, etc., in JSON format, including time, location, and category). The coverage category class includes {person, knife, bottle, cup, mobile phone, fire}; The labeled scenarios include classrooms, cafeterias, corridors, and staircases, covering daytime / nighttime lighting (0-1000 lux). The flame images include visual samples used for training and detecting fire risks, collected and labeled from campus security databases and real-time videos; The rectangular frame includes x and y as the coordinates of the center point of the detection frame, and w and h as the width and height of the detection frame.

[0025] By fusing multimodal data, the system not only enhances its ability to perceive complex scenes but also improves the accuracy and robustness of anomaly detection. Video data is downsampled and labeled with bounding boxes to ensure computational efficiency and accuracy. Audio data is preprocessed more precisely through spectral feature extraction and sensor data standardization, effectively reducing the interference of environmental noise on monitoring. Historical data is vectorized using Word2Vec to capture the semantic features of historical events, providing in-depth support for the prediction of future events and risk analysis. Through this multi-layered and refined data processing approach, the system can more accurately identify and predict safety hazards on campus, especially in the early warning of fires and emergencies. This overcomes the problem of traditional monitoring systems lacking comprehensive judgment and rapid response capabilities, thereby significantly improving the intelligence level and practicality of the campus security system.

[0026] Furthermore, a feature set is generated using the dataset, and an anomaly correlation score is calculated based on the feature set to generate an anomaly feature subset. This anomaly feature subset is then used to output a fused feature vector through a GAT model. Finally, a risk value index is calculated based on the fused feature vector, using the dataset... A YOLOv8 object detection model was trained using the Ultralytics framework. The model structure includes: a backbone (composed of Conv, C2f, and SPPF modules) for feature extraction; a neck (FPN+PAN structure) for multi-scale feature fusion; and a decoupled detection head that uses BCE loss for classification and DFL and CIoU loss for bounding box regression. Training was performed on an NVIDIA Jetson Nano (4GB RAM) platform with a batch size of 16, an initial learning rate of 0.001, and the Adam optimizer. The model was trained for 100 epochs. After training, the model outputs the center coordinates (x, y), width w, height h, area S, and corresponding class for each detected object. The training process for a Haar cascade classifier based on flame images using OpenCV includes: extracting Haar features, constructing an integral image to accelerate computation, selecting strong features from 2000 candidate features using the AdaBoost algorithm, and constructing a 20-level cascade classifier. The entire training process is run on a PC equipped with 16GB of RAM, and the trained cascade model file is output for flame detection in real-time video. Extracting numerical features from multimodal data , ,..., Specific operations include: extracting "class" and "target bounding box area" values ​​from the YOLOv8 detection results; obtaining "flame detection confidence" from the Haar cascade output; calculating the average "sound energy" from the audio spectrum; and statistically analyzing the time-series indicators "PIR trigger frequency" and "temperature change rate" from the sensor data. Corresponding to a quantifiable risk indicator (such as A value of 0.92 indicates a high tooling risk. =85 dB represents high sound intensity), and a structured feature set is generated from i numerical features extracted from multimodal data. ; For each feature ∈F, calculate the frequency of the eigenvalue m (e.g., 0.92). , represents the normal mode, the formula is: , in, These are feature values ​​from historical data (such as tool confidence 0.2, PIR frequency 0.05). For indicator functions, This represents the total number of feature values ​​in the historical data. Calculate real-time features Deviation from normal pattern The formula is: , For a feature set F, calculate the Pearson correlation coefficient between the features using the following formula: , in, Features and The absolute value of the correlation coefficient between two features, ranging from 0 to 1, indicates the degree of linear correlation between the two features. For feature covariance (using DataFrame.cov in the Pandas library) (obtained by method) The feature standard deviation is calculated using the Pandas std function. For each feature Calculate the entropy value The formula is: , in, For each feature The probability distribution of the values ​​in the dataset is obtained using the Pandas hist function. It is the natural logarithm function, that is, the logarithm with the mathematical constant e (approximately equal to 2.71828) as its base; Calculate entropy weights The formula is: , in, The feature vector of node k is obtained by collecting data from the multimodal device bound to this node (using the actual device coordinates as the node). Calculation features Abnormal correlation score The formula is: , in, Features variance =0.5 indicates the adjustment parameter; Constructing a spatial map of the physical area involves the following steps: using the coordinates of actual devices (cameras, microphones, sensors, access control points, etc.) as node k, and establishing edges based on the k-nearest neighbors of geographical proximity (geometric distance along roads / pedestrian paths). Each node is directly bound to the multimodal feature vector collected by the device. Detections within the node's coverage area (such as a knife detected by a camera) are assigned to that node. The attention weight assigned to neighbor node j by node i when aggregating neighbor features at layer l is calculated. The formula is: , in, and Let i be the feature vector of node i and node j in the l-th layer. Let be the learnable weight matrix of the l-th layer, obtained through backpropagation in a graph neural network (GNN). Edge features include calculating the actual geometric distance between nodes i and j, determined using Dijkstra's algorithm (shortest path), and edge features generated through image processing (such as convolutional neural networks) or audio analysis (such as MFCC features). The learnable attention vector for layer l is obtained through an adaptive learning method and is used to map the concatenated features to a scalar score. For activation function, For normalization operations, T represents transpose; By fusing multimodal anomaly features (video, audio, sensor data) through the GAT (Graph Attention Network) model, node i is generated. Layer characteristics Combining spatial neighborhood relationships (physical region spatial map), a 2-layer GAT is run to output fused features. The formula is: , in, For the GAT model The linear transformation weight matrix of the layer, For GAT number The layer's bias vector, It is the ReLU activation function. It is used for nonlinear transformations to enhance the expressive power of the model; The Transformer is used to generate a risk field (referring to the continuous spatiotemporal distribution of risk levels at various locations on campus over time, specifically a three-dimensional function R(x,y,t)). The risk value R(x,y,t) of node (x,y) at time t is calculated ([0,1], where 0 represents no risk and 1 represents high risk). The formula is as follows: , in, To leverage historical data features, a Graph Attention Network (GAT) is used to model and aggregate the historical risk values ​​of each node on campus, capturing spatial dependencies and temporal evolution patterns to generate a feature representation for each node. The Transformer weights are obtained by training the model for 100 epochs using the PyTorch framework, setting the Adam optimizer.

[0027] By utilizing multimodal inputs such as the YOLOv8 object detection model, Haar cascade classifier, audio spectrum analysis, and sensor data, this system can comprehensively capture abnormal situations on campus, such as fires and violent acts. Based on this information, it performs deep fusion using a graph neural network (GAT) to accurately extract potential correlations between features and generate more accurate risk values. Unlike the inefficient analysis of traditional methods, this solution captures spatiotemporal evolution patterns through a Transformer model, thereby achieving dynamic risk assessment of different areas and time periods on campus. This risk prediction based on multi-level data processing and deep learning not only improves the real-time performance and accuracy of event detection but also optimizes the response speed and resource utilization efficiency of the entire security system by intelligently adjusting device priorities and network resources, thus achieving more comprehensive and intelligent security protection in campus safety management.

[0028] S2. Calculate the risk field change rate based on the risk value, use the change rate to calculate the risk probability, update the risk value through a graph convolutional network, and classify equipment priorities based on the risk probability and the updated risk value. Z-score anomaly detection is generated based on risk probability, and the abnormal region is obtained based on the detection result; Specifically, based on the risk value, the rate of change of the risk field is calculated. The rate of change is then used to calculate the risk probability using logistic regression. The risk value is updated using a graph convolutional network. Equipment priority is then determined based on the risk probability and the updated risk value. Finally, the detection results of the YOLOv8 model are combined with the flame detection results of a Haar cascade classifier to generate fused features. ); Calculate at position ) and time Risk increment at time The formula reflects the impact of local risk characteristics on the overall risk field: , in, This is the weight matrix of the neural network. The bias term for the neural network is obtained by training it in PyTorch for 50 epochs using a campus dataset. Based on the risk value R(x,y,t), partial differential equations (PDEs) are used to simulate the rate of change of the risk field over time, i.e., the rate at which the risk value increases / decreases over time. The formula is: , in, The diffusion term (calculated based on 8-neighborhood difference (implemented in NumPy)) is represented by D = 0.1, which is the diffusion coefficient (obtained through grid search optimization). The symbol for partial derivative indicates the operation of partial derivatives with respect to a multivariable function; Calculate the risk probability using logistic regression combined with the sigmoid function. The formula is: , in, This is the Sigmoid function (PyTorch). These are the logistic regression coefficients fitted using historical alarm data. The combined anomaly detection value (e.g., weighted average) obtained from the detection scores of the YOLOv8 model and the Haar feature classifier. The features are updated using a two-layer graph convolutional network (GCN), with the following formula: , in, For node i at the th The updated feature vector after the layer, This represents the feature vector of node j in the (k-1)th layer; The feature vector updated by the two-layer graph convolutional network is input into the linear decoder to obtain the risk prediction value at the next time step. The formula is: , in, It is the transpose of w, and can be used with the eigenvector. Performing a dot product yields a scalar value; Equipment priority scoring is calculated based on risk probability and risk prediction value. The formula is: , in, Let c be the set of nodes covered by device c. For the weight of the prediction item, For the spatial and directional weighting coefficients of device c on node i, device c is a logical sensing unit with a camera as its core, and integrates peripheral sensors (such as microphone, PIR, flame or RFID, etc.) as needed according to the actual deployment location. Send control instructions through the MQTT protocol (Paho-MQTT) to optimize resources and generate a resource allocation plan, including if > the preset threshold Q, the device is of high priority, and the data stream is preferentially transmitted, and more bandwidth is allocated (through the QoS protocol and network priority setting) to ensure the real-time transmission of high-definition video streams. The sensor sampling frequency is increased to 1 Hz to ensure the high-frequency acquisition of information such as environmental parameters, human activities, and flame detection. The network priority is set to the highest (DSCP = EF) to ensure the low-latency transmission of data; If the preset threshold V ≤ ≤ Q, the device is of medium priority, maintaining normal data transmission and routine monitoring. The video stream resolution and frame rate are moderate, the sensor sampling frequency remains 0.5 Hz, and the network priority is set to medium to ensure the reliable transmission of data; If < V, the device is of low priority, reducing the data transmission priority, reducing bandwidth occupancy, with a lower video stream resolution and frame rate. The sensor sampling frequency is reduced to 0.1 Hz, and the network priority is set to the lowest (DSCP = BE), allowing data to be delayed in case of network congestion.

[0029] By calculating the change rate of the risk field and using the Graph Convolutional Network (GCN) to dynamically update the risk value, the problems of static evaluation and slow response in the existing technology are solved; by combining multi-modal inputs such as YOLOv8 object detection results, Haar cascade flame detection, audio data, and sensor data, fused features are generated and the risk value is calculated in real time, which can dynamically simulate the change of the risk field over time, accurately capture the impact of local risks on the overall scene, use logistic regression to calculate the risk probability and the graph convolutional network to update the risk value, further enhancing the model's prediction ability in the spatio-temporal dimension; through the device priority division based on risk probability, resource allocation is optimized to ensure the provision of high-priority data streams and low-latency transmission at critical moments, greatly improving the real-time performance and accuracy of the system, thus significantly enhancing the response speed and resource utilization efficiency of security management.

[0030] Furthermore, generate Z-score anomaly detection based on risk probability, and obtain the anomaly region based on the detection result, which means calculating the feature mean and variance , the formula is: , , where N is the number of all nodes; Use Z-score to detect anomaly points , the formula is: , like Preset threshold It is marked as an abnormal region A (high-risk anomaly).

[0031] By calculating the mean and variance of node features and combining them with the Z-score method to detect anomalies, abnormal areas can be accurately identified, thus providing a more refined risk assessment. The advantage of Z-score anomaly detection is that it takes into account the risk distribution and fluctuations between nodes, and can automatically adjust the detection standard in complex scenarios. It avoids the misjudgment problem caused by manually setting thresholds in traditional methods, significantly improves sensitivity and accuracy in dynamic environments, and can detect potential security risks in real time, optimizing the allocation of security resources and response efficiency.

[0032] S3. Implement graded intervention for detected abnormal areas and classified equipment priorities, calculate the prediction effect based on the graded intervention, and adjust the intervention strategy based on the prediction effect; Based on the prediction results and anomalies, generate visual reports and perform operation and maintenance optimization; Specifically, tiered interventions are implemented for detected abnormal areas and prioritized equipment. Based on these tiered interventions, predictive effects are calculated, and intervention strategies are adjusted accordingly. This involves triggering multi-level interventions based on abnormal area A and the prioritized equipment to optimize security response efficiency. and In case of emergency, an emergency alarm is pushed to the security platform via MQTT (APP / large screen flashing), the sound and light alarm is activated on site (such as buzzer + LED flashing), the camera zooms to track, and the video is encrypted and uploaded; like and (Indicates the position of device c) The distance to the boundary of abnormal region A is less than a preset threshold. This means that the device is located within the influence range of the abnormal area (adjacent area) and is also affected by the risk spread, so an early warning or response preparation needs to be initiated. Play a broadcast prompt: "Please note that an anomaly has occurred in the nearby area. Please remain vigilant." Remind personnel to pay attention to safety, appropriately increase the brightness of the lights (PWM control, increase by 10%) to improve on-site visibility, increase the sampling rate of the PIR / flame sensor to 1Hz, and enter the "standby" state. like or Preset threshold (Indicates the location of device c) If the camera is not in abnormal area A, reduce the bit rate and resolution, use smart encoding (only uploads to the moving area), suspend data stream transmission in non-critical areas, and release network resources. Collect risk probability after multi-level intervention ( (risk probability after minutes), calculate prediction effect The formula is: , like <Preset threshold B, adjust intervention strategies (such as increasing alarm frequency or changing prompt sounds).

[0033] Through a multi-layered intervention mechanism, high-intensity intervention in emergency areas, early warning and standby response in neighboring areas, and resource degradation and network optimization in non-critical areas are achieved, thereby realizing dynamic resource scheduling and differentiated management on a global scale. The solution introduces the calculation of predictive effects and automatically adjusts the intervention strategy according to the changes in the probability of risk caused by the intervention measures, avoiding the problems of "over-response" or "under-response" in traditional methods. This dynamic, layered, and feedback-optimizable intervention mode significantly improves the real-time performance, precision, and adaptability of the security system, making campus security more efficient, intelligent, and reliable.

[0034] Furthermore, based on the prediction results and anomalies, visualized reports are generated, and operational optimization is performed. This includes displaying the risk field (heatmap, with high-risk areas flashing red, implemented using Plotly) on a GIS map, highlighting anomaly areas (marked in yellow), and generating visualized reports, including: line charts showing the change in risk probability over time, and bar charts showing the prediction results. The distribution and heatmap show the priority distribution of camera data streams (high priority areas are green, medium priority areas are blue, and low priority areas are gray). If an anomaly Preset threshold This triggers a web interface alarm (Flask framework) and highlights the area of ​​the camera data stream that is being transmitted first, with weekly statistical feature averages. and variance And combined with the prediction effect Refresh the preset threshold Q and the abnormal threshold. Monitor camera bandwidth (QoS protocol traffic statistics, FFmpeg detection), ESP32 heartbeat (MQTT subscription), and generate a device health report (including bandwidth allocation status) if bandwidth is congested or the device is offline, triggering a maintenance notification.

[0035] By utilizing GIS maps and Plotly visualization techniques, the risk field is visually presented as a heatmap, highlighting abnormal areas in real time. Multi-dimensional reports, including line charts and bar charts, dynamically display changes in risk probability and the distribution of predicted effects, enabling traceable and trend-based analysis of the risk situation. Simultaneously, by monitoring the number of abnormal points and the distribution of device priorities, combined with QoS bandwidth statistics and MQTT heartbeat detection, device health reports can be automatically generated and maintenance notifications triggered, effectively avoiding monitoring blind spots caused by bandwidth congestion or device failure. Thresholds are automatically refreshed weekly based on feature mean, variance, and predicted effects, achieving adaptive threshold optimization and solving the false alarm and missed alarm problems caused by fixed thresholds in traditional systems. This significantly improves the visualization management capabilities, operational efficiency, and intelligence level of the security system, providing comprehensive, dynamic, and sustainable technical support for campus security.

[0036] This embodiment also provides a security video surveillance management system for smart campuses, including: a data processing and feature extraction module, used to generate a structured dataset based on collected multimodal data through feature extraction methods; The anomaly detection and risk assessment module is used to calculate anomaly correlation scores and feature fusion based on graph neural networks, generate a subset of anomaly features, and calculate the risk value of each area of ​​the campus using these features. The tiered intervention and equipment priority classification module is used to calculate the risk probability and the updated risk value, and to classify equipment priorities. The anomaly detection and prediction effect adjustment module is used to use the Z-score anomaly detection method to calculate the feature mean and variance based on the risk probability, and further identify anomaly regions. The visualization and report generation module is used to generate visual reports on the effectiveness of risk prediction and information on abnormal areas.

[0037] This embodiment also provides a computer device applicable to a security video surveillance management method for a smart campus, comprising: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the security video surveillance management method for a smart campus as proposed in the above embodiment.

[0038] The computer device can be a terminal, comprising a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, carrier networks, NFC (Near Field Communication), or other technologies. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0039] This embodiment also provides a storage medium storing a computer program. When executed by a processor, the program implements a security video surveillance management method and system for smart campuses as proposed in the above embodiments. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Red-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0040] In summary, this invention achieves deep fusion of multi-source information by constructing a multimodal feature set and combining the detection results of YOLOv8 and Haar cascade classifiers, effectively overcoming the problem of insufficient monitoring accuracy of single data sources in existing technologies. By introducing anomaly correlation scores and entropy weight calculation mechanisms, it can dynamically identify key features that have a significant impact on risk assessment, improving the accuracy and robustness of anomaly detection. By using the GAT model and Transformer to generate a risk field and combining it with PDE to achieve dynamic updates of risk values, it overcomes the shortcomings of existing technologies in capturing spatiotemporal dependencies and risk evolution patterns. At the resource scheduling level, by calculating device priorities based on risk probability and updated risk values, it achieves differentiated configuration of monitoring resources. Through hierarchical intervention and visualization output, it enhances the system's proactive early warning and rapid response capabilities, improving the intelligence level of the smart campus security system.

[0041] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A security video surveillance management method for smart campuses, characterized in that: include, Deploy multimodal sensing devices, collect and preprocess data to generate datasets, use the datasets to generate feature sets, calculate anomaly correlation scores based on the feature sets to generate anomaly feature subsets, use the anomaly feature subsets to output fused feature vectors through the GAT model, and calculate risk values ​​based on the fused feature vectors. The risk field change rate is calculated based on the risk value, the risk probability is calculated using the change rate, the risk value is updated through a graph convolutional network, and the device priority is determined based on the risk probability and the updated risk value. Z-score anomaly detection is generated based on risk probability, and the abnormal region is obtained based on the detection result; Implement tiered interventions for detected abnormal areas and prioritized equipment, calculate the predicted effects based on the tiered interventions, and adjust the intervention strategies based on the predicted effects; Visual reports are generated based on the prediction results and anomalies, and operation and maintenance optimizations are performed.

2. The security video surveillance management method for smart campuses as described in claim 1, characterized in that: The process involves generating a feature set using a dataset, calculating anomaly correlation scores based on the feature set to generate anomaly feature subsets, using the anomaly feature subsets to output a fused feature vector through a GAT model, and calculating a risk value index based on the fused feature vector. The YOLOv8 object detection model is trained using the Ultralytics framework. After training, the model outputs the center coordinates (x, y) and area S of each detected object, as well as the corresponding category. Based on the flame image, the Haar cascade classifier is trained using OpenCV. After training, the trained cascade model file is output. Extracting numerical features from multimodal data , ,..., Each Corresponding to a quantifiable risk indicator, a structured feature set is generated from i numerical features extracted from multimodal data. For each feature Calculate eigenvalue frequencies Calculate real-time features Deviation from normal pattern Calculate features and The absolute value of the correlation coefficient between them For each feature Calculate the entropy value and entropy weight And calculate the abnormal correlation score. ; Construct a spatial graph of the entity region and calculate attention weights. By fusing multimodal anomaly features through the GAT model, node i is generated. Features of the layer Combine spatial neighborhood relationships to output fused features The risk field is generated using Transformer, and the node risk value R(x,y,t) at time t is calculated.

3. The security video surveillance management method for smart campuses as described in claim 2, characterized in that: The process involves calculating the risk field change rate based on risk values, using the change rate to calculate the risk probability via logistic regression, updating the risk values ​​using a graph convolutional network, and prioritizing equipment based on the risk probability and updated risk values. This process utilizes the detection results of the YOLOv8 model combined with the flame detection results of a Haar cascade classifier to generate fused features. Calculate at position and time Risk increment at time Based on the risk value R(x,y,t), partial differential equations (PDEs) are used to calculate the rate of increase / decrease of the risk value over time. The risk probability is calculated using logistic regression combined with the Sigmoid function. Updated features are generated through a two-layer graph convolutional network. ; The feature vector updated by the two-layer graph convolutional network is input into the linear decoder to obtain the risk prediction value at the next time step. Equipment priority scoring is calculated based on risk probability and risk prediction value. It generates resource allocation schemes by sending control commands via the MQTT protocol.

4. The security video surveillance management method for smart campuses as described in claim 3, characterized in that: The Z-score anomaly detection based on risk probability is used to obtain anomaly region indices based on the detection results and to calculate the feature mean based on risk probability. and variance Z-score detection is used to generate outliers. ,like Preset threshold This is marked as an abnormal region A.

5. A security video surveillance management method for smart campuses as described in claim 4, characterized in that: The process of implementing tiered intervention based on detected abnormal regions and assigned device priorities, calculating prediction effects based on these tiered interventions, and adjusting intervention strategies according to these prediction effects refers to triggering multi-level interventions based on abnormal region A and assigned device priorities. and In case of emergency, an emergency alert is pushed to the security platform via MQTT, the sound and light alarm is activated on site, and the camera zooms in to track, records, and uploads the encrypted video. like and The system plays a broadcast warning, "Please note that an anomaly has occurred in the vicinity. Please remain vigilant," reminding personnel to pay attention to safety. It also increases the brightness of lights to improve visibility on site, raises the sampling rate of the PIR / flame sensor to 1Hz, and enters "standby" mode. like or Preset threshold The camera reduces its bit rate and resolution, uses intelligent encoding, and pauses data transmission in non-critical areas to free up network resources. Collect risk probability after multi-level intervention Calculate the prediction effect ,like <Preset threshold B, adjust intervention strategy.

6. A security video surveillance management method for smart campuses as described in claim 5, characterized in that: The process of generating a visual report based on prediction results and anomalies, and performing operation and maintenance optimization, refers to displaying the risk field on a GIS map, highlighting abnormal areas, and generating a visual report. If anomalies are identified... Preset threshold This triggers a web interface alarm and highlights the area of ​​the camera data stream that is being transmitted first, with weekly statistical feature averages. and variance And combined with the prediction effect Refresh the preset threshold Q and the abnormal threshold. Monitor camera bandwidth and ESP32 heartbeat. If bandwidth is congested or the device is offline, generate a device health report and trigger a maintenance notification.

7. A security video surveillance management method for smart campuses as described in claim 6, characterized in that: The deployment of multimodal sensing devices and the generation of datasets through data acquisition and preprocessing refer to the deployment of fixed high-definition cameras, microphones, environmental sensors, PIR sensors, DFR0076 flame sensors, and RFID access control card readers in key areas of the campus. Data is acquired through multimodal sensing devices, and the data is preprocessed and labeled. A labeled dataset is generated based on the processed data.

8. A security video surveillance management system for smart campuses, based on the security video surveillance management method for smart campuses as described in any one of claims 1 to 7, characterized in that: include, The data processing and feature extraction module is used to generate structured datasets based on the collected multimodal data using feature extraction methods. The anomaly detection and risk assessment module is used to calculate anomaly correlation scores and feature fusion based on graph neural networks, generate a subset of anomaly features, and calculate the risk value of each area of ​​the campus using these features. The tiered intervention and equipment priority classification module is used to calculate the risk probability and the updated risk value, and to classify equipment priorities. The anomaly detection and prediction effect adjustment module is used to use the Z-score anomaly detection method to calculate the feature mean and variance based on the risk probability, and further identify anomaly regions. The visualization and report generation module is used to generate visual reports on the effectiveness of risk prediction and information on abnormal areas.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, it implements the steps of the security video surveillance management method for smart campus as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, it implements the steps of the security video surveillance management method for smart campus as described in any one of claims 1 to 7.

Images