Smart campus security monitoring method and system based on visual understanding

Abstract

The present application relates to the field of image processing technology, and discloses a smart campus security monitoring method and system based on visual understanding. The method includes: performing adaptive semantic segmentation on campus scene images through prior knowledge of campus functional areas to obtain a campus functional area segmentation map; extracting multi-dimensional identity features of campus personnel based on the segmentation map to obtain personnel identity detection results; inputting the identity detection results into the campus behavior semantic perception network to perform abnormal behavior recognition to obtain abnormal behavior recognition results; performing a comprehensive evaluation based on the segmentation map, identity detection results and abnormal behavior recognition results through the campus scene context perception fusion algorithm to obtain campus security monitoring decision results. The present application solves the technical problem that abnormal behavior detection in existing campus security monitoring technologies lacks spatial semantic correction and temporal correlation analysis.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a smart campus security monitoring method and system based on visual understanding. Background Art

[0002] Existing campus security surveillance technologies are primarily based on traditional video surveillance and single-algorithm detection methods. These typically employ general-purpose target detection algorithms, such as the YOLO family or the SSD algorithm, for person and object recognition, combined with simple motion detection or background modeling techniques for identifying abnormal behavior. These technologies have demonstrated limited detection capabilities in general surveillance scenarios, capable of identifying basic human targets and simple abnormal behavior patterns, and have been widely adopted in commercial and industrial surveillance. Furthermore, some existing behavior recognition methods based on deep learning also exist, employing convolutional neural networks or recurrent neural networks to classify and identify human behavior in video sequences.

[0003] However, existing technologies suffer from significant technical deficiencies, manifesting themselves in three key areas: First, there's the high false alarm rate. General algorithms fail to understand the unique semantics of campus environments, often mistaking the normal work of maintenance personnel for suspicious behavior, resulting in a large number of false alarms that impact monitoring efficiency. Second, algorithms lack generalization capabilities. Existing models are often trained on general datasets, making them difficult to adapt to the unique layout, demographics, and activity patterns of campus environments, and unable to accurately identify identity categories and behavioral patterns unique to campuses. Third, there's a lack of contextual awareness of campus scenarios. Existing technologies are unable to correlate detection results with campus functional areas, time periods, and personnel identities, lacking an understanding of campus spatial semantics and temporal logic.

[0004] Based on an in-depth analysis of the aforementioned technical deficiencies, existing technologies face even more complex, progressive technical problems: Current abnormal behavior detection results lack a spatial semantic correction mechanism, making it impossible to judge the rationality of the behavior based on the specific functional area where the behavior occurs, resulting in inconsistent rationality judgment criteria for the same behavior in different areas. Furthermore, existing technologies lack the ability to analyze the temporal correlation of abnormal events, are unable to identify the causal relationship between related abnormal events, and struggle to construct a complete abnormal event development chain. A deeper problem lies in the inability of existing technologies to assess the potential impact and spread risk of a single abnormal behavior on the surrounding group, and lack the ability to quantitatively analyze the impact of group behavior. Ultimately, existing monitoring systems are unable to formulate preventive intervention strategies based on the development trajectory and impact range of abnormal events, lack proactive early warning and risk prediction capabilities, and can only passively respond to security incidents that have already occurred. Summary of the Invention

[0005] This application provides a smart campus security monitoring method and system based on visual understanding, which solves the technical problem of the lack of spatial semantic correction and temporal correlation analysis in abnormal behavior detection in existing campus security monitoring technologies.

[0006] In the first aspect, the present application provides a smart campus security monitoring method based on visual understanding, and the smart campus security monitoring method based on visual understanding includes: performing adaptive semantic segmentation processing on the campus scene image through prior knowledge of the campus functional area to obtain a campus functional area segmentation map; performing multi-dimensional identity feature extraction processing on campus personnel according to the campus functional area segmentation map to obtain campus personnel identity detection results; inputting the campus personnel identity detection results into the campus behavior semantic perception network to perform abnormal behavior pattern recognition processing to obtain campus abnormal behavior recognition results; performing comprehensive evaluation processing based on the campus functional area segmentation map, campus personnel identity detection results and campus abnormal behavior recognition results through the campus scene context perception fusion algorithm to obtain a campus security monitoring decision result.

[0007] Optionally, the adaptive semantic segmentation processing of the campus scene image by using prior knowledge of campus functional areas to obtain a campus functional area segmentation map includes:

[0008] The campus scene image is processed to extract the geometric features of campus buildings to obtain campus spatial structure information including building outline features, road direction features and green layout features;

[0009] Inputting the campus spatial structure information into the improved SegNet network for encoding processing to obtain a basic feature map of the campus area;

[0010] Performing differential decoding processing on the basic feature map of the campus area based on the campus functional area attention mechanism to obtain an enhanced feature map that integrates campus prior knowledge;

[0011] According to the enhanced feature map, pixel-level classification processing is performed to obtain a campus functional area segmentation map containing five functional area labels: teaching area, living area, sports area, management area, and transportation area.

[0012] Optionally, performing multi-dimensional identity feature extraction processing on campus personnel according to the campus functional area segmentation map to obtain campus personnel identity detection results includes:

[0013] Based on the campus functional area segmentation map, a three-branch feature extraction process is performed on the campus personnel image, including facial features, clothing pattern features, and behavioral posture features, to obtain a 128-dimensional face embedding vector, a 64-dimensional clothing feature vector, and 17 key point spatial relationship features;

[0014] Inputting the face embedding vector, clothing feature vector and key point spatial relationship feature into the gated fusion mechanism for multi-dimensional feature fusion processing to obtain a comprehensive identity feature vector of campus personnel;

[0015] Performing identity classification processing using a campus identity classifier based on the comprehensive identity feature vector of campus personnel to obtain identity classification results including on-campus students, faculty and staff, security personnel, cleaners, visitors, couriers, maintenance workers, and suspicious persons;

[0016] Based on the campus functional area segmentation map, the identity classification result is subjected to regional rationality verification processing to obtain a campus personnel identity detection result including identity category, confidence level and regional location information.

[0017] Optionally, inputting the campus personnel identity detection result into a campus behavior semantic perception network to perform abnormal behavior pattern recognition processing to obtain a campus abnormal behavior recognition result includes:

[0018] Inputting the campus personnel identity detection results into the campus behavior semantic perception network to extract personnel trajectory sequences and obtain campus personnel behavior trajectory data including location coordinates and timestamps;

[0019] Based on the campus personnel behavior trajectory data, multi-scale behavior pattern encoding processing is performed through a cascaded time series coding unit to obtain a 256-dimensional campus personnel behavior time series feature vector that integrates short-term behavior characteristics and long-term behavior trends;

[0020] Inputting the temporal feature vector of campus personnel behavior into the cascaded spatial memory unit for hierarchical campus behavior pattern matching processing, and obtaining similarity matching results in three time dimensions: teaching period, break period, and non-teaching period;

[0021] Based on the similarity matching results, a cascade abnormal behavior discrimination process is performed using a campus scene adaptive contrast learning loss function to obtain a campus-specific abnormal behavior classification result that distinguishes between classroom disruption and illegal intrusion;

[0022] Based on the classification results of the specific abnormal behaviors on campus, a multi-dimensional abnormality degree quantification process is performed through a cascade risk assessment unit to obtain an abnormal behavior score value that integrates the dangerousness and urgency of the behavior;

[0023] According to the abnormal behavior score value, a cascade result integration process is performed through the campus security level mapping algorithm to obtain a campus abnormal behavior identification result including campus specific abnormal type, risk level and early warning suggestion.

[0024] Optionally, the campus personnel behavior time series feature vector is input into a cascaded spatial memory unit for hierarchical campus behavior pattern matching processing to obtain similarity matching results in three time dimensions: teaching period, break period, and non-teaching period, including:

[0025] Performing time window segmentation processing on the campus personnel behavior time series feature vector to obtain teaching period behavior feature sub-vectors, break period behavior feature sub-vectors, and non-teaching period behavior feature sub-vectors;

[0026] Inputting the teaching period behavior feature subvector into the teaching behavior pattern library of the cascaded spatial memory unit for similarity calculation processing to obtain a first similarity matching value with the standard classroom behavior, laboratory behavior, and library behavior;

[0027] Inputting the inter-class period behavior feature subvector into the inter-class behavior pattern library of the cascaded spatial memory unit for similarity calculation processing to obtain a second similarity matching value with the corridor passage behavior, the cafeteria dining behavior, and the playground activity behavior;

[0028] The non-teaching period behavior feature sub-vector is input into the non-teaching behavior pattern library of the cascaded spatial memory unit for similarity calculation processing to obtain a third similarity matching value with the dormitory area behavior, campus patrol behavior and facility maintenance behavior, and based on the first similarity matching value, the second similarity matching value and the third similarity matching value, a time series fusion processing is performed to obtain similarity matching results in the three time dimensions of teaching period, break period and non-teaching period.

[0029] Optionally, the campus functional area segmentation map, the campus personnel identity detection results, and the campus abnormal behavior recognition results are comprehensively evaluated and processed by a campus scene context perception fusion algorithm to obtain a campus security monitoring decision result, including:

[0030] Based on the campus functional area segmentation map, campus personnel identity detection results, and campus abnormal behavior recognition results, reverse verification processing is performed to obtain campus abnormal behavior screening results that have been spatially semantically corrected by cross-validating the detected abnormal behavior with the rationality of the functional area in which it is located;

[0031] Performing time series correlation mining based on the results of the campus abnormal behavior screening, and obtaining a chain of campus abnormal events with causal relationships by analyzing the behavior change patterns of personnel in the previous and next time windows;

[0032] Conducting a group behavior impact assessment on the campus abnormal event chain, and calculating the potential impact radius and impact intensity of a single abnormal behavior on the surrounding population, to obtain a campus group safety risk diffusion assessment result;

[0033] Based on the results of the campus group safety risk diffusion assessment, preventive intervention strategies are generated and processed. By predicting the development trajectory and impact range of abnormal events, campus safety monitoring decision-making results including early warning opportunities, types of intervention measures and resource allocation recommendations are obtained.

[0034] Optionally, the reverse verification process is performed based on the campus functional area segmentation map, the campus personnel identity detection results, and the campus abnormal behavior identification results, and a spatial semantically corrected campus abnormal behavior screening result is obtained by cross-validating the detected abnormal behavior with the rationality of the functional area in which it is located, including:

[0035] Extracting the spatial coordinates of the abnormal behavior based on the abnormal behavior recognition results on campus to perform regional attribution determination processing and obtain the functional area identifier to which the abnormal behavior belongs;

[0036] Based on the functional area identifier, the campus functional area reasonable behavior database is queried to perform behavior matching verification processing to obtain a verification benchmark for the allowed behavior type and the prohibited behavior type in the corresponding area;

[0037] Performing a logical comparison between the abnormal behavior recognition result and the verification benchmark to obtain a behavior classification label that distinguishes true abnormalities from false abnormalities;

[0038] The abnormal campus behavior identification result is screened and filtered according to the behavior classification label to obtain a spatial semantically corrected abnormal campus behavior screening result.

[0039] In a second aspect, the present application provides a smart campus security monitoring system based on visual understanding, the smart campus security monitoring system based on visual understanding comprising:

[0040] The segmentation module is used to perform adaptive semantic segmentation processing on the campus scene image based on the prior knowledge of the campus functional areas to obtain the campus functional area segmentation map;

[0041] An extraction module is used to extract multi-dimensional identity features of campus personnel based on the campus functional area segmentation map to obtain campus personnel identity detection results;

[0042] An identification module is used to input the campus personnel identity detection results into the campus behavior semantic perception network to perform abnormal behavior pattern recognition processing to obtain campus abnormal behavior recognition results;

[0043] The evaluation module is used to perform comprehensive evaluation processing based on the campus functional area segmentation map, campus personnel identity detection results and campus abnormal behavior recognition results through the campus scene context perception fusion algorithm to obtain the campus security monitoring decision result.

[0044] In the third aspect, a smart campus security monitoring device based on visual understanding is provided, comprising: a memory and at least one processor, wherein instructions are stored in the memory; the at least one processor calls the instructions in the memory so that the smart campus security monitoring device based on visual understanding executes the above-mentioned smart campus security monitoring method based on visual understanding.

[0045] In a fourth aspect, a computer-readable storage medium is provided, in which instructions are stored. When the computer-readable storage medium is run on a computer, the computer executes the above-mentioned smart campus security monitoring method based on visual understanding.

[0046] The technical solution provided in this application can accurately identify functional areas such as teaching areas, living areas, sports areas, management areas, and transportation areas by integrating campus-specific spatial semantic information. Compared with general semantic segmentation algorithms, this technology is specially optimized for the special layout and architectural features of campus scenes, significantly improving the accuracy of regional recognition in campus environments and providing accurate spatial context information for subsequent identity detection and behavior analysis. Multi-dimensional identity feature extraction processing can accurately distinguish eight campus-specific identity categories such as students, faculty and staff, security personnel, cleaners, visitors, couriers, maintenance workers, and suspicious persons by integrating facial features, clothing pattern features, and behavioral posture features. This solves the technical problem that general identity recognition algorithms cannot understand the unique types of people on campus. At the same time, it combines functional area information to verify regional rationality, effectively reducing the misjudgment rate of identity recognition. The campus behavior semantic perception network, through the design of cascaded temporal coding and spatial memory units, can deeply understand the complex behavior patterns in campus scenes and accurately identify campus-specific abnormal behaviors such as classroom disruptions and illegal intrusions. Compared with traditional anomaly detection methods, it has stronger scene adaptability and lower false alarm rate. The campus scene context-aware fusion algorithm uses innovative technologies such as reverse verification mechanism, time series correlation mining and group behavior impact assessment to achieve intelligent screening, correlation analysis and risk assessment of abnormal events. It can generate proactive security decision-making plans that include warning opportunities, intervention measures and resource allocation, effectively solving the technical limitations of existing technologies that can only passively respond to security incidents.

[0047] In the field of campus security monitoring, the campus functional area attention mechanism, as a core component of an AI algorithm, learns campus-specific spatial feature patterns and automatically adjusts the feature weights of different functional areas, enabling the algorithm to better understand the spatial semantic relationships of the campus environment. This targeted algorithm design exhibits greater domain adaptability than general-purpose attention mechanisms. The cascaded architecture of the campus behavior semantic perception network demonstrates the advantages of AI algorithms in temporal modeling. Through multi-scale time windows and hierarchical behavioral pattern matching, the algorithm captures both short-term dynamic characteristics and long-term trends of campus behavior. This hierarchical feature learning capability enables the system to accurately distinguish between normal campus activities and abnormal behavior, significantly reducing false alarm rates. The campus scene-adaptive contrastive learning loss function, a specially designed AI training strategy, optimizes feature representation through a positive-negative sample comparison mechanism, giving the algorithm stronger discriminative power in specific campus scenarios and effectively identifying subtle behavioral differences. This targeted algorithm optimization strategy demonstrates superior performance in campus security monitoring tasks compared to traditional loss functions. The group behavior impact assessment model, through AI algorithms modeling the propagation patterns and impact range of abnormal events, can quantitatively analyze the diffusion trends of security risks and provide a basis for preventive intervention. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0049] Figure 1 This is a schematic diagram of an embodiment of a smart campus security monitoring method based on visual understanding in an embodiment of the present application;

[0050] Figure 2 This is a schematic diagram of an embodiment of a smart campus security monitoring system based on visual understanding in an embodiment of the present application;

[0051] Figure 3 This is a schematic block diagram of the structure of a smart campus security monitoring device based on visual understanding in an embodiment of the present invention. DETAILED DESCRIPTION

[0052] The embodiments of the present application provide a method and system for smart campus security monitoring based on visual understanding. The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or that are inherent to these processes, methods, products or devices.

[0053] For ease of understanding, the specific process of the embodiment of the present application is described below. Figure 1 In the embodiments of the present application, an embodiment of a smart campus security monitoring method based on visual understanding includes:

[0054] Step S101: Adaptively perform semantic segmentation processing on the campus scene image based on prior knowledge of campus functional areas to obtain a campus functional area segmentation map;

[0055] Step S102: extracting multi-dimensional identity features of campus personnel based on the campus functional area segmentation map to obtain campus personnel identity detection results;

[0056] Step S103: input the campus personnel identity detection results into the campus behavior semantic perception network to perform abnormal behavior pattern recognition processing to obtain campus abnormal behavior recognition results;

[0057] Step S104: Comprehensively evaluate and process the campus scene context perception fusion algorithm based on the campus functional area segmentation map, campus personnel identity detection results, and campus abnormal behavior recognition results to obtain a campus security monitoring decision result.

[0058] It is understandable that the execution subject of this application can be a smart campus security monitoring system based on visual understanding, or a terminal or a server, which is not limited here. The embodiment of this application is described by taking the server as the execution subject as an example.

[0059] Specifically, adaptive semantic segmentation is performed on campus scene images using prior knowledge of campus functional areas. This prior knowledge refers to pre-constructed spatial semantic information based on campus layout patterns, including building distribution patterns, road connectivity, and functional area demarcation criteria. Adaptive semantic segmentation first extracts geometric features from the input campus scene image. A depthwise separable convolution operation is used to identify building boundary points. Road orientation features and greenery layout features are then extracted to form the campus spatial structure. This spatial structure information is then fed into an improved SegNet network for encoding. The encoder progressively compresses feature dimensions through convolutional and pooling layers, mapping the original image features into low-dimensional feature representations. The campus functional area attention mechanism plays a key role in the decoding stage. By calculating the importance weights of different regional features, it assigns higher weights to teaching buildings and lower weights to ordinary roads, achieving differentiated feature decoding. Finally, pixel-level classification is used to categorize each pixel as a teaching area, living area, sports area, administrative area, or transportation area, forming a campus functional area segmentation map.

[0060] Based on the campus functional area segmentation map, multi-dimensional identity features are extracted for campus personnel. A three-branch feature extraction architecture is used to process different types of identity information. The facial feature branch uses a convolutional neural network to extract facial landmark locations and texture information. A feature mapping layer compresses the high-dimensional facial image into a 128-dimensional feature vector, which incorporates biometric features such as eye distance, nose bridge outline, and facial symmetry. The clothing pattern feature branch analyzes the color distribution and texture patterns of a person's clothing. Using color space conversion and texture statistics, a 64-dimensional clothing feature vector is extracted. This vector can distinguish between different clothing types, such as student uniforms, faculty and staff formal attire, and uniforms. The behavioral posture feature branch uses a keypoint detection algorithm to identify the locations of 17 major human joints, calculate the angular relationships and distance ratios between adjacent joints, and generate spatial relationship features. A gated fusion mechanism dynamically adjusts the importance of the three features using learnable weight parameters, increasing the weight of facial features when facial information is clear and increasing the contribution of clothing features when clothing features are prominent. Ultimately, these features are integrated to generate a comprehensive identity feature vector for campus personnel. The campus identity classifier outputs the probability distribution of eight identity categories based on this comprehensive feature vector, and checks the matching degree between the identity and the functional area through regional rationality verification.

[0061] The results of campus person identification are fed into a campus behavior semantic perception network for abnormal behavior pattern recognition. This network is specifically designed to understand complex behavioral patterns in campus scenarios. The network first extracts person trajectory sequences from the identification results, recording the position coordinate changes of each detected target in consecutive time frames and labeling them with corresponding timestamps to form spatiotemporal trajectory data. The cascaded temporal encoding unit employs a multi-scale temporal window analysis strategy. Short time windows capture transient behavioral features such as walking speed and direction changes, while long time windows identify persistent behavioral patterns such as the duration of stay in a certain area. These two scales are cascaded to form a 256-dimensional behavior temporal feature vector. The cascaded spatial memory unit maintains three independent behavior pattern libraries: standard behavior patterns during teaching hours, transitional behavior patterns during breaks, and free behavior patterns during non-teaching hours. The network calculates the similarity between the input behavior feature vector and the behavior pattern library for the corresponding time period, and obtains a matching score using the cosine similarity metric. The adaptive contrastive learning loss function for campus scenarios incorporates a positive-negative sample comparison mechanism, treating normal campus behavior as positive samples and abnormal behavior as negative samples. The feature representation is optimized by maximizing the similarity of positive samples while minimizing the similarity of negative samples. The cascade risk assessment unit comprehensively considers the dangerousness and urgency level of the behavior and assigns a quantitative score to the identified abnormal behavior.

[0062] Based on the outputs of the three aforementioned steps, a comprehensive safety assessment is performed using a campus scenario context-aware fusion algorithm. The algorithm's core innovation lies in the introduction of a reverse validation mechanism and temporal correlation analysis. The reverse validation process verifies the plausibility of abnormal behavior detection results using spatial semantic constraints. For example, if strenuous exercise is detected within a teaching building, the algorithm queries whether such activity is permitted in the area during the corresponding time period. If it is not normal, the abnormality mark is retained; if it is normal physical education activity, it is reclassified as normal. The temporal correlation mining process analyzes the sequence of behavioral changes within a continuous time window to identify causal chains of events, such as a pattern where a gathering of people precedes a conflict. The group behavior impact assessment process calculates the potential impact of a single abnormal behavior on the surrounding population and determines the impact radius and intensity parameters by analyzing the diffusion patterns of similar events in historical data. The preventive intervention strategy generation process, based on the predicted trajectory of abnormal events and combined with campus management rules and emergency response plan requirements, automatically generates a decision plan that includes warning timing, intervention measures, and resource allocation recommendations, resulting in a campus safety monitoring decision. This effectively addresses the technical issues of high false alarm rates and lack of contextual awareness in existing technologies.

[0063] In a specific embodiment, the process of executing step S101 may specifically include the following steps:

[0064] The campus scene image is processed to extract the geometric features of campus buildings to obtain campus spatial structure information including building outline features, road direction features and green space layout features;

[0065] The campus spatial structure information is input into the improved SegNet network for encoding processing to obtain the basic feature map of the campus area;

[0066] Based on the attention mechanism of campus functional areas, the basic feature map of the campus area is differentially decoded to obtain an enhanced feature map that integrates campus prior knowledge;

[0067] Based on the enhanced feature map, pixel-level classification processing is performed to obtain a campus functional area segmentation map containing five functional area labels: teaching area, living area, sports area, management area, and transportation area.

[0068] Specifically, the campus building geometric feature extraction process uses multi-layer convolution operations to identify the spatial structure information of the input campus scene image. The building contour feature extraction uses an edge detection operator to identify the boundary between the building and the background, and calculates the pixel gradient change to find the contour boundary points and connect them to form a complete set of building contour lines. The road direction feature extraction identifies the road area in the image based on the linear structure detection algorithm, and obtains the main direction vector information of the road by analyzing the continuity and directionality of the road pixels. The green layout feature extraction identifies the vegetation area through color space analysis, converts the RGB image into the HSV color space, and extracts the pixel area with a higher green component. The distribution density and spatial layout pattern of the green area are calculated, and finally the building contour features, road direction features and green layout features are integrated to form a campus spatial structure information dataset.

[0069] The improved SegNet network uses an encoder-decoder architecture for encoding, performing deep feature learning on campus spatial structure information. The encoder consists of a cascade of multiple convolutional layers and pooling layers. Each convolutional layer uses a convolution kernel of a specific size to extract local features from the input feature map. The pooling layer uses downsampling to reduce the spatial resolution of the feature map while retaining key feature information. During the encoding process, the network updates its weight parameters using a backpropagation algorithm. The training data contains a large number of annotated campus scene images and corresponding functional area labels. The network learns how to map the input spatial structure information into high-level semantic feature representations. During training, the loss function calculates the difference between the predicted results and the true labels. The network parameters are continuously adjusted using the gradient descent method until convergence. Finally, the encoder outputs a basic feature map of the campus area, which contains the deep semantic information of the image but has a low spatial resolution.

[0070] The campus functional area attention mechanism performs differentiated decoding on the basic feature map of the campus area. The attention mechanism achieves selective feature enhancement by calculating the importance weights of features at different spatial locations. The mechanism first calculates an attention score for each feature vector in the basic feature map. The score is calculated based on the similarity between the feature vector and the prototype vectors of the campus functional areas. The prototype vectors of the teaching area encode the characteristic patterns of typical teaching buildings, while the prototype vectors of the living area contain the characteristic information of living facilities such as dormitories and cafeterias. The attention score is normalized using a softmax function and used as a weight coefficient. This is multiplied by the feature vector at the corresponding position to produce a weighted feature representation. Features of areas with higher weights are enhanced, while features of areas with lower weights are suppressed. The decoder combines the weighted feature information with prior knowledge of the campus to perform upsampling. The spatial resolution of the feature map is gradually restored through transposed convolution operations. Pre-built prior knowledge of the campus layout is also incorporated into the decoding process to constrain the decoding process, resulting in an enhanced feature map that incorporates prior knowledge of the campus.

[0071] Pixel-level classification processing assigns each pixel in the enhanced feature map to a corresponding functional area category. The classifier uses a fully connected layer structure to predict the category of the feature vector at each pixel position and outputs a probability distribution vector of five functional area categories. The category with the highest probability is used as the final classification result for the pixel. During the classification process, the feature vector of each pixel contains the spatial context information and semantic feature information of the position. The classifier establishes classification rules by learning the correspondence between pixel features and functional area labels in a large number of training samples. The final output campus functional area segmentation map is a label map of the same size as the original image. Each pixel value in the map corresponds to a label in the teaching area, living area, sports area, management area or transportation area, forming a complete campus space functional division result.

[0072] In a specific embodiment, the process of executing step S102 may specifically include the following steps:

[0073] Based on the campus functional area segmentation map, a three-branch feature extraction process is performed on the campus personnel images, including facial features, clothing pattern features, and behavioral posture features. A 128-dimensional face embedding vector, a 64-dimensional clothing feature vector, and 17 key point spatial relationship features are obtained.

[0074] The face embedding vector, clothing feature vector and key point spatial relationship feature are input into the gated fusion mechanism for multi-dimensional feature fusion processing to obtain the comprehensive identity feature vector of campus personnel;

[0075] According to the comprehensive identity feature vector of campus personnel, the campus identity classifier is used to perform identity classification processing, and the identity classification results including students, faculty and staff, security personnel, cleaners, visitors, couriers, maintenance workers and suspicious persons are obtained;

[0076] Based on the campus functional area segmentation map, the regional rationality verification processing of the identity classification results is carried out to obtain the campus personnel identity detection results containing identity category, confidence and regional location information.

[0077] Specifically, the three-branch feature extraction process uses a parallel architecture to simultaneously process three different types of identity information: face, clothing, and posture. The facial feature branch uses a convolutional neural network to extract key facial features from the detected face area. The network uses multi-layer convolution operations to identify facial geometric information such as eye contours, nose bridge shape, and mouth features. Then, through a fully connected layer, the high-dimensional facial features are mapped into a fixed-length 128-dimensional face embedding vector, which encodes the individual's biometric information. The clothing pattern feature branch analyzes the visual characteristics of the person's clothing, calculates the distribution of the main color of the clothing through color histogram statistics, and uses the local binary pattern operator to extract clothing texture features. After splicing the color features and texture features and performing dimensionality reduction processing, a 64-dimensional clothing feature vector is obtained. This vector can distinguish different types of clothing such as school uniforms, formal wear, and uniforms. The behavioral posture feature branch uses a keypoint detection algorithm to identify 17 major human joints, including the head, shoulders, elbows, wrists, hips, knees, and ankles. By calculating the Euclidean distance and angular relationship between adjacent joints, it generates keypoint spatial relationship features, which describe the body's posture information and movement patterns. A gated fusion mechanism performs multidimensional feature fusion processing, dynamically adjusting the importance of three features using a learnable weighted gating structure. The mechanism first calculates a quality assessment score for each feature: the quality score for facial features is based on the clarity and completeness of the facial region; the quality score for clothing features is based on the visibility and contrast of the clothing region; and the quality score for posture features is based on the confidence level of keypoint detection. The gated unit uses a sigmoid activation function to convert the quality scores into weight coefficients. The weight coefficients are multiplied by the corresponding feature vectors to produce a weighted feature representation. The three weighted feature vectors are combined through a concatenation operation to form a comprehensive campus identity feature vector, which integrates multimodal identity information and is adaptively adjusted based on feature quality.

[0078] The campus identity classifier uses a multi-layer perceptron structure to classify and predict the comprehensive identity feature vector for identity category recognition. The classifier contains two hidden layers and an output layer. The hidden layer uses the ReLU activation function for nonlinear transformation, and the output layer uses the softmax activation function to output the probability distribution of eight identity categories. The classifier learns the characteristic patterns of different identity categories through training. Students on campus usually wear uniforms and are young. Faculty and staff dress formally and behave steadily. Security personnel wear uniforms and have upright postures. Cleaners dress simply and carry cleaning tools. Visitors dress diversely and behave more restrainedly. Couriers wear work clothes and carry packages. Maintenance workers wear work clothes and carry tools. Suspicious persons have obvious differences in dress or behavior from common identities.

[0079] Regional legitimacy verification checks the match between the identity recognition result and the area in which it is located based on the campus functional area segmentation map. The verification process first searches the functional area segmentation map for the corresponding area label based on the center coordinates of the person detection box. Then, it queries the preset identity-area legitimacy mapping table to determine the legitimacy of the identity's presence in the area. For example, a student's presence in the teaching area is reasonable, while their presence in the management area requires further verification. A cleaner's presence in the living and transportation areas is reasonable, while their presence in the teaching area during specific hours is abnormal. The verification result includes a confidence score. The confidence is high when the identity and area match well, and low when the match is low. The final output includes a complete detection result containing identity category, confidence, and area location information.

[0080] In a specific embodiment, the process of executing step S103 may specifically include the following steps:

[0081] The campus personnel identity detection results are input into the campus behavior semantic perception network to extract the personnel trajectory sequence and obtain the campus personnel behavior trajectory data containing location coordinates and timestamps;

[0082] Based on the campus personnel behavior trajectory data, multi-scale behavior pattern encoding is performed through a cascaded temporal coding unit to obtain a 256-dimensional campus personnel behavior temporal feature vector that integrates short-term behavior characteristics and long-term behavior trends.

[0083] The temporal feature vectors of campus personnel behavior are input into the cascaded spatial memory unit for hierarchical campus behavior pattern matching, and similarity matching results are obtained in the three time dimensions of teaching period, break period and non-teaching period.

[0084] Based on the similarity matching results, a cascade abnormal behavior discrimination process is performed using the campus scene adaptive contrast learning loss function to obtain the campus-specific abnormal behavior classification results that distinguish between classroom disruptions and illegal intrusions;

[0085] Based on the classification results of specific abnormal behaviors on campus, a cascade risk assessment unit is used to quantify the degree of abnormality in multiple dimensions, and a numerical abnormal behavior score is obtained that integrates the dangerousness and urgency of the behavior;

[0086] According to the abnormal behavior score value, the campus security level mapping algorithm is used to perform cascade result integration processing to obtain the campus abnormal behavior recognition results including campus-specific abnormality type, risk level and early warning suggestions.

[0087] Specifically, the campus behavior semantic perception network extracts and processes personnel trajectory sequences and tracks personnel targets in continuous video frames through a tracking algorithm. The network extracts the center coordinates of each detection frame from the campus personnel identity detection results as the spatial position of the person at that moment, and records the corresponding video frame timestamp to form a spatiotemporal data point. Through target association between multiple frames, the position coordinates of the same person at different moments are connected in chronological order to form a trajectory sequence. The trajectory data structure contains the personnel identifier, the two-dimensional coordinate position at each moment, and the corresponding timestamp information. The network predicts the possible position of the person in the next frame through the Kalman filter algorithm and matches it with the actual detection results to ensure the continuity and accuracy of the trajectory. The cascaded temporal coding unit performs multi-scale behavioral pattern encoding processing and adopts a two-layer cascade structure to process behavioral information on short-term and long-term time scales respectively. The short-term coding unit uses a sliding window mechanism to analyze trajectory changes within several consecutive seconds and calculate instantaneous behavioral characteristics such as the person's movement speed, acceleration, and direction change rate. The long-term coding unit analyzes trajectory patterns within a time range of several minutes and identifies persistent behavioral characteristics such as the person's stay area, movement path, and activity cycle. The outputs of the two coding units are combined through a splicing operation to form a 256-dimensional comprehensive behavioral temporal feature vector, which contains both the instantaneous dynamics and long-term trend information of the behavior.

[0088] Cascaded spatial memory units perform hierarchical campus behavior pattern matching and maintain three independent behavior pattern repositories, corresponding to typical behavior patterns during teaching, recess, and non-teaching periods. The teaching period behavior pattern library stores standard teaching behaviors, such as students remaining relatively still after entering the classroom and teachers moving around the podium. The recess behavior pattern library includes transitional behaviors such as students moving quickly in the corridor, queuing for meals in the cafeteria, and engaging in physical activities on the playground. The non-teaching period behavior pattern library records daily activity patterns, such as students' daily activities in the dormitory area, security personnel's patrol routes, and cleaners' work trajectories. The matching process calculates the cosine similarity between the input behavior feature vector and the patterns stored in each pattern library to obtain a similarity score. The similarity calculation is based on the cosine value of the angle between the feature vectors. Values ​​closer to 1 indicate more similar behavior patterns, while values ​​closer to 0 indicate greater dissimilarity. The campus scene adaptive contrast learning loss function is used to perform cascade abnormal behavior discrimination processing. A positive and negative sample comparison mechanism is used to train the behavior discrimination model. Positive samples include normal campus behavior patterns in different time periods, and negative samples include labeled abnormal behavior samples such as classroom disruptions and illegal intrusions. The loss function optimizes the model parameters by maximizing the feature similarity between normal behaviors while minimizing the similarity between normal and abnormal behaviors. The discrimination processing outputs the probability distribution of each abnormal behavior type, and the category with the highest probability is used as the final abnormal behavior classification result.

[0089] The cascaded risk assessment unit quantifies the severity of abnormalities across multiple dimensions, comprehensively considering both the dangerousness and urgency of abnormal behaviors. Danger is assessed based on the potential threat the behavior poses to personnel safety. Classroom disruptions are considered low-risk behaviors. Urgency is assessed based on the urgency of the behavior requiring an immediate response. Illegal intrusions are considered more urgent at night and less urgent during the day. The assessment unit combines the dangerousness and urgency scores to calculate a comprehensive risk score using a weighted summation. The weighting coefficients are set based on campus security management policies, with dangerousness weights typically set high to prioritize high-risk behaviors. The campus security level mapping algorithm integrates the cascaded results and maps risk scores to standardized security levels. The algorithm establishes a correspondence between score ranges and warning levels: low scores correspond to green (normal), medium scores correspond to yellow (minor abnormality), high scores correspond to orange (moderate threat), and the highest scores correspond to red (severe threat). The algorithm also generates warning recommendations and remedial measures based on the abnormality type and risk level.

[0090] In a specific embodiment, the step of inputting the campus personnel behavior time series feature vector into the cascaded spatial memory unit for hierarchical campus behavior pattern matching processing may specifically include the following steps:

[0091] Perform time window segmentation on the time series feature vectors of campus personnel behavior to obtain the teaching period behavior feature sub-vectors, the break period behavior feature sub-vectors, and the non-teaching period behavior feature sub-vectors;

[0092] Inputting the teaching period behavior feature subvector into the teaching behavior pattern library of the cascaded spatial memory unit for similarity calculation processing, and obtaining the first similarity matching value with the standard classroom behavior, laboratory behavior and library behavior;

[0093] Inputting the behavior feature subvectors of the recess period into the recess behavior pattern library of the cascaded spatial memory unit for similarity calculation processing, and obtaining a second similarity matching value with the corridor passage behavior, the cafeteria dining behavior, and the playground activity behavior;

[0094] The behavioral feature subvectors of the non-teaching period are input into the non-teaching behavior pattern library of the cascaded spatial memory unit for similarity calculation processing to obtain the third similarity matching value with the dormitory area behavior, campus patrol behavior and facility maintenance behavior. Based on the first similarity matching value, the second similarity matching value and the third similarity matching value, time series fusion processing is performed to obtain the similarity matching results of the three time dimensions of teaching period, break period and non-teaching period.

[0095] Specifically, the time window segmentation process divides the 256-dimensional temporal feature vector of campus personnel behavior into three independent sub-vectors based on the campus schedule. The segmentation algorithm divides the complete behavior feature vector into three independent sub-vectors according to the preset time boundaries. The teaching period generally corresponds to the formal class time in the morning and afternoon. The inter-class period includes transition time such as breaks and lunch breaks. The non-teaching period covers free time in the morning, evening, and night. The segmentation process locates the feature components corresponding to different time periods in the feature vector by timestamp index. Each sub-vector maintains the dimensional structure of the original feature but only contains behavioral information for a specific time period. The teaching period behavior feature sub-vector encodes the personnel behavior pattern in the classroom environment, the inter-class period behavior feature sub-vector reflects the activity characteristics in the transition state, and the non-teaching period behavior feature sub-vector describes the behavioral patterns during free time. The teaching behavior pattern library performs similarity calculations and maintains behavioral prototypes for three typical teaching scenarios: standard classroom behavior, laboratory behavior, and library behavior. The standard classroom behavior prototype includes characteristic patterns such as students sitting quietly listening, raising their hands to speak, and teachers lecturing. The laboratory behavior prototype describes the behavioral characteristics of students working in groups with experimental equipment and teachers guiding experiments. The library behavior prototype records activity patterns such as quiet reading, whispering conversation, and searching for books. Similarity calculations use vector inner product operations to measure the degree of match between the input teaching period behavior feature subvectors and each behavior prototype. The calculation process multiplies each dimensional component of the subvector with the component of the corresponding prototype vector, and the sum is calculated to obtain a similarity score. A higher score indicates that the behavior pattern is closer to the prototype. By comparing the similarity scores of the three prototypes, the most matching teaching behavior type is determined and the first similarity match value is output.

[0096] The inter-class behavior pattern library performs similarity calculations and stores typical activity patterns for three inter-class periods: corridor passage, cafeteria dining, and playground activity. The corridor passage behavior prototype records characteristics such as normal walking speed, direction changes, and dwell time in the corridor. The cafeteria dining behavior prototype includes the behavioral sequence characteristics of waiting in line, moving to pick up food, and sitting down to eat. The playground activity behavior prototype describes sports behavior patterns such as physical exercise, leisure activities, and group games. The calculation process matches the sub-vectors of the inter-class period behavior characteristics with the three inter-class behavior prototypes. The degree of conformity between the behavior characteristics and each prototype is evaluated using a vector similarity measurement method. The similarity calculation results reflect the consistency of the observed inter-class behavior with the standard inter-class activity pattern, and the output is a second similarity matching value containing the similarity scores of the three prototypes.

[0097] The non-teaching behavior pattern library, through similarity calculations, includes three types of behavior patterns during non-teaching periods: dormitory area behavior, campus patrol behavior, and facility maintenance behavior. The dormitory area behavior prototype records student daily behavior characteristics within the dormitory building, including entering and exiting the dormitory, activities within the building, and resting. The campus patrol behavior prototype describes the security personnel's regular patrol routes, inspection frequency, and emergency response behavior patterns. The facility maintenance behavior prototype includes the work behavior characteristics of logistics personnel such as cleaners and maintenance workers, such as equipment inspection, cleaning, and maintenance operations. The similarity calculation matches and analyzes the non-teaching period behavior feature subvectors with the three non-teaching behavior prototypes. The calculation process evaluates the similarity between the observed behavior and standard non-teaching activities. The matching result outputs a third similarity match value, which contains the similarity scores with the three non-teaching behavior prototypes.

[0098] The temporal fusion process comprehensively analyzes the first, second, and third similarity matching values. The fusion algorithm considers the interrelationships and temporal continuity constraints of the similarity results across the three time periods, combining the similarity scores of different time periods into a comprehensive evaluation result using a weighted average method. The weight assignment is based on the time period type and the temporal characteristics of the behavior at the current moment. Behavior within the teaching period relies more on the matching results of the teaching behavior pattern library, while behavior during recess mainly refers to the output of the recess behavior pattern library. The fusion process also considers the behavioral transition characteristics between adjacent time periods. Under normal circumstances, recess behavior follows the teaching behavior in time. Under abnormal circumstances, the behavioral patterns of different time periods conflict or are discontinuous. The final output is a matching result that includes a comprehensive similarity evaluation of the three time dimensions of teaching period, recess period, and non-teaching period.

[0099] In a specific embodiment, the process of executing step S104 may specifically include the following steps:

[0100] Based on the campus functional area segmentation map, campus personnel identity detection results, and campus abnormal behavior recognition results, reverse verification processing is performed. By cross-validating the detected abnormal behavior with the rationality of the functional area where it is located, a campus abnormal behavior screening result with spatial semantic correction is obtained.

[0101] Based on the results of campus abnormal behavior screening, time series correlation mining is performed. By analyzing the changing patterns of personnel behavior within the previous and next time windows, a chain of abnormal campus events with causal relationships is obtained.

[0102] Conduct group behavior impact assessment on the chain of abnormal events on campus, and calculate the potential impact radius and impact intensity of a single abnormal behavior on the surrounding population to obtain the campus group safety risk diffusion assessment results;

[0103] Preventive intervention strategies are generated based on the results of campus group safety risk diffusion assessment. By predicting the development trajectory and impact range of abnormal events, campus safety monitoring decision-making results are obtained, including early warning opportunities, types of intervention measures, and resource allocation recommendations.

[0104] Specifically, the reverse validation process verifies the rationality of abnormal behavior detection results based on a spatial semantic constraint mechanism. The validation algorithm first extracts the spatial coordinate information of each abnormal event in the campus abnormal behavior identification results. It then searches the corresponding location's region label within the campus functional area segmentation map to determine the specific functional area type where the abnormal behavior occurred. The cross-validation process determines the rationality of a specific abnormal behavior in a specific area by querying a pre-built functional area-behavior rationality mapping table. This mapping table records the types of behaviors permitted and prohibited in different functional areas at different times. For example, in teaching areas, loud noises or strenuous exercise are prohibited during class time, while in sports areas, strenuous exercise is permitted during physical education classes but restricted during non-sports hours. The validation algorithm matches the detected abnormal behavior type with the behavioral constraints of the area in which it occurs. If the abnormal behavior conflicts with the area's constraints, the abnormality label is retained. If the abnormal behavior conforms to the normal activity pattern of the area, it is reclassified as normal behavior. The validation process outputs abnormal behavior screening results that have been spatially semantically corrected, effectively filtering out false positives caused by insufficient understanding of regional semantics.

[0105] Time series correlation mining uses sliding time window analysis technology to identify temporal dependencies between abnormal events. The mining algorithm constructs a time series data structure based on the results of campus abnormal behavior screening, recording the time of occurrence, duration, and event type of each abnormal event. Correlation analysis uses a forward and backward time window scanning method, setting a fixed-size time window for each abnormal event. The occurrence of other abnormal events within this window is searched for, and the degree of correlation between events is determined by calculating the time interval and spatial distance between events. Causal relationship identification is based on the temporal sequence of events and the logical relationship between behaviors. When the occurrence of a previous abnormal event creates conditions for subsequent abnormal events or triggers a chain reaction, the algorithm connects these events to form a chain of abnormal events. The chain structure consists of event nodes, temporal edges, and causal relationship labels. Each chain represents a complete abnormal event development process. The mining process outputs correlation analysis results containing multiple abnormal event chains.

[0106] The crowd behavior impact assessment process uses a spatial impact model to calculate the potential impact range and intensity of a single abnormal behavior on the surrounding population. The assessment algorithm first determines the coordinates of the spatial center point of the abnormal event. It then sets the impact radius parameter based on the type and intensity of the abnormal behavior. Minor violations have a smaller impact radius. The impact intensity calculation takes into account factors such as the dangerousness of the abnormal behavior, crowd density, and transmission speed. The algorithm analyzes the diffusion patterns of similar historical events to establish an impact intensity attenuation model. The impact intensity gradually decreases with increasing distance and is also affected by physical obstacles and crowd flow direction. The assessment process also considers crowd psychology and the herd effect. When abnormal behavior attracts onlookers or imitation, its impact range is amplified. The algorithm determines the intensity of the crowd reaction by monitoring the behavioral changes of people around the abnormal event. The final output is a crowd safety risk diffusion assessment result that includes the impact radius, impact intensity distribution, and potential diffusion paths.

[0107] Preventive intervention strategy generation and processing utilizes a model predicting the trajectory of abnormal events to formulate proactive intervention plans. The strategy generation algorithm analyzes the risk propagation patterns and impact scope of campus group safety risk diffusion assessments. It then integrates historical experience with abnormal event handling and the requirements of the campus emergency plan to generate targeted intervention measures. Warning timing is determined through risk threshold monitoring and trend prediction algorithms to calculate the optimal intervention time. Early warning mechanisms are triggered when the risk diffusion assessment indicates a trend of further deterioration. The timing of warnings balances intervention effectiveness with resource consumption. Intervention measures are graded based on the nature and severity of the abnormal event. Minor abnormalities require on-site alerts and guidance, moderate abnormalities require security personnel intervention and on-site control, and severe abnormalities require the activation of emergency plans and a multi-departmental response. Resource allocation recommendations generate an optimized allocation plan based on the required intervention measures and the availability of campus resources. The algorithm considers the time cost and distance of personnel scheduling, prioritizing security personnel closest to the incident while reserving backup resources to mitigate potential chain reactions.

[0108] In a specific embodiment, the process of performing reverse verification based on the campus functional area segmentation map, the campus personnel identity detection results, and the campus abnormal behavior recognition results may specifically include the following steps:

[0109] According to the results of abnormal behavior recognition on campus, the spatial coordinates of abnormal behavior are extracted to determine the region attribution and obtain the functional region identification to which the abnormal behavior belongs;

[0110] Based on the functional area identification, the reasonable behavior database of the campus functional area is queried to perform behavior matching verification processing, and the verification benchmark of the allowed behavior type and prohibited behavior type in the corresponding area is obtained;

[0111] Perform a logical comparison between the abnormal behavior recognition results and the verification benchmark to obtain the behavior classification labels that distinguish true anomalies from false anomalies.

[0112] The abnormal behavior recognition results on campus are screened and filtered according to the behavior classification labels to obtain the abnormal behavior screening results on campus after spatial semantic correction.

[0113] Specifically, the abnormal behavior spatial location coordinate extraction process parses the bounding box information in the campus abnormal behavior identification results to obtain the precise spatial location of the abnormal event. The extraction algorithm reads the bounding box coordinate parameters of each abnormal event from the abnormal behavior detection data and calculates the geometric center point of the bounding box as the representative spatial location coordinate of the abnormal behavior. The regional attribution determination process spatially matches the extracted spatial coordinates with the campus functional area segmentation map. The determination algorithm searches the segmentation map for the corresponding pixel area based on the coordinate position. The label value directly corresponds to a functional area type such as teaching area, living area, sports area, management area, or transportation area. The specific functional area to which the abnormal behavior belongs is determined through the pixel value mapping relationship.

[0114] The database query process for reasonable behavior in campus functional areas retrieves the behavioral constraints for the corresponding area based on the obtained functional area identifiers. The database uses the functional area identifiers as index keys to store behavioral norms for each area over different time periods. The query algorithm locates the corresponding behavioral constraint records based on the area identifier and the current timestamp. The behavior matching verification process extracts a list of permitted and prohibited behavior types from the database records. The permitted behavior type records the activity patterns normally permitted in the area, while the prohibited behavior type lists the behavior patterns explicitly prohibited in the area. The verification process outputs a verification benchmark data structure containing these two behavior type lists.

[0115] The logical comparison process compares the behavior category information in the abnormal behavior identification results with the behavior type list in the verification benchmark. The comparison algorithm first checks whether the abnormal behavior category exists in the prohibited behavior type list. If a match is successful, the behavior is confirmed as a true abnormality. If a match is not found, the behavior is further checked to see if it belongs to the allowed behavior type list. Detection results for allowed behaviors are marked as false positives. The comparison process also considers the influence of time. The reasonableness criteria for the same behavior vary in different time periods. The comparison algorithm adjusts the judgment criteria based on the specific time of the abnormal event, and ultimately outputs a behavior classification result containing both true anomaly and false positive anomaly classification labels.

[0116] The filtering process cleans the raw campus abnormal behavior identification results based on the behavior classification labels. The filtering algorithm traverses all abnormal behavior detection records, retaining detection results marked as true anomalies and deleting records marked as false anomalies. The filtering process also includes a confidence adjustment operation. For abnormal behaviors occurring near the area boundary, the algorithm lowers the confidence score to reflect the uncertainty of the spatial location. For abnormal behaviors occurring in the center of the area and clearly violating the regional behavioral norms, the algorithm increases the confidence score to enhance the reliability of the detection results. The filtering process outputs abnormal behavior screening results that have been spatially semantically corrected.

[0117] To illustrate the complete data processing flow, a coordinate extraction algorithm calculates the center coordinates of the fight from the bounding box information of the detection result. Region attribution is determined by querying the functional region segmentation map to determine the corresponding sports zone identifier for these coordinates. A database query process retrieves the relevant behavioral constraint rules based on the sports zone identifier and the current time period. It finds that the sports zone allows intense exercise during physical education classes, but prohibits conflicting behaviors. The validation benchmark includes a detailed classification of permitted and prohibited behaviors. A logical comparison process analyzes the detected behavior category and finds that the behavior falls into the conflict category and matches a prohibited behavior list. This comparison results in the behavior being marked as a true anomaly. The filtering process retains the detection record of this true anomaly. Furthermore, because the anomaly occurred in the center of the sports zone and clearly violated the zone's behavioral norms, the algorithm increases the confidence score of the detection result. The final output includes a correction result with a high-confidence true anomaly marker, resolving the technical issue of existing technologies that cannot distinguish between normal and abnormal exercise behavior.

[0118] The above describes the smart campus security monitoring method based on visual understanding in the embodiment of the present application. The following describes the smart campus security monitoring system based on visual understanding in the embodiment of the present application. Figure 2 In the embodiment of the present application, an embodiment of a smart campus security monitoring system based on visual understanding includes:

[0119] Segmentation module 201, configured to perform adaptive semantic segmentation processing on the campus scene image based on prior knowledge of campus functional areas to obtain a campus functional area segmentation map;

[0120] An extraction module 202 is configured to extract multi-dimensional identity features of campus personnel based on the campus functional area segmentation map to obtain campus personnel identity detection results;

[0121] Identification module 203, configured to input the campus personnel identity detection result into the campus behavior semantic perception network to perform abnormal behavior pattern recognition processing to obtain campus abnormal behavior recognition results;

[0122] The evaluation module 204 is used to perform comprehensive evaluation processing based on the campus functional area segmentation map, campus personnel identity detection results and campus abnormal behavior recognition results through a campus scene context perception fusion algorithm to obtain a campus security monitoring decision result.

[0123] above Figure 2 The smart campus security monitoring system based on visual understanding in the embodiment of the present invention is described in detail from the perspective of modular functional entities. The smart campus security monitoring equipment based on visual understanding in the embodiment of the present invention is described in detail from the perspective of hardware processing.

[0124] Reference Figure 3 In the embodiment of the present invention, a smart campus security monitoring device based on visual understanding is also provided. The smart campus security monitoring device based on visual understanding can be a server, and its internal structure can be as follows: Figure 3 As shown. The smart campus security monitoring device based on visual understanding includes a processor, a memory, a display screen, an input device, a network interface and a database connected via a system bus. Among them, the computer-designed processor is used to provide computing and control capabilities. The memory of the smart campus security monitoring device based on visual understanding includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the smart campus security monitoring device based on visual understanding is used to store the corresponding data in this embodiment. The network interface of the smart campus security monitoring device based on visual understanding is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the above method is implemented.

[0125] Those skilled in the art will understand that Figure 3 The structure shown in is merely a block diagram of a portion of the structure related to the solution of the present invention, and does not constitute a limitation on the visual understanding-based smart campus security monitoring equipment to which the solution of the present invention is applied.

[0126] The present invention also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. The computer-readable storage medium stores instructions, which, when executed on a computer, enable the computer to execute the steps of the smart campus security monitoring method based on visual understanding.

[0127] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described systems, systems and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0128] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a smart campus security monitoring device based on visual understanding (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, and other media that can store program code.

[0129] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A smart campus security monitoring method based on visual understanding, characterized in that: The method comprises: Adaptive semantic segmentation processing is performed on the campus scene image based on the prior knowledge of campus functional areas to obtain the campus functional area segmentation map; Performing multi-dimensional identity feature extraction processing on campus personnel according to the campus functional area segmentation map to obtain campus personnel identity detection results; The campus personnel identity detection result is input into the campus behavior semantic perception network for abnormal behavior pattern recognition processing to obtain the campus abnormal behavior recognition result, including: inputting the campus personnel identity detection result into the campus behavior semantic perception network for personnel trajectory sequence extraction processing to obtain campus personnel behavior trajectory data containing location coordinates and timestamps; based on the campus personnel behavior trajectory data, multi-scale behavior pattern encoding processing is performed through a cascaded temporal coding unit to obtain a 256-dimensional campus personnel behavior temporal feature vector that integrates short-term behavior characteristics and long-term behavior trends; inputting the campus personnel behavior temporal feature vector into a cascaded spatial memory unit for hierarchical campus behavior pattern matching processing to obtain The results are compared to the similarity matching results of the three time dimensions of teaching period, break period and non-teaching period; based on the similarity matching results, a cascade abnormal behavior discrimination process is performed through the campus scene adaptive contrast learning loss function to obtain a campus-specific abnormal behavior classification result that distinguishes classroom disruption and illegal intrusion; based on the campus-specific abnormal behavior classification result, a multi-dimensional abnormality degree quantification process is performed through a cascade risk assessment unit to obtain an abnormal behavior score value that integrates the dangerousness and urgency of the behavior; based on the abnormal behavior score value, a cascade result integration process is performed through a campus safety level mapping algorithm to obtain a campus abnormal behavior recognition result that includes campus-specific abnormality type, risk level and warning suggestions; Based on the campus functional area segmentation map, campus personnel identity detection results and campus abnormal behavior recognition results, a comprehensive evaluation process is performed through the campus scene context perception fusion algorithm to obtain a campus security monitoring decision result.

2. The method for smart campus security monitoring based on visual understanding according to claim 1 is characterized in that: The adaptive semantic segmentation processing of the campus scene image is performed based on the prior knowledge of the campus functional areas to obtain the campus functional area segmentation map, including: The campus scene image is processed to extract the geometric features of campus buildings to obtain campus spatial structure information including building outline features, road direction features and green space layout features; Inputting the campus spatial structure information into the improved SegNet network for encoding processing to obtain a basic feature map of the campus area; Performing differential decoding processing on the basic feature map of the campus area based on the campus functional area attention mechanism to obtain an enhanced feature map that integrates campus prior knowledge; According to the enhanced feature map, pixel-level classification processing is performed to obtain a campus functional area segmentation map containing five functional area labels: teaching area, living area, sports area, management area, and transportation area.

3. The method for smart campus security monitoring based on visual understanding according to claim 1 is characterized in that: The multi-dimensional identity feature extraction process is performed on campus personnel according to the campus functional area segmentation map to obtain campus personnel identity detection results, including: Based on the campus functional area segmentation map, a three-branch feature extraction process is performed on the campus personnel image, including facial features, clothing pattern features, and behavioral posture features, to obtain a 128-dimensional face embedding vector, a 64-dimensional clothing feature vector, and 17 key point spatial relationship features; Inputting the face embedding vector, clothing feature vector and key point spatial relationship feature into the gated fusion mechanism for multi-dimensional feature fusion processing to obtain a comprehensive identity feature vector of campus personnel; Performing identity classification processing using a campus identity classifier based on the comprehensive identity feature vector of campus personnel to obtain identity classification results including on-campus students, faculty and staff, security personnel, cleaners, visitors, couriers, maintenance workers, and suspicious persons; Based on the campus functional area segmentation map, the identity classification result is subjected to regional rationality verification processing to obtain a campus personnel identity detection result including identity category, confidence level and regional location information.

4. The method for smart campus security monitoring based on visual understanding according to claim 1 is characterized in that: The campus personnel behavior time series feature vector is input into the cascaded spatial memory unit for hierarchical campus behavior pattern matching processing to obtain similarity matching results in three time dimensions: teaching period, break period, and non-teaching period, including: Performing time window segmentation processing on the campus personnel behavior time series feature vector to obtain teaching period behavior feature sub-vectors, break period behavior feature sub-vectors, and non-teaching period behavior feature sub-vectors; Inputting the teaching period behavior feature subvector into the teaching behavior pattern library of the cascaded spatial memory unit for similarity calculation processing to obtain a first similarity matching value with the standard classroom behavior, laboratory behavior, and library behavior; Inputting the inter-class period behavior feature subvector into the inter-class behavior pattern library of the cascaded spatial memory unit for similarity calculation processing to obtain a second similarity matching value with the corridor passage behavior, the cafeteria dining behavior, and the playground activity behavior; The non-teaching period behavior feature sub-vector is input into the non-teaching behavior pattern library of the cascaded spatial memory unit for similarity calculation processing to obtain a third similarity matching value with the dormitory area behavior, campus patrol behavior and facility maintenance behavior, and based on the first similarity matching value, the second similarity matching value and the third similarity matching value, a time series fusion processing is performed to obtain similarity matching results in the three time dimensions of teaching period, break period and non-teaching period.

5. The method for smart campus security monitoring based on visual understanding according to claim 1 is characterized in that: The campus functional area segmentation map, campus personnel identity detection results, and campus abnormal behavior recognition results are comprehensively evaluated and processed using a campus scene context perception fusion algorithm to obtain a campus security monitoring decision result, including: Based on the campus functional area segmentation map, campus personnel identity detection results, and campus abnormal behavior recognition results, reverse verification processing is performed to obtain campus abnormal behavior screening results that have been spatially semantically corrected by cross-validating the detected abnormal behavior with the rationality of the functional area in which it is located; Performing time series correlation mining based on the results of the campus abnormal behavior screening, and obtaining a chain of campus abnormal events with causal relationships by analyzing the behavior change patterns of personnel in the previous and next time windows; Conducting a group behavior impact assessment on the campus abnormal event chain, and calculating the potential impact radius and impact intensity of a single abnormal behavior on the surrounding population, to obtain a campus group safety risk diffusion assessment result; Based on the results of the campus group safety risk diffusion assessment, preventive intervention strategies are generated and processed. By predicting the development trajectory and impact range of abnormal events, campus safety monitoring decision-making results including early warning opportunities, types of intervention measures and resource allocation recommendations are obtained.

6. The method for smart campus security monitoring based on visual understanding according to claim 5 is characterized in that: The reverse verification process is performed based on the campus functional area segmentation map, the campus personnel identity detection results, and the campus abnormal behavior identification results. By cross-validating the detected abnormal behavior with the rationality of the functional area in which it is located, a campus abnormal behavior screening result that has undergone spatial semantic correction is obtained, including: Extracting the spatial coordinates of the abnormal behavior based on the abnormal behavior recognition results on campus to perform regional attribution determination processing and obtain the functional area identifier to which the abnormal behavior belongs; Based on the functional area identifier, the campus functional area reasonable behavior database is queried to perform behavior matching verification processing to obtain a verification benchmark for the allowed behavior type and the prohibited behavior type in the corresponding area; The abnormal behavior recognition result is logically compared with the verification benchmark to obtain a behavior classification label that distinguishes true anomalies from falsely detected anomalies; the campus abnormal behavior recognition result is screened and filtered according to the behavior classification label to obtain a campus abnormal behavior screening result that has been spatially semantically corrected.

7. A smart campus security monitoring system based on visual understanding, characterized by: For implementing the smart campus security monitoring method based on visual understanding according to any one of claims 1 to 6, the smart campus security monitoring system based on visual understanding comprises: The segmentation module is used to perform adaptive semantic segmentation processing on the campus scene image based on the prior knowledge of the campus functional areas to obtain the campus functional area segmentation map; An extraction module is used to extract multi-dimensional identity features of campus personnel based on the campus functional area segmentation map to obtain campus personnel identity detection results; The identification module is used to input the campus personnel identity detection results into the campus behavior semantic perception network for abnormal behavior pattern recognition processing to obtain campus abnormal behavior recognition results, including: inputting the campus personnel identity detection results into the campus behavior semantic perception network for personnel trajectory sequence extraction processing to obtain campus personnel behavior trajectory data containing location coordinates and timestamps; based on the campus personnel behavior trajectory data, multi-scale behavior pattern encoding processing is performed through a cascaded temporal coding unit to obtain a 256-dimensional campus personnel behavior temporal feature vector that integrates short-term behavior characteristics and long-term behavior trends; inputting the campus personnel behavior temporal feature vector into a cascaded spatial memory unit for hierarchical campus behavior pattern matching Processing is performed to obtain similarity matching results in the three time dimensions of teaching period, break period and non-teaching period; based on the similarity matching results, cascade abnormal behavior discrimination processing is performed through the campus scene adaptive contrast learning loss function to obtain a campus-specific abnormal behavior classification result that distinguishes classroom disruption and illegal intrusion; based on the campus-specific abnormal behavior classification result, multi-dimensional abnormality degree quantification processing is performed through the cascade risk assessment unit to obtain an abnormal behavior score value that integrates the dangerousness and urgency of the behavior; based on the abnormal behavior score value, cascade result integration processing is performed through the campus safety level mapping algorithm to obtain a campus abnormal behavior recognition result that includes campus-specific abnormality type, risk level and warning suggestions; The evaluation module is used to perform comprehensive evaluation processing based on the campus functional area segmentation map, campus personnel identity detection results and campus abnormal behavior recognition results through the campus scene context perception fusion algorithm to obtain the campus security monitoring decision result.

8. A smart campus security monitoring device based on visual understanding, characterized by: It includes a memory and a processor, the memory stores a computer program that can be run on the processor, and when the processor executes the computer program, it implements the smart campus security monitoring method based on visual understanding as described in any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the processor executes the smart campus security monitoring method based on visual understanding as described in any one of claims 1 to 6.

Images