Intelligent campus safety monitoring method and system based on visual understanding

Abstract

The invention relates to the technical field of image processing, and discloses a smart campus security monitoring method and system based on visual understanding. The method comprises the following steps: carrying out adaptive semantic segmentation on a campus scene image through priori knowledge of a campus functional region to obtain a campus functional region segmentation map; performing multi-dimensional identity feature extraction on the campus personnel according to the segmentation map to obtain a personnel identity detection result; the identity detection result is input into a campus behavior semantic perception network for abnormal behavior recognition, and an abnormal behavior recognition result is obtained; and according to the segmentation map, the identity detection result and the abnormal behavior identification result, performing comprehensive evaluation through a campus scene context sensing fusion algorithm to obtain a campus security monitoring decision result. The abnormal behavior detection method and device solve the technical problem that abnormal behavior detection in an existing campus security monitoring technology lacks spatial semantic correction and time sequence correlation analysis.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and particularly to an intelligent campus security monitoring method and system based on visual understanding. Background Art

[0002] Existing campus security monitoring technologies mainly rely on traditional video monitoring and single-algorithm detection methods. Generally, common object detection algorithms such as the YOLO series or SSD algorithm are used for personnel and object recognition, combined with simple motion detection or background modeling techniques for abnormal behavior recognition. These technologies have certain detection capabilities in general monitoring scenarios, can identify basic personnel targets and simple abnormal behavior patterns, and have been widely applied in the commercial and industrial monitoring fields. At the same time, there are also some behavior recognition methods based on deep learning in the existing technologies, which classify and recognize the behaviors of personnel in video sequences through convolutional neural networks or recurrent neural networks.

[0003] However, the existing technologies have significant technical deficiencies, mainly manifested in three aspects: First is the problem of high false alarm rate. The general algorithms cannot understand the special semantics of the campus scenario and often mistake the normal work of maintenance personnel for suspicious behaviors, resulting in a large number of false alarms that affect the monitoring efficiency. Second is the insufficient algorithm generalization ability. The existing models are often trained based on general datasets and are difficult to adapt to the special layout, population characteristics, and activity patterns of the campus environment, and cannot accurately identify the unique identity categories and behavior patterns of the campus. Third is the lack of campus scenario context awareness ability. The existing technologies cannot associate the detection results with campus functional areas, time periods, and personnel identities for correlation analysis, and lack the understanding of campus space semantics and temporal logic.

[0004] Based on the in-depth analysis of the above technical deficiencies, the existing technologies face more complex progressive technical problems: The current abnormal behavior detection results lack a spatial semantic correction mechanism and cannot judge the rationality of the behavior according to the specific functional area where the behavior occurs, resulting in inconsistent rationality judgment criteria for the same behavior in different areas. Further, the existing technologies lack the ability to analyze the temporal correlation of abnormal events, cannot identify the causal relationship between related abnormal events before and after, and are difficult to construct a complete development chain of abnormal events. The deeper problem is that the existing technologies cannot evaluate the potential influence range and diffusion risk of a single abnormal behavior on the surrounding group, and lack the quantitative analysis ability of group behavior influence. Eventually, the existing monitoring systems cannot formulate preventive intervention strategies according to the development trajectory and influence range of abnormal events, lack the ability of active early warning and risk prediction, and can only passively respond to security events that have occurred. Summary of the Invention

[0005] This application provides a smart campus security monitoring method and system based on visual understanding, which solves the technical problems of the lack of spatial semantic correction and temporal correlation analysis in abnormal behavior detection in the existing campus security monitoring technology.

[0006] In the first aspect, this application provides a smart campus security monitoring method based on visual understanding. The smart campus security monitoring method based on visual understanding includes: performing adaptive semantic segmentation processing on the campus scene image through the prior knowledge of campus functional areas to obtain a campus functional area segmentation map; performing multi-dimensional identity feature extraction processing on campus personnel according to the campus functional area segmentation map to obtain a campus personnel identity detection result; inputting the campus personnel identity detection result into a campus behavior semantic perception network for abnormal behavior pattern recognition processing to obtain a campus abnormal behavior recognition result; and performing comprehensive evaluation processing through a campus scene context perception fusion algorithm according to the campus functional area segmentation map, the campus personnel identity detection result, and the campus abnormal behavior recognition result to obtain a campus security monitoring decision result.

[0007] Optionally, the performing adaptive semantic segmentation processing on the campus scene image through the prior knowledge of campus functional areas to obtain a campus functional area segmentation map includes: Performing campus building geometric feature extraction processing on the campus scene image to obtain campus spatial structure information including building contour features, road orientation features, and greening layout features; Inputting the campus spatial structure information into an improved SegNet network for encoding processing to obtain a campus area basic feature map; Performing differential decoding processing on the campus area basic feature map based on the campus functional area attention mechanism to obtain an enhanced feature map integrating campus prior knowledge; Obtaining a campus functional area segmentation map including five functional area labels of teaching area, living area, sports area, management area, and traffic area through pixel-level classification processing according to the enhanced feature map.

[0008] Optionally, the performing multi-dimensional identity feature extraction processing on campus personnel according to the campus functional area segmentation map to obtain a campus personnel identity detection result includes: Performing three-branch feature extraction processing of face features, dressing pattern features, and behavior posture features on the campus personnel image based on the campus functional area segmentation map to obtain a 128-dimensional face embedding vector, a 64-dimensional dressing feature vector, and 17 key point spatial relationship features; Inputting the face embedding vector, the dressing feature vector, and the key point spatial relationship features into a gated fusion mechanism for multi-dimensional feature fusion processing to obtain a campus personnel comprehensive identity feature vector; Performing identity category recognition processing on the comprehensive campus personnel identity feature vector through a campus identity classifier to obtain an identity classification result including on-campus students, faculty members, security personnel, cleaners, visitors, couriers, maintenance workers, and suspicious personnel; Performing regional rationality verification processing on the identity classification result based on the campus functional area segmentation map to obtain a campus personnel identity detection result including identity category, confidence level, and regional location information.

[0009] Optionally, inputting the campus personnel identity detection result into a campus behavior semantic perception network for abnormal behavior pattern recognition processing to obtain a campus abnormal behavior recognition result, including: Inputting the campus personnel identity detection result into a campus behavior semantic perception network for personnel trajectory sequence extraction processing to obtain campus personnel behavior trajectory data including position coordinates and timestamps; Performing multi-scale behavior pattern encoding processing on the campus personnel behavior trajectory data through a cascaded temporal coding unit to obtain a 256-dimensional campus personnel behavior temporal feature vector integrating short-term behavior features and long-term behavior trends; Inputting the campus personnel behavior temporal feature vector into a cascaded spatial memory unit for hierarchical campus behavior pattern matching processing to obtain similarity matching results in three time dimensions of teaching period, break period, and non-teaching period; Performing cascaded abnormal behavior discrimination processing on the similarity matching result through a campus scene adaptability contrast learning loss function to obtain a campus-specific abnormal behavior classification result distinguishing classroom disruption and illegal intrusion; Performing multi-dimensional abnormal degree quantification processing on the campus-specific abnormal behavior classification result through a cascaded risk assessment unit to obtain an abnormal behavior score value integrating behavior danger and urgency; Performing cascaded result integration processing on the abnormal behavior score value through a campus security level mapping algorithm to obtain a campus abnormal behavior recognition result including campus-specific abnormal types, risk levels, and warning suggestions.

[0010] Optionally, inputting the campus personnel behavior temporal feature vector into a cascaded spatial memory unit for hierarchical campus behavior pattern matching processing to obtain similarity matching results in three time dimensions of teaching period, break period, and non-teaching period, including: Performing time window segmentation processing on the campus personnel behavior temporal feature vector to obtain a teaching period behavior feature sub-vector, a break period behavior feature sub-vector, and a non-teaching period behavior feature sub-vector; Inputting the teaching period behavior feature subvector into the teaching behavior pattern library of the cascaded spatial memory unit for similarity calculation processing to obtain a first similarity matching value with the standard classroom behavior, laboratory behavior and library behavior; Inputting the inter-class period behavior feature subvector into the inter-class behavior pattern library of the cascaded spatial memory unit for similarity calculation processing to obtain a second similarity matching value with the corridor passage behavior, the cafeteria dining behavior, and the playground activity behavior; The non-teaching period behavior feature sub-vector is input into the non-teaching behavior pattern library of the cascaded spatial memory unit for similarity calculation processing to obtain a third similarity matching value with the dormitory area behavior, campus patrol behavior and facility maintenance behavior, and based on the first similarity matching value, the second similarity matching value and the third similarity matching value, a time series fusion processing is performed to obtain similarity matching results in the three time dimensions of teaching period, break period and non-teaching period.

[0011] Optionally, the campus functional area segmentation map, the campus personnel identity detection results, and the campus abnormal behavior recognition results are comprehensively evaluated and processed by a campus scene context perception fusion algorithm to obtain a campus security monitoring decision result, including: Based on the campus functional area segmentation map, campus personnel identity detection results, and campus abnormal behavior recognition results, reverse verification processing is performed to obtain campus abnormal behavior screening results that have been spatially semantically corrected by cross-validating the detected abnormal behavior with the rationality of the functional area in which it is located; Performing time series correlation mining based on the results of the campus abnormal behavior screening, and obtaining a chain of campus abnormal events with causal relationships by analyzing the behavior change patterns of personnel in the previous and next time windows; Conducting a group behavior impact assessment on the campus abnormal event chain, and calculating the potential impact radius and impact intensity of a single abnormal behavior on the surrounding population, to obtain a campus group safety risk diffusion assessment result; Based on the results of the campus group safety risk diffusion assessment, preventive intervention strategies are generated and processed. By predicting the development trajectory and impact range of abnormal events, campus safety monitoring decision-making results including early warning opportunities, types of intervention measures and resource allocation recommendations are obtained.

[0012] Optionally, the reverse verification process is performed based on the campus functional area segmentation map, the campus personnel identity detection results, and the campus abnormal behavior identification results, and a spatial semantically corrected campus abnormal behavior screening result is obtained by cross-validating the detected abnormal behavior with the rationality of the functional area in which it is located, including: Extract the spatial location coordinates of the abnormal behavior based on the recognition result of the campus abnormal behavior, and perform area attribution determination processing to obtain the functional area identifier to which the abnormal behavior belongs; Query the reasonable behavior database of the campus functional area based on the functional area identifier, and perform behavior matching verification processing to obtain the verification benchmarks for the allowed behavior types and prohibited behavior types within the corresponding area; Perform logical comparison processing on the abnormal behavior recognition result and the verification benchmark to obtain a behavior classification label for distinguishing real abnormal and misdetected abnormal; Perform screening and filtering processing on the campus abnormal behavior recognition result according to the behavior classification label to obtain a screened result of the campus abnormal behavior with spatial semantic correction.

[0013] In a second aspect, the present application provides a smart campus security monitoring system based on visual understanding. The smart campus security monitoring system based on visual understanding includes: A segmentation module, configured to perform adaptive semantic segmentation processing on the campus scene image through the prior knowledge of the campus functional area to obtain a campus functional area segmentation map; An extraction module, configured to perform multi-dimensional identity feature extraction processing on campus personnel according to the campus functional area segmentation map to obtain a campus personnel identity detection result; An identification module, configured to input the campus personnel identity detection result into a campus behavior semantic perception network for abnormal behavior pattern recognition processing to obtain a campus abnormal behavior recognition result; An evaluation module, configured to perform comprehensive evaluation processing through a campus scene context awareness fusion algorithm according to the campus functional area segmentation map, the campus personnel identity detection result, and the campus abnormal behavior recognition result to obtain a campus security monitoring decision result.

[0014] In a third aspect, a smart campus security monitoring device based on visual understanding is provided, including: a memory and at least one processor, where instructions are stored in the memory; the at least one processor invokes the instructions in the memory so that the smart campus security monitoring device based on visual understanding executes the above-mentioned smart campus security monitoring method based on visual understanding.

[0015] In a fourth aspect, a computer-readable storage medium is provided, where instructions are stored in the computer-readable storage medium, and when it runs on a computer, it causes the computer to execute the above-mentioned smart campus security monitoring method based on visual understanding.

[0016] The technical solution provided in this application can accurately identify functional areas such as teaching areas, living areas, sports areas, management areas, and transportation areas by integrating campus-specific spatial semantic information. Compared with general semantic segmentation algorithms, this technology is specially optimized for the special layout and architectural features of campus scenes, significantly improving the accuracy of regional recognition in campus environments and providing accurate spatial context information for subsequent identity detection and behavior analysis. Multi-dimensional identity feature extraction processing can accurately distinguish eight campus-specific identity categories such as students, faculty and staff, security personnel, cleaners, visitors, couriers, maintenance workers, and suspicious persons by integrating facial features, clothing pattern features, and behavioral posture features. This solves the technical problem that general identity recognition algorithms cannot understand the unique types of people on campus. At the same time, it combines functional area information to verify regional rationality, effectively reducing the misjudgment rate of identity recognition. The campus behavior semantic perception network, through the design of cascaded temporal coding and spatial memory units, can deeply understand the complex behavior patterns in campus scenes and accurately identify campus-specific abnormal behaviors such as classroom disruptions and illegal intrusions. Compared with traditional anomaly detection methods, it has stronger scene adaptability and lower false alarm rate. The campus scene context-aware fusion algorithm uses innovative technologies such as reverse verification mechanism, time series correlation mining and group behavior impact assessment to achieve intelligent screening, correlation analysis and risk assessment of abnormal events. It can generate proactive security decision-making plans that include warning opportunities, intervention measures and resource allocation, effectively solving the technical limitations of existing technologies that can only passively respond to security incidents.

[0017] In the field of campus security monitoring, the campus functional area attention mechanism, as a core component of an AI algorithm, learns campus-specific spatial feature patterns and automatically adjusts the feature weights of different functional areas, enabling the algorithm to better understand the spatial semantic relationships of the campus environment. This targeted algorithm design exhibits greater domain adaptability than general-purpose attention mechanisms. The cascaded architecture of the campus behavior semantic perception network demonstrates the advantages of AI algorithms in temporal modeling. Through multi-scale time windows and hierarchical behavioral pattern matching, the algorithm captures both short-term dynamic characteristics and long-term trends of campus behavior. This hierarchical feature learning capability enables the system to accurately distinguish between normal campus activities and abnormal behavior, significantly reducing false alarm rates. The campus scene-adaptive contrastive learning loss function, a specially designed AI training strategy, optimizes feature representation through a positive-negative sample comparison mechanism, giving the algorithm stronger discriminative power in specific campus scenarios and effectively identifying subtle behavioral differences. This targeted algorithm optimization strategy demonstrates superior performance in campus security monitoring tasks compared to traditional loss functions. The group behavior impact assessment model, through AI algorithms modeling the propagation patterns and impact range of abnormal events, can quantitatively analyze the diffusion trends of security risks and provide a basis for preventive intervention. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] To more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the accompanying drawings required for the description of the embodiments. Obviously, the accompanying drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other accompanying drawings can also be obtained based on these drawings.

[0019] Figure 1 It is a schematic diagram of an embodiment of the intelligent campus security monitoring method based on visual understanding in the embodiments of the present application; Figure 2 It is a schematic diagram of an embodiment of the intelligent campus security monitoring system based on visual understanding in the embodiments of the present application; Figure 3 It is a structural schematic block diagram of the intelligent campus security monitoring device based on visual understanding in the embodiments of the present invention. Specific Embodiments

[0020] The embodiments of the present application provide an intelligent campus security monitoring method and system based on visual understanding. The terms "first", "second", "third", "fourth", etc. (if any) in the specification, claims and above-mentioned accompanying drawings of the present application are used to distinguish similar objects, and do not have to be used to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments described here can be implemented in an order other than that illustrated or described here. In addition, the terms "comprising" or "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device including a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0021] For ease of understanding, the following describes the specific process of the embodiments of the present application. Please refer to Figure 1 , an embodiment of the intelligent campus security monitoring method based on visual understanding in the embodiments of the present application includes: Step S101: Perform adaptive semantic segmentation processing on the campus scene image through the prior knowledge of campus functional areas to obtain a campus functional area segmentation map; Step S102: Perform multi-dimensional identity feature extraction processing on campus personnel according to the campus functional area segmentation map to obtain a campus personnel identity detection result; Step S103: Input the campus personnel identity detection result into the campus behavior semantic perception network for abnormal behavior pattern recognition processing to obtain a campus abnormal behavior recognition result; Step S104: Perform comprehensive evaluation and processing on the campus function area segmentation map, campus personnel identity detection results, and campus abnormal behavior recognition results through the campus scene context awareness fusion algorithm to obtain the campus security monitoring decision result.

[0022] It can be understood that the execution subject of this application can be an intelligent campus security monitoring system based on visual understanding, or a terminal or a server. Specifically, it is not limited here. In this embodiment of the application, the server is used as the execution subject for illustration.

[0023] Specifically, adaptive semantic segmentation processing is performed on the campus scene image through the prior knowledge of campus function areas. The prior knowledge of campus function areas refers to the spatial semantic information pre-constructed based on the campus layout rules, including the building distribution pattern, road connection relationship, and function area division standard. The adaptive semantic segmentation processing first extracts the geometric features of campus buildings from the input campus scene image, identifies the set of building contour boundary points through depthwise separable convolution operations, and at the same time extracts the road direction features and greening layout features to form the campus spatial structure information. Subsequently, this spatial structure information is input into the improved SegNet network for encoding processing. The encoder gradually compresses the feature dimensions through convolutional layers and pooling layers, mapping the original image features into low-dimensional feature representations. The campus function area attention mechanism plays a key role in the decoding stage. By calculating the importance weights of different region features, a higher weight is assigned to the teaching building area and a lower weight is assigned to the ordinary road, realizing differential feature decoding. Finally, each pixel point is classified into one of the teaching area, living area, sports area, management area, or traffic area through pixel-level classification to form the campus function area segmentation map.

[0024] Extract multi-dimensional identity features of campus personnel according to the campus functional area segmentation map, and use a three-branch feature extraction architecture to process different types of identity information respectively. The face feature branch uses a convolutional neural network to extract the positions of facial key points and texture information, and compresses the high-dimensional face image into a 128-dimensional feature vector through a feature mapping layer. This vector contains biometric features such as the distance between eyes, the contour of the nose bridge, and facial symmetry. The clothing pattern feature branch analyzes the color distribution and texture pattern of the personnel's clothing, and extracts a 64-dimensional clothing feature vector through color space conversion and texture statistical operators, which can distinguish different clothing types such as student school uniforms, formal clothes for teaching staff, and uniforms for staff. The behavior and posture feature branch identifies the positions of 17 main joints of the human body based on a key point detection algorithm, calculates the angular relationship and distance ratio between adjacent joints, and forms a spatial relationship feature. The gating fusion mechanism dynamically adjusts the importance of the three features through learnable weight parameters, increases the weight of face features when the face information is clear, and enhances the contribution of clothing features when the clothing features are obvious. Finally, it fuses and generates a comprehensive identity feature vector of campus personnel. The campus identity classifier outputs the probability distribution of eight identity categories based on this comprehensive feature vector, and checks the matching degree between the identity and the functional area where it is located through regional rationality verification.

[0025] Input the campus personnel identity detection results into the campus behavior semantic perception network for abnormal behavior pattern recognition processing. This network is specifically designed to understand complex behavior patterns in campus scenarios. The network first extracts the personnel trajectory sequence from the identity detection results, records the change of the position coordinates of each detection target in consecutive time frames, and at the same time marks the corresponding timestamp information to form spatio-temporal trajectory data. The cascaded temporal encoding unit adopts a multi-scale time window analysis strategy. The short time window captures instantaneous behavior features such as walking speed and direction changes, and the long time window identifies persistent behavior patterns such as the staying duration in a certain area. The two-scale features are concatenated to form a 256-dimensional behavior temporal feature vector. The cascaded spatial memory unit maintains three independent behavior pattern libraries, which store the standard behavior patterns during teaching periods, the transitional behavior patterns during break periods, and the free behavior patterns during non-teaching periods respectively. The network calculates the similarity between the input behavior feature vector and the behavior pattern library of the corresponding period, and obtains the matching score through the cosine similarity measurement method. The campus scene adaptability contrast learning loss function introduces a positive and negative sample contrast mechanism, takes normal campus behaviors as positive samples and abnormal behaviors as negative samples, and optimizes the feature representation by maximizing the similarity of positive samples and minimizing the similarity of negative samples at the same time. The cascaded risk assessment unit comprehensively considers the degree of danger and the level of urgency of the behavior, and quantitatively scores the identified abnormal behaviors.

[0026] Based on the output results of the above three steps, a comprehensive security assessment is carried out through the campus scene context-aware fusion algorithm. The core innovation of this algorithm lies in the introduction of a reverse verification mechanism and temporal correlation analysis. The reverse verification process examines the rationality of the abnormal behavior detection results through spatial semantic constraints. For example, when a violent movement behavior is detected in the teaching building, the algorithm will query whether such activities are allowed in this area during the corresponding time period. If it does not conform to the normal rules, the abnormal mark will be retained. If it belongs to a normal physical education activity, it will be reclassified as a normal behavior. The temporal correlation mining process analyzes the behavior change sequence within a continuous time window and identifies the event chain with causal relationships, such as the pattern of people gathering first and then a conflict behavior occurring. The group behavior impact assessment process calculates the potential impact range of a single abnormal behavior on the surrounding crowd, and determines the impact radius and impact intensity parameters by analyzing the diffusion law of similar events in historical data. The preventive intervention strategy generation process is based on the prediction of the development trajectory of abnormal events. Combining the campus management rules and the requirements of the emergency plan, it automatically generates a decision-making plan including the early warning timing, intervention measure types, and resource allocation suggestions, forming the campus security monitoring decision result, effectively solving the technical problems of high false alarm rate and lack of context awareness in the existing technologies.

[0027] In a specific embodiment, the process of executing step S101 may specifically include the following steps: Perform campus building geometric feature extraction processing on the campus scene image to obtain campus spatial structure information including building contour features, road orientation features, and greening layout features; Input the campus spatial structure information into the improved SegNet network for encoding processing to obtain the campus area basic feature map; Based on the campus functional area attention mechanism, perform differential decoding processing on the campus area basic feature map to obtain an enhanced feature map integrating campus prior knowledge; Through pixel-level classification processing according to the enhanced feature map, obtain a campus functional area segmentation map including five functional area labels: teaching area, living area, sports area, management area, and traffic area.

[0028] Specifically, the geometric feature extraction of campus buildings identifies the spatial structure information of the input campus scene images through multi-layer convolution operations. For the extraction of building contour features, an edge detection operator is used to identify the boundary between buildings and the background. By calculating the pixel gradient changes, the contour boundary points are found and connected to form a complete set of building contour lines. The extraction of road direction features is based on a linear structure detection algorithm to identify the road areas in the image. By analyzing the continuity and directionality of road pixels, the main direction vector information of the roads is obtained. The extraction of greening layout features identifies the vegetation areas through color space analysis. After converting the RGB image to the HSV color space, the pixel areas with higher green components are extracted, and the distribution density and spatial layout pattern of the greening areas are calculated. Finally, the building contour features, road direction features, and greening layout features are integrated to form a campus spatial structure information dataset.

[0029] The improved SegNet network performs encoding processing using an encoder-decoder architecture for in-depth feature learning of campus spatial structure information. The encoder part consists of a cascaded structure of multiple convolutional layers and pooling layers. Each convolutional layer uses a convolutional kernel of a specific size to extract local features from the input feature map. The pooling layer reduces the spatial resolution of the feature map through downsampling operations while retaining key feature information. During the encoding process, the network updates the weight parameters through the backpropagation algorithm. The training data includes a large number of labeled campus scene images and corresponding functional area labels, and the network learns how to map the input spatial structure information into high-level semantic feature representations. During the training process, the loss function calculates the difference between the predicted results and the true labels, and the network parameters are continuously adjusted by the gradient descent method until convergence. Finally, the encoder outputs the basic feature map of the campus area, which contains the deep semantic information of the image but has a low spatial resolution.

[0030] The attention mechanism for campus functional areas performs differential decoding processing on the basic feature map of the campus area. The attention mechanism realizes selective feature enhancement by calculating the importance weights of features at different spatial positions. First, the attention scores are calculated for each feature vector in the basic feature map. The score calculation is based on the similarity between the feature vector and the prototype vectors of campus functional areas. The prototype vector of the teaching area encodes the feature patterns of typical teaching buildings, and the prototype vector of the living area contains the feature information of living facilities such as dormitories and canteens. After normalizing the attention scores through the softmax function, they are used as weight coefficients and multiplied by the feature vectors at the corresponding positions to obtain the weighted feature representations. The feature areas with higher weights are enhanced, while the feature areas with lower weights are suppressed. The decoder combines the attention-weighted feature information and the campus prior knowledge for upsampling processing. Through transposed convolution operations, the spatial resolution of the feature map is gradually restored, and at the same time, the pre-constructed campus layout prior knowledge is incorporated to constrain the decoding process, resulting in an enhanced feature map that integrates the campus prior knowledge.

[0031] Pixel-level classification processing assigns each pixel in the enhanced feature map to a corresponding functional area category. The classifier uses a fully connected layer structure to predict the category of the feature vector at each pixel position and outputs a probability distribution vector of five functional area categories. The category with the highest probability is used as the final classification result for the pixel. During the classification process, the feature vector of each pixel contains the spatial context information and semantic feature information of the position. The classifier establishes classification rules by learning the correspondence between pixel features and functional area labels in a large number of training samples. The final output campus functional area segmentation map is a label map of the same size as the original image. Each pixel value in the map corresponds to a label in the teaching area, living area, sports area, management area or transportation area, forming a complete campus space functional division result.

[0032] In a specific embodiment, the process of executing step S102 may specifically include the following steps: Based on the campus functional area segmentation map, a three-branch feature extraction process is performed on the campus personnel images, including facial features, clothing pattern features, and behavioral posture features. A 128-dimensional face embedding vector, a 64-dimensional clothing feature vector, and 17 key point spatial relationship features are obtained. The face embedding vector, clothing feature vector and key point spatial relationship feature are input into the gated fusion mechanism for multi-dimensional feature fusion processing to obtain the comprehensive identity feature vector of campus personnel; According to the comprehensive identity feature vector of campus personnel, the campus identity classifier is used to perform identity classification processing, and the identity classification results including students, faculty and staff, security personnel, cleaners, visitors, couriers, maintenance workers and suspicious persons are obtained; Based on the campus functional area segmentation map, the regional rationality verification processing of the identity classification results is carried out to obtain the campus personnel identity detection results containing identity category, confidence and regional location information.

[0033] Specifically, the three-branch feature extraction process uses a parallel architecture to simultaneously process three different types of identity information: face, clothing, and posture. The facial feature branch uses a convolutional neural network to extract key facial features from the detected face area. The network uses multi-layer convolution operations to identify facial geometric information such as eye contours, nose bridge shape, and mouth features. Then, through a fully connected layer, the high-dimensional facial features are mapped into a fixed-length 128-dimensional face embedding vector, which encodes the individual's biometric information. The clothing pattern feature branch analyzes the visual characteristics of the person's clothing, calculates the distribution of the main color of the clothing through color histogram statistics, and uses the local binary pattern operator to extract clothing texture features. After splicing the color features and texture features and performing dimensionality reduction processing, a 64-dimensional clothing feature vector is obtained. This vector can distinguish different types of clothing such as school uniforms, formal wear, and uniforms. The behavioral posture feature branch uses a keypoint detection algorithm to identify 17 major human joints, including the head, shoulders, elbows, wrists, hips, knees, and ankles. By calculating the Euclidean distance and angular relationship between adjacent joints, it generates keypoint spatial relationship features, which describe the body's posture information and movement patterns. A gated fusion mechanism performs multidimensional feature fusion processing, dynamically adjusting the importance of three features using a learnable weighted gating structure. The mechanism first calculates a quality assessment score for each feature: the quality score for facial features is based on the clarity and completeness of the facial region; the quality score for clothing features is based on the visibility and contrast of the clothing region; and the quality score for posture features is based on the confidence level of keypoint detection. The gated unit uses a sigmoid activation function to convert the quality scores into weight coefficients. The weight coefficients are multiplied by the corresponding feature vectors to produce a weighted feature representation. The three weighted feature vectors are combined through a concatenation operation to form a comprehensive campus identity feature vector, which integrates multimodal identity information and is adaptively adjusted based on feature quality.

[0034] The campus identity classifier uses a multi-layer perceptron structure to classify and predict the comprehensive identity feature vector for identity category recognition. The classifier contains two hidden layers and an output layer. The hidden layer uses the ReLU activation function for nonlinear transformation, and the output layer uses the softmax activation function to output the probability distribution of eight identity categories. The classifier learns the characteristic patterns of different identity categories through training. Students on campus usually wear uniforms and are young. Faculty and staff dress formally and behave steadily. Security personnel wear uniforms and have upright postures. Cleaners dress simply and carry cleaning tools. Visitors dress diversely and behave more restrainedly. Couriers wear work clothes and carry packages. Maintenance workers wear work clothes and carry tools. Suspicious persons have obvious differences in dress or behavior from common identities.

[0035] The regional rationality verification process checks the matching degree between the identity recognition result and the location area based on the campus functional area segmentation map. In the verification process, first, the corresponding area label is found in the functional area segmentation map according to the center coordinates of the person detection box, and then the preset identity-area rationality mapping table is queried to judge the rationality of the identity appearing in this area. For example, it is reasonable for a student to appear in the teaching area, but further verification is required if the student appears in the management area. It is reasonable for a cleaner to appear in the living area and traffic area, but it is an abnormal situation if the cleaner appears in the teaching area during specific time periods. The verification result includes a confidence score. When the identity and the area have a high matching degree, the confidence is high; when the matching degree is low, the confidence decreases. Finally, a complete detection result including the identity category, confidence, and area location information is output.

[0036] In a specific embodiment, the process of executing step S103 may specifically include the following steps: Input the campus personnel identity detection result into the campus behavior semantic perception network for personnel trajectory sequence extraction processing to obtain campus personnel behavior trajectory data including location coordinates and timestamps; Based on the campus personnel behavior trajectory data, perform multi-scale behavior pattern encoding processing through a cascaded time series encoding unit to obtain a 256-dimensional campus personnel behavior time series feature vector that integrates short-term behavior characteristics and long-term behavior trends; Input the campus personnel behavior time series feature vector into the cascaded spatial memory unit for hierarchical campus behavior pattern matching processing to obtain similarity matching results in three time dimensions: teaching period, break period, and non-teaching period; According to the similarity matching results, perform cascaded abnormal behavior discrimination processing through the campus scene adaptability contrast learning loss function to obtain the campus specific abnormal behavior classification results that distinguish classroom disruption and illegal intrusion; Based on the campus specific abnormal behavior classification results, perform multi-dimensional abnormal degree quantification processing through the cascaded risk assessment unit to obtain an abnormal behavior score value that integrates behavior danger and urgency; According to the abnormal behavior score value, perform cascaded result integration processing through the campus safety level mapping algorithm to obtain the campus abnormal behavior recognition results including campus specific abnormal types, risk levels, and warning suggestions.

[0037] Specifically, the campus behavior semantic perception network extracts the personnel trajectory sequence through a tracking algorithm to track the personnel targets in consecutive video frames. The network extracts the center coordinates of each detection box from the campus personnel identity detection results as the spatial position of the personnel at that moment, and at the same time records the corresponding video frame timestamps to form spatio-temporal data points. By associating the targets between multiple frames, the position coordinates of the same personnel at different times are connected in chronological order to form a trajectory sequence. The trajectory data structure includes personnel identifiers, two-dimensional coordinate positions at each moment, and corresponding timestamp information. The network predicts the possible position of the personnel in the next frame through the Kalman filtering algorithm and matches it with the actual detection results to ensure the continuity and accuracy of the trajectory. The cascaded temporal encoding unit performs multi-scale behavior pattern encoding using a two-layer cascaded structure to process behavior information at short-term and long-term time scales respectively. The short-term encoding unit uses a sliding window mechanism to analyze the trajectory changes within consecutive seconds, calculating instantaneous behavior features such as the moving speed, acceleration, and direction change rate of the personnel. The long-term encoding unit analyzes the trajectory patterns within a time range of several minutes to identify persistent behavior features such as the staying areas, moving paths, and activity cycles of the personnel. The outputs of the two encoding units are combined through a concatenation operation to form a 256-dimensional comprehensive behavior temporal feature vector, which contains both the instantaneous dynamics and long-term trend information of the behavior.

[0038] The cascaded spatial memory unit performs hierarchical campus behavior pattern matching by maintaining three independent behavior pattern repositories, corresponding to the typical behavior patterns during teaching periods, break periods, and non-teaching periods respectively. The behavior pattern repository for teaching periods stores standard teaching behaviors such as students remaining relatively stationary after entering the classroom and teachers moving in the podium area. The behavior pattern repository for break periods includes transitional behaviors such as students moving quickly in the corridor, queuing for meals in the cafeteria, and engaging in sports activities on the playground. The behavior pattern repository for non-teaching periods records daily activity patterns such as the living behaviors of students in the dormitory area, the patrol paths of security personnel, and the working trajectories of cleaners. The matching process obtains similarity scores by calculating the cosine similarity between the input behavior feature vector and the stored patterns in each pattern repository. The similarity calculation is based on the cosine value of the angle between feature vectors. The closer the value is to 1, the more similar the behavior patterns are, and the closer the value is to 0, the greater the difference. The campus scene adaptability contrastive learning loss function performs cascaded abnormal behavior discrimination by training a behavior discrimination model using a positive and negative sample contrast mechanism. The positive samples include the normal campus behavior patterns in each period, and the negative samples include the labeled abnormal behavior samples such as classroom disruptions and illegal intrusions. The loss function optimizes the model parameters by maximizing the feature similarity between normal behaviors while minimizing the similarity between normal behaviors and abnormal behaviors. The discrimination process outputs the probability distribution of each abnormal behavior type, and the category with the highest probability is used as the final abnormal behavior classification result.

[0039] The cascaded risk assessment unit conducts multi-dimensional quantification of the anomaly degree, comprehensively considering two dimensions: the danger level and the urgency level of abnormal behaviors. The danger assessment is based on the potential threat degree of the behavior to personnel safety. Classroom disruptions belong to low-danger behaviors. The urgency assessment is based on the urgency degree that the behavior requires immediate response. Illegal intrusion has a relatively high urgency at night and a relatively low urgency during the day. The assessment unit combines the danger score and the urgency score through weighted summation to calculate the comprehensive risk score. The weight coefficients are set according to the campus security management strategy. The danger weight is usually set higher to ensure the priority handling of high-risk behaviors. The campus security level mapping algorithm conducts cascade result integration processing to map the risk score to a standardized security level. The algorithm establishes the corresponding relationship between the score range and the warning level. A low score corresponds to the green normal state, a medium score corresponds to the yellow slight anomaly, a relatively high score corresponds to the orange moderate threat, and the highest score corresponds to the red severe threat. At the same time, corresponding warning suggestions and disposal measure suggestions are generated according to the anomaly type and risk level.

[0040] In a specific embodiment, the process of inputting the campus personnel behavior time series feature vector into the cascaded spatial memory unit for hierarchical campus behavior pattern matching processing may specifically include the following steps: Perform time window segmentation processing on the campus personnel behavior time series feature vector to obtain teaching period behavior feature sub-vectors, break period behavior feature sub-vectors, and non-teaching period behavior feature sub-vectors; Input the teaching period behavior feature sub-vector into the teaching behavior pattern library of the cascaded spatial memory unit for similarity calculation processing to obtain the first similarity matching values with standard classroom behaviors, laboratory behaviors, and library behaviors; Input the break period behavior feature sub-vector into the break period behavior pattern library of the cascaded spatial memory unit for similarity calculation processing to obtain the second similarity matching values with corridor passing behaviors, cafeteria dining behaviors, and playground activity behaviors; Input the non-teaching period behavior feature sub-vector into the non-teaching period behavior pattern library of the cascaded spatial memory unit for similarity calculation processing to obtain the third similarity matching values with dormitory area behaviors, campus patrol behaviors, and facility maintenance behaviors, and perform time series fusion processing based on the first similarity matching values, the second similarity matching values, and the third similarity matching values to obtain the similarity matching results in three time dimensions of the teaching period, the break period, and the non-teaching period.

[0041] Specifically, the time window segmentation process divides the 256-dimensional campus personnel behavior time series feature vector along the time dimension based on the campus schedule. The segmentation algorithm divides the complete behavior feature vector into three independent sub-vectors according to the preset time boundaries. The teaching period usually corresponds to the formal class time in the morning and afternoon. The break period includes transition times such as breaks and lunch breaks. The non-teaching period covers free activity times in the morning, evening, and night. The segmentation process locates the feature components corresponding to different periods in the feature vector through timestamp indexing. Each sub-vector maintains the dimensional structure of the original features but only contains the behavior information of a specific period. The teaching period behavior feature sub-vector encodes the personnel behavior patterns in the classroom environment. The break period behavior feature sub-vector reflects the activity characteristics in the transition state. The non-teaching period behavior feature sub-vector describes the behavior rules during free time. The teaching behavior pattern library performs similarity calculation processing to maintain the behavior prototypes of three typical teaching scenarios: standard classroom behavior, laboratory behavior, and library behavior. The standard classroom behavior prototype includes feature patterns such as students sitting quietly listening, raising hands to speak, and teachers lecturing. The laboratory behavior prototype describes the behavior characteristics of students operating experimental equipment in groups and teachers guiding experiments. The library behavior prototype records activity patterns such as people reading quietly, whispering, and searching for books. The similarity calculation uses the vector inner product operation to measure the matching degree between the input teaching period behavior feature sub-vector and each behavior prototype. In the calculation process, each dimensional component of the sub-vector is multiplied by the corresponding component of the prototype vector and then summed to obtain the similarity score. The higher the score, the closer the behavior pattern is to the prototype. By comparing the similarity scores of the three prototypes, the most matching teaching behavior type is determined and the first similarity matching value is output.

[0042] The break behavior pattern library performs similarity calculation processing to store three typical activity patterns during the break period: corridor passage behavior, cafeteria dining behavior, and playground activity behavior. The corridor passage behavior prototype records features such as the normal walking speed, direction changes, and staying time of people in the corridor. The cafeteria dining behavior prototype includes the behavior sequence features of queuing, picking up food and moving, and sitting down to eat. The playground activity behavior prototype describes exercise behaviors such as physical exercise, leisure activities, and group games. The calculation process matches the break period behavior feature sub-vector with the three break behavior prototypes and evaluates the compliance of the behavior features with each prototype through the vector similarity measurement method. The similarity calculation result reflects the consistency between the observed break behavior and the standard break activity pattern, and the second similarity matching value containing the similarity scores of the three prototypes is output.

[0043] The non-teaching behavior pattern library, through similarity calculations, includes three types of behavior patterns during non-teaching periods: dormitory area behavior, campus patrol behavior, and facility maintenance behavior. The dormitory area behavior prototype records student daily behavior characteristics within the dormitory building, including entering and exiting the dormitory, activities within the building, and resting. The campus patrol behavior prototype describes the security personnel's regular patrol routes, inspection frequency, and emergency response behavior patterns. The facility maintenance behavior prototype includes the work behavior characteristics of logistics personnel such as cleaners and maintenance workers, such as equipment inspection, cleaning, and maintenance operations. The similarity calculation matches and analyzes the non-teaching period behavior feature subvectors with the three non-teaching behavior prototypes. The calculation process evaluates the similarity between the observed behavior and standard non-teaching activities. The matching result outputs a third similarity match value, which contains the similarity scores with the three non-teaching behavior prototypes.

[0044] The temporal fusion process comprehensively analyzes the first, second, and third similarity matching values. The fusion algorithm considers the interrelationships and temporal continuity constraints of the similarity results across the three time periods, combining the similarity scores of different time periods into a comprehensive evaluation result using a weighted average method. The weight assignment is based on the time period type and the temporal characteristics of the behavior at the current moment. Behavior within the teaching period relies more on the matching results of the teaching behavior pattern library, while behavior during recess mainly refers to the output of the recess behavior pattern library. The fusion process also considers the behavioral transition characteristics between adjacent time periods. Under normal circumstances, recess behavior follows the teaching behavior in time. Under abnormal circumstances, the behavioral patterns of different time periods conflict or are discontinuous. The final output is a matching result that includes a comprehensive similarity evaluation of the three time dimensions of teaching period, recess period, and non-teaching period.

[0045] In a specific embodiment, the process of executing step S104 may specifically include the following steps: Based on the campus functional area segmentation map, campus personnel identity detection results, and campus abnormal behavior recognition results, reverse verification processing is performed. By cross-validating the detected abnormal behavior with the rationality of the functional area where it is located, a campus abnormal behavior screening result with spatial semantic correction is obtained. Based on the results of campus abnormal behavior screening, time series correlation mining is performed. By analyzing the changing patterns of personnel behavior within the previous and next time windows, a chain of abnormal campus events with causal relationships is obtained. Conduct group behavior impact assessment on the chain of abnormal events on campus, and calculate the potential impact radius and impact intensity of a single abnormal behavior on the surrounding population to obtain the campus group safety risk diffusion assessment results; Preventive intervention strategies are generated based on the results of campus group safety risk diffusion assessment. By predicting the development trajectory and impact range of abnormal events, campus safety monitoring decision-making results are obtained, including early warning opportunities, types of intervention measures, and resource allocation recommendations.

[0046] Specifically, the reverse verification process verifies the rationality of the abnormal behavior detection results based on the spatial semantic constraint mechanism. The verification algorithm first extracts the spatial coordinate information of each abnormal event in the campus abnormal behavior recognition results, and then searches for the area labels at the corresponding positions in the campus functional area segmentation map to determine the specific functional area type where the abnormal behavior occurs. The cross-verification process judges the rationality degree of a specific abnormal behavior in a specific area by querying the pre-constructed functional area-behavior rationality mapping table. The mapping table records the behavior types allowed and prohibited in different functional areas at different time periods. For example, loud noise or intense exercise is not allowed in the teaching area during class time, and intense exercise is allowed in the sports area during physical education classes but restricted at other times. The verification algorithm matches the detected abnormal behavior types with the behavior constraint rules in their respective areas. When an abnormal behavior conflicts with the area constraint rules, the abnormal mark is retained. When an abnormal behavior conforms to the normal activity rules of the area, it is reclassified as a normal behavior. The verification process outputs the abnormal behavior screening results corrected by spatial semantics, effectively filtering out false detection abnormalities caused by insufficient understanding of area semantics.

[0047] The temporal correlation mining process identifies the time-dependent relationships between abnormal events through the sliding time window analysis technique. The mining algorithm constructs a time series data structure based on the campus abnormal behavior screening results, recording the occurrence time, duration, and event type information of each abnormal event. The correlation analysis uses the front and back time window scanning method. A fixed-size time window is set for each abnormal event, and the occurrence of other abnormal events is searched within this window. The degree of association between events is judged by calculating the time interval and spatial distance between events. Causal relationship identification is based on the time sequence and behavioral logic relationship of events. When the occurrence of a previous abnormal event creates conditions or triggers a chain reaction for subsequent abnormal events, the algorithm connects these events to form an abnormal event chain. The chain structure includes event nodes, time edges, and causal relationship labels. Each chain represents a complete development process of an abnormal event. The mining process outputs the correlation analysis results containing multiple abnormal event chains.

[0048] The evaluation process of the impact of group behavior calculates the potential impact range and intensity of a single abnormal behavior on the surrounding crowd through a spatial impact model. The evaluation algorithm first determines the spatial center point coordinates of the abnormal event, and then sets the influence radius parameter based on the type and intensity of the abnormal behavior. The influence radius of minor disciplinary violations is relatively small. The calculation of the influence intensity considers factors such as the danger level of the abnormal behavior, the crowd density, and the propagation speed. The algorithm establishes an influence intensity attenuation model by analyzing the diffusion law of historical similar events. The influence intensity gradually attenuates with the increase of distance and is also affected by physical obstacles and the direction of crowd flow. The evaluation process also considers group psychological factors and the herd effect. When the abnormal behavior causes people to gather and watch or imitate, its influence range will be amplified. The algorithm judges the intensity of the group reaction by monitoring the behavior change patterns of the people around the abnormal event, and finally outputs the evaluation result of the group safety risk diffusion, including the influence radius, the influence intensity distribution, and the potential diffusion path.

[0049] The generation process of preventive intervention strategies formulates proactive intervention plans based on the development trajectory prediction model of abnormal events. The strategy generation algorithm analyzes the risk propagation pattern and influence range in the evaluation result of campus group safety risk diffusion, and generates targeted intervention measures in combination with the disposal experience of historical abnormal events and the requirements of campus emergency plans. The determination of the early warning timing calculates the optimal intervention time point through risk threshold monitoring and trend prediction algorithms. When the risk diffusion evaluation shows that the abnormal event has a further deteriorating trend, the early warning mechanism is triggered. The selection of the early warning timing balances the relationship between the intervention effect and resource consumption. The types of intervention measures are classified according to the nature and severity of the abnormal event. For minor abnormalities, on-site reminders and guidance measures are adopted. For medium abnormalities, security personnel need to intervene and conduct on-site control. For serious abnormalities, it is required to activate the emergency plan and conduct joint disposal by multiple departments. The resource allocation suggestion generates an optimized allocation plan based on the requirements of the intervention measures and the status of available campus resources. The algorithm considers the time cost and spatial distance of personnel scheduling, and preferentially arranges the security personnel closest to the incident site to respond. At the same time, reserve backup resources to cope with potential chain reactions.

[0050] In a specific embodiment, the process of performing the reverse verification process based on the campus functional area segmentation map, the campus personnel identity detection result, and the campus abnormal behavior recognition result may specifically include the following steps: Extract the spatial position coordinates of the abnormal behavior according to the campus abnormal behavior recognition result for area attribution determination processing, and obtain the functional area identifier to which the abnormal behavior belongs; Query the campus functional area reasonable behavior database based on the functional area identifier for behavior matching verification processing, and obtain the verification benchmarks of the allowed behavior types and prohibited behavior types within the corresponding area; Perform a logical comparison between the abnormal behavior recognition result and the verification benchmark to obtain a behavior classification label for distinguishing true abnormalities and misdetected abnormalities; Filter and process the recognition results of campus abnormal behaviors according to the behavior classification tags to obtain the screening results of campus abnormal behaviors corrected by spatial semantics.

[0051] Specifically, the extraction of the spatial location coordinates of abnormal behaviors obtains the precise spatial location of abnormal events by parsing the bounding box information in the recognition results of campus abnormal behaviors. The extraction algorithm reads the bounding box coordinate parameters of each abnormal event from the abnormal behavior detection data and calculates the geometric center point of the bounding box as the representative spatial location coordinates of the abnormal behavior. The determination process of regional attribution performs spatial matching between the extracted spatial coordinates and the campus functional area segmentation map. The determination algorithm searches for the regional label value of the corresponding pixel point in the segmentation map according to the coordinate position. The label value directly corresponds to one of the functional area types in the teaching area, living area, sports area, management area, or traffic area, and determines the specific functional area identifier to which the abnormal behavior belongs through the pixel value mapping relationship.

[0052] The query process of the reasonable behavior database for campus functional areas retrieves the behavior constraint rules for the corresponding area based on the obtained functional area identifier. The database stores the behavior specification information for each area at different time periods with the functional area identifier as the index key. The query algorithm locates the corresponding behavior constraint record according to the area identifier and the current timestamp. The behavior matching and verification process extracts the list of allowed behavior types and the list of prohibited behavior types from the database record. The allowed behavior types record the activity patterns permitted under normal circumstances in this area, and the prohibited behavior types list the behavior patterns clearly prohibited in this area. The verification process outputs a verification reference data structure containing the two behavior type lists.

[0053] The logical comparison process matches and compares the behavior category information in the recognition results of abnormal behaviors with the behavior type lists in the verification reference. The comparison algorithm first checks whether the abnormal behavior category exists in the list of prohibited behavior types. If the match is successful, it is confirmed that the behavior belongs to a real abnormality. If there is no match, it further checks whether the behavior belongs to the list of allowed behavior types. The detection results belonging to allowed behaviors are marked as false positive abnormalities. The comparison process also considers the influence of time factors. The rationality judgment criteria for the same behavior are different at different time periods. The comparison algorithm adjusts the judgment criteria according to the specific time when the abnormal event occurs, and finally outputs a behavior classification result containing two classification labels: real abnormality and false positive abnormality.

[0054] The screening and filtering process performs data cleaning on the original recognition results of campus abnormal behaviors according to the behavior classification tags. The filtering algorithm traverses all abnormal behavior detection records, retains the detection results marked as true abnormalities, and deletes the records marked as false positives. The filtering process also includes confidence adjustment operations. For abnormal behaviors occurring near the region boundary, the algorithm reduces their confidence scores to reflect the uncertainty of the spatial position. For abnormal behaviors occurring at the center of the region and clearly violating the regional behavior norms, the algorithm increases their confidence scores to enhance the reliability of the detection results. The screening and filtering process outputs the screening results of abnormal behaviors corrected by spatial semantics.

[0055] To illustrate the complete data processing flow, when a violent fight behavior is detected among people in the sports area, the coordinate extraction algorithm calculates the central coordinate position of the fight event from the bounding box information of the detection result. The regional attribution determination determines the sports area identifier corresponding to this coordinate by querying the functional area segmentation map of the campus. The database query processing retrieves relevant behavior constraint rules based on the sports area identifier and the current time period, and finds that violent exercise behaviors are allowed in the sports area during physical education classes but conflict behaviors are prohibited. The verification benchmark includes detailed allowed and prohibited behavior classifications. The logical comparison processing analyzes the detected behavior category and finds that this behavior belongs to the conflict type and there is a matching item in the prohibited behavior list. The comparison result marks this behavior as a true abnormality. The screening and filtering process retains this detection record of the true abnormality. At the same time, since this abnormality occurs at the center of the sports area and clearly violates the regional behavior norms, the algorithm increases the confidence score of this detection result. Finally, a corrected result containing a high-confidence true abnormality mark is output, solving the technical problem that the prior art cannot distinguish normal exercise behaviors from abnormal behaviors.

[0056] The above describes the intelligent campus security monitoring method based on visual understanding in the embodiments of the present application. Next, the intelligent campus security monitoring system based on visual understanding in the embodiments of the present application will be described. Please refer to Figure 2 , an embodiment of the intelligent campus security monitoring system based on visual understanding in the embodiments of the present application includes: A segmentation module 201, configured to perform adaptive semantic segmentation processing on the campus scene image through the prior knowledge of campus functional areas to obtain a campus functional area segmentation map; An extraction module 202, configured to perform multi-dimensional identity feature extraction processing on campus personnel according to the campus functional area segmentation map to obtain campus personnel identity detection results; An identification module 203, configured to input the campus personnel identity detection results into a campus behavior semantic perception network for abnormal behavior pattern recognition processing to obtain campus abnormal behavior recognition results; An evaluation module 204, configured to perform comprehensive evaluation processing through a campus scene context awareness fusion algorithm based on the campus functional area segmentation map, the campus personnel identity detection result, and the campus abnormal behavior recognition result, so as to obtain a campus security monitoring decision result.

[0057] above Figure 2 The intelligent campus security monitoring system based on visual understanding in the embodiments of the present invention is described in detail from the perspective of modular functional entities. Next, the intelligent campus security monitoring device based on visual understanding in the embodiments of the present invention is described in detail from the perspective of hardware processing.

[0058] Refer to Figure 3 , an intelligent campus security monitoring device based on visual understanding is further provided in the embodiments of the present invention. The intelligent campus security monitoring device based on visual understanding may be a server, and its internal structure may be as Figure 3 shown. The intelligent campus security monitoring device based on visual understanding includes a processor, a memory, a display screen, an input device, a network interface, and a database connected through a system bus. Among them, the processor of this computer design is used to provide computing and control capabilities. The memory of the intelligent campus security monitoring device based on visual understanding includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the intelligent campus security monitoring device based on visual understanding is used to store the corresponding data in this embodiment. The network interface of the intelligent campus security monitoring device based on visual understanding is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the above method is implemented.

[0059] Those skilled in the art can understand that Figure 3 the structure shown in is only a block diagram of a part of the structure related to the solution of the present invention, and does not constitute a limitation on the intelligent campus security monitoring device based on visual understanding to which the solution of the present invention is applied.

[0060] The present invention also provides a computer-readable storage medium. The computer-readable storage medium may be a non-volatile computer-readable storage medium, or the computer-readable storage medium may also be a volatile computer-readable storage medium. Instructions are stored in the computer-readable storage medium. When the instructions run on a computer, the computer is caused to execute the steps of the intelligent campus security monitoring method based on visual understanding.

[0061] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the above-described systems, systems, and units can refer to the corresponding processes in the foregoing method embodiments, and will not be elaborated herein.

[0062] When the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a smart campus security monitoring device based on visual understanding (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical discs that can store program codes.

[0063] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of various embodiments of the present invention.

Claims

1. A smart campus security monitoring method based on visual understanding, characterized in that, The method includes: Performing adaptive semantic segmentation processing on the campus scene image through the prior knowledge of campus functional areas to obtain a campus functional area segmentation map; Performing multi-dimensional identity feature extraction processing on campus personnel according to the campus functional area segmentation map to obtain a campus personnel identity detection result; Inputting the campus personnel identity detection result into the campus behavior semantic perception network for abnormal behavior pattern recognition processing to obtain a campus abnormal behavior recognition result; Performing comprehensive evaluation processing through the campus scene context perception fusion algorithm according to the campus functional area segmentation map, the campus personnel identity detection result, and the campus abnormal behavior recognition result to obtain a campus security monitoring decision result.

2. The intelligent campus security monitoring method based on visual understanding according to claim 1, wherein, The performing adaptive semantic segmentation processing on the campus scene image through the prior knowledge of campus functional areas to obtain a campus functional area segmentation map includes: Performing campus building geometric feature extraction processing on the campus scene image to obtain campus spatial structure information including building contour features, road direction features, and greening layout features; Inputting the campus spatial structure information into an improved SegNet network for encoding processing to obtain a campus area basic feature map; Performing differential decoding processing on the campus area basic feature map based on the campus functional area attention mechanism to obtain an enhanced feature map integrating campus prior knowledge; Obtaining a campus functional area segmentation map including five functional area labels of teaching area, living area, sports area, management area, and traffic area through pixel-level classification processing according to the enhanced feature map.

3. The intelligent campus security monitoring method based on visual understanding according to claim 1, wherein The performing multi-dimensional identity feature extraction processing on campus personnel according to the campus functional area segmentation map to obtain a campus personnel identity detection result includes: Performing three-branch feature extraction processing of face features, dressing pattern features, and behavior posture features on the campus personnel image based on the campus functional area segmentation map to obtain a 128-dimensional face embedding vector, a 64-dimensional dressing feature vector, and 17 key point spatial relationship features; Inputting the face embedding vector, the dressing feature vector, and the key point spatial relationship features into a gated fusion mechanism for multi-dimensional feature fusion processing to obtain a campus personnel comprehensive identity feature vector; Performing identity category recognition processing through a campus identity classifier according to the campus personnel comprehensive identity feature vector to obtain an identity classification result including on-campus students, faculty members, security personnel, cleaners, visitors, couriers, maintenance workers, and suspicious personnel; Performing regional rationality verification processing on the identity classification result based on the campus functional area segmentation map to obtain a campus personnel identity detection result including identity category, confidence level, and regional location information.

4. The intelligent campus security monitoring method based on visual understanding according to claim 1, characterized in that, The inputting the campus personnel identity detection result into the campus behavior semantic perception network for abnormal behavior pattern recognition processing to obtain a campus abnormal behavior recognition result includes: Inputting the campus personnel identity detection result into the campus behavior semantic perception network for personnel trajectory sequence extraction processing to obtain campus personnel behavior trajectory data including position coordinates and timestamps; Based on the campus personnel behavior trajectory data, multi-scale behavior pattern encoding processing is performed through a cascaded time series coding unit to obtain a 256-dimensional campus personnel behavior time series feature vector that integrates short-term behavior characteristics and long-term behavior trends; Inputting the temporal feature vector of campus personnel behavior into the cascaded spatial memory unit for hierarchical campus behavior pattern matching processing, and obtaining similarity matching results in three time dimensions: teaching period, break period, and non-teaching period; Based on the similarity matching results, a cascade abnormal behavior discrimination process is performed using a campus scene adaptive contrast learning loss function to obtain a campus-specific abnormal behavior classification result that distinguishes between classroom disruption and illegal intrusion; Based on the classification results of the specific abnormal behaviors on campus, a multi-dimensional abnormality degree quantification process is performed through a cascade risk assessment unit to obtain an abnormal behavior score value that integrates the dangerousness and urgency of the behavior; According to the abnormal behavior score value, a cascade result integration process is performed through the campus security level mapping algorithm to obtain a campus abnormal behavior identification result including campus specific abnormal type, risk level and early warning suggestion.

5. The intelligent campus security monitoring method based on visual understanding according to claim 4, wherein The campus personnel behavior time series feature vector is input into the cascaded spatial memory unit for hierarchical campus behavior pattern matching processing to obtain similarity matching results in three time dimensions: teaching period, break period, and non-teaching period, including: Performing time window segmentation processing on the campus personnel behavior time series feature vector to obtain teaching period behavior feature sub-vectors, break period behavior feature sub-vectors, and non-teaching period behavior feature sub-vectors; Inputting the teaching period behavior feature subvector into the teaching behavior pattern library of the cascaded spatial memory unit for similarity calculation processing to obtain a first similarity matching value with the standard classroom behavior, laboratory behavior, and library behavior; Inputting the inter-class period behavior feature subvector into the inter-class behavior pattern library of the cascaded spatial memory unit for similarity calculation processing to obtain a second similarity matching value with the corridor passage behavior, the cafeteria dining behavior, and the playground activity behavior; The non-teaching period behavior feature sub-vector is input into the non-teaching behavior pattern library of the cascaded spatial memory unit for similarity calculation processing to obtain a third similarity matching value with the dormitory area behavior, campus patrol behavior and facility maintenance behavior, and based on the first similarity matching value, the second similarity matching value and the third similarity matching value, a time series fusion processing is performed to obtain similarity matching results in the three time dimensions of teaching period, break period and non-teaching period.

6. The method for intelligent campus security monitoring based on visual understanding according to claim 1, wherein, The campus functional area segmentation map, campus personnel identity detection results, and campus abnormal behavior recognition results are comprehensively evaluated and processed using a campus scene context perception fusion algorithm to obtain a campus security monitoring decision result, including: Based on the campus functional area segmentation map, campus personnel identity detection results, and campus abnormal behavior recognition results, reverse verification processing is performed to obtain campus abnormal behavior screening results that have been spatially semantically corrected by cross-validating the detected abnormal behavior with the rationality of the functional area in which it is located; Perform temporal correlation mining processing based on the screened results of campus abnormal behaviors. By analyzing the change patterns of personnel behaviors within the front and back time windows, obtain a chain of campus abnormal events with causal relationships; Perform group behavior impact assessment processing on the chain of campus abnormal events. By calculating the potential influence radius and influence intensity of a single abnormal behavior on the surrounding crowd, obtain the assessment result of the spread of campus group safety risks; Generate preventive intervention strategies based on the assessment result of the spread of campus group safety risks. By predicting the development trajectory and influence scope of abnormal events, obtain the campus safety monitoring decision result including the early warning timing, intervention measure types, and resource allocation suggestions.

7. The intelligent campus security monitoring method based on visual understanding according to claim 6, characterized in that Perform reverse verification processing based on the campus functional area segmentation map, campus personnel identity detection results, and campus abnormal behavior recognition results. By cross-verifying the detected abnormal behaviors with the rationality of their corresponding functional areas, obtain the screened results of campus abnormal behaviors corrected by spatial semantics, including: Extract the spatial position coordinates of abnormal behaviors according to the campus abnormal behavior recognition results and perform regional attribution determination processing to obtain the functional area identifier to which the abnormal behaviors belong; Query the reasonable behavior database of campus functional areas based on the functional area identifier and perform behavior matching verification processing to obtain the verification benchmarks for the allowed behavior types and prohibited behavior types within the corresponding area; Perform logical comparison processing on the campus abnormal behavior recognition results and the verification benchmarks to obtain behavior classification labels for distinguishing real abnormalities and false detections; Perform screening and filtering processing on the campus abnormal behavior recognition results according to the behavior classification labels to obtain the screened results of campus abnormal behaviors corrected by spatial semantics.

8. An intelligent campus security monitoring system based on visual understanding, characterized in that, For implementing the intelligent campus safety monitoring method based on visual understanding as described in any one of claims 1-7, the intelligent campus safety monitoring system based on visual understanding includes: A segmentation module for performing adaptive semantic segmentation processing on campus scene images through prior knowledge of campus functional areas to obtain a campus functional area segmentation map; An extraction module for performing multi-dimensional identity feature extraction processing on campus personnel according to the campus functional area segmentation map to obtain campus personnel identity detection results; A recognition module for inputting the campus personnel identity detection results into a campus behavior semantic perception network for abnormal behavior pattern recognition processing to obtain campus abnormal behavior recognition results; An evaluation module for performing comprehensive evaluation processing through a campus scene context awareness fusion algorithm based on the campus functional area segmentation map, campus personnel identity detection results, and campus abnormal behavior recognition results to obtain a campus safety monitoring decision result.

9. An intelligent campus security monitoring device based on visual understanding, characterized in that, It includes a memory and a processor. The memory stores a computer program that can run on the processor. When the processor executes the computer program, it implements the intelligent campus safety monitoring method based on visual understanding as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program runs on the processor, it causes the processor to execute the intelligent campus safety monitoring method based on visual understanding as described in any one of claims 1 to 7.

Images